mirror of
https://github.com/Microsoft/sql-server-samples.git
synced 2025-12-08 14:58:54 +00:00
adding etl docs
This commit is contained in:
@@ -1,7 +1,58 @@
|
||||
# WideWorldImporters ETL Workflow
|
||||
|
||||
This folder contains documentation for the sample.
|
||||
The ETL package WWI_Integration is used to migrate data from the WideWorldImporters database to the WideWorldImportersDW database as the data changes. The package is run periodically (most commonly daily).
|
||||
|
||||
Start with [root.md](root.md)
|
||||
## Overview
|
||||
|
||||
Note that these contents will most likely be migrated to MSDN.
|
||||
The design of the package uses SQL Server Integration Services (SSIS) to orchestrate bulk T-SQL operations (rather than as separate transformations within SSIS) to ensure high performance.
|
||||
|
||||
Dimensions are loaded first, followed by Fact tables. The package can be re-run at any time after a failure.
|
||||
|
||||
The workflow is as follows:
|
||||
|
||||

|
||||
|
||||
It starts with an expression task that works out the appropriate cutoff time. This time is the current time less a few seconds. (This is more robust than requesting data right to the current time). It then truncates any milliseconds from the time.
|
||||
|
||||
The main processing starts by populating the Date dimension table. It ensures that all dates for the current year have been populated in the table.
|
||||
|
||||
After this, a series of data flow tasks loads each dimension, then each fact.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- SQL Server 2016 (or higher) with the databases WideWorldImporters and WideWorldImportersDW. These can be on the same or different instances of SQL Server.
|
||||
- SQL Server Management Studio (SSMS)
|
||||
- SQL Server 2016 Integration Services (SSIS).
|
||||
- Make sure you have created an SSIS Catalog. If not, right click **Integration Services** in SSMS Object Explorer, and choose **Add Catalog**. Follow the defaults. It will ask you to enable sqlclr and provide a password.
|
||||
|
||||
|
||||
## Download
|
||||
|
||||
The latest release of the sample:
|
||||
|
||||
[wide-world-importers-v0.1](https://github.com/Microsoft/sql-server-samples/releases/tag/wide-world-importers-v0.1)
|
||||
|
||||
Download the SSIS package file **Daily ETL.ispac**.
|
||||
|
||||
Source code to recreate the sample database is available from the following location.
|
||||
|
||||
[wide-world-importers](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/wwi-integration-etl)
|
||||
|
||||
## Install
|
||||
|
||||
1. Deploy the SSIS package.
|
||||
- Open the "Daily ETL.ispac" package from Windows Explorer. This will launch the Integration Services Deployment Wizard.
|
||||
- Under "Select Source" follow the default Project Deployment, with the path pointing to the "Daily ETL.ispac" package.
|
||||
- Under "Select Destination" enter the name of the server that hosts the SSIS catalog.
|
||||
- Select a path under the SSIS catalog, for example under a new folder "WideWorldImporters".
|
||||
- Finalize the wizard by clicking Deploy.
|
||||
|
||||
2. Create a SQL Server Agent job for the ETL process.
|
||||
- In SSMS, right-click "SQL Server Agent" and select New->Job.
|
||||
- Pick a name, for example "WideWorldImporters ETL".
|
||||
- Add a Job Step of type "SQL Server Integration Services Package".
|
||||
- Select the server with the SSIS catalog, and select the "Daily ETL" package.
|
||||
- Under Configuration->Connection Managers ensure the connections to the source and target are configured correctly. The default is to connect to the local instance.
|
||||
- Click OK to create the Job.
|
||||
|
||||
3. Execute or schedule the Job.
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
# WideWorldImportersDW OLAP Sample Queries
|
||||
|
||||
This folder contains documentation for the sample.
|
||||
Refer to the sample-scripts.zip file that is included with the release of the sample, or refer to the source code:
|
||||
|
||||
Start with [root.md](root.md)
|
||||
|
||||
Note that these contents will most likely be migrated to MSDN.
|
||||
[wide-world-importers/sample-scripts](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/sample-scripts)
|
||||
|
||||
Reference in New Issue
Block a user