github/sql-server-samples
1
0
Fork 0
mirror of https://github.com/Microsoft/sql-server-samples.git synced 2025-12-08 14:58:54 +00:00
Code Issues Releases Wiki Activity
Files
ae1073d6f6c5c3fecb8332338ada54cb3c316922
sql-server-samples/samples/databases/wide-world-importers/wwi-integration-etl/README.md
Jos de Bruijn ae1073d6f6 added image
2016-05-25 14:49:58 -07:00

2.6 KiB
Raw Blame History

ETL Process for WideWorldImporters

This SSIS project performs ETL from the transactional database WideWorldImporters into the OLAP database WideWorldImportersDW for long-term storage and analytics.

Contents

About this sample
Before you begin
Running the sample
Sample details
Disclaimers
Related links

About this sample

  1. Applies to: SQL Server 2016 (or higher)
  2. Key features: Core database features
  3. Workload: ETL
  4. Programming Language:
  5. Authors: Greg Low, Jos de Bruijn
  6. Update history: 25 May 2016 - initial revision

Before you begin

To run this sample, you need the following prerequisites.

Software prerequisites:

  1. SQL Server 2016 (or higher).
  2. Visual Studio 2015.
  3. SQL Server 2016 Integration Services. This needs to be installed on the same machine as Visual Studio to be able to build the project.

Running the sample

  1. Open the solution file WWI_Integration.sln in Visual Studio.

  2. Build the solution.

  3. TBD: running the ETL process

Sample details

The ETL package WWI_Integration is used to migrate data from the WideWorldImporters database to the WideWorldImportersDW database as the data changes. The package is run periodically (most commonly daily).

The design of the package uses SSIS to orchestrate bulk T-SQL operations (rather than as separate transformations within SSIS) to ensure high performance.

Dimensions are loaded first, followed by Fact tables. The package can be re-run at any time after a failure.

The workflow is as follows:

Alt text

It starts with an expression task that works out the appropriate cutoff time. This time is the current time less a few seconds. (This is more robust than requesting data right to the current time). It then truncates any milliseconds from the time.

The main processing starts by populating the Date dimension table. It ensures that all dates for the current year have been populated in the table.

After this, a series of data flow tasks loads each dimension, then each fact.

Disclaimers

The code included in this sample is not intended to be used for production purposes.

Related Links

TBD For more information, see these articles: