github/sql-server-samples
1
0
Fork 0
mirror of https://github.com/Microsoft/sql-server-samples.git synced 2025-12-08 14:58:54 +00:00
Code Issues Releases Wiki Activity

Added readme file

Browse Source
This commit is contained in:
Alexander Nosov
2020-12-08 12:49:11 -08:00
parent d5b403a2b5
commit 91c2f1d81f
3 changed files with 25 additions and 214 deletions
Download Patch File Download Diff File Expand all files Collapse all files
samples/databases/wide-world-importers/sample-scripts/polybase/DemonstratePolybase.sql
-139
View File
@@ -1,139 +0,0 @@
-- Demonstrate WorldWideImporters Polybase connections
-- Requires PolyBase to be installed.
USE WideWorldImportersDW;
GO
-- WideWorldImporters have customers in a variety of cities but feel they are likely missing
-- other important cities. They have decided to try to find other cities have a growth rate of more
-- than 20% over the last 3 years, and where they do not have existing customers.
-- They have obtained census data (a CSV file) and have loaded it into an Azure storage account.
-- They want to combine that data with other data in their main OLTP database to work out where
-- they should try to find new customers.
-- First, let's apply Polybase connectivity and set up an external table to point to the data
-- in the Azure storage account.
EXEC [Application].Configuration_ApplyPolybase;
GO
-- In Object Explorer, refresh the WideWorldImporters database, then expand the Tables node.
-- Note that SQL Server 2016 added a new entry here for External Tables. Expand that node.
-- Expand the dbo.CityPopulationStatistics table, expand the list of columns and note the
-- values that are contained. Let's look at the data:
SELECT CityID, StateProvinceCode, CityName, YearNumber, LatestRecordedPopulation FROM dbo.CityPopulationStatistics;
GO
-- How did that work? First the procedure created an external data source like this:
/*
CREATE EXTERNAL DATA SOURCE AzureStorage
WITH
(
TYPE=HADOOP, LOCATION = 'wasbs://data@sqldwdatasets.blob.core.windows.net'
);
*/
-- This shows how to connect to AzureStorage. Next the procedure created an
-- external file format to describe the layout of the CSV file:
/*
CREATE EXTERNAL FILE FORMAT CommaDelimitedTextFileFormat
WITH
(
FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS
(
FIELD_TERMINATOR = ','
)
);
*/
-- Finally the external table was defined like this:
/*
CREATE EXTERNAL TABLE dbo.CityPopulationStatistics
(
CityID int NOT NULL,
StateProvinceCode nvarchar(5) NOT NULL,
CityName nvarchar(50) NOT NULL,
YearNumber int NOT NULL,
LatestRecordedPopulation bigint NULL
)
WITH
(
LOCATION = '/',
DATA_SOURCE = AzureStorage,
FILE_FORMAT = CommaDelimitedTextFileFormat,
REJECT_TYPE = VALUE,
REJECT_VALUE = 4 -- skipping 1 header row per file
);
*/
-- From that point onwards, the external table can be used like a local table. Let's run that
-- query that they wanted to use to find out which cities they should be finding new customers
-- in. We'll start building the query by grouping the cities from the external table
-- and finding those with more than a 20% growth rate for the period:
WITH PotentialCities
AS
(
SELECT cps.CityName,
cps.StateProvinceCode,
MAX(cps.LatestRecordedPopulation) AS PopulationIn2016,
(MAX(cps.LatestRecordedPopulation) - MIN(cps.LatestRecordedPopulation)) * 100.0
/ MIN(cps.LatestRecordedPopulation) AS GrowthRate
FROM dbo.CityPopulationStatistics AS cps
WHERE cps.LatestRecordedPopulation IS NOT NULL
AND cps.LatestRecordedPopulation <> 0
GROUP BY cps.CityName, cps.StateProvinceCode
)
SELECT CityName, StateProvinceCode, PopulationIn2016, GrowthRate
FROM PotentialCities
WHERE GrowthRate > 2.0;
GO
-- Now let's combine that with our local city and sales data to exclude those where we already
-- have customers. We'll find the 100 most interesting cities based upon population.
WITH PotentialCities
AS
(
SELECT cps.CityName,
cps.StateProvinceCode,
MAX(cps.LatestRecordedPopulation) AS PopulationIn2016,
(MAX(cps.LatestRecordedPopulation) - MIN(cps.LatestRecordedPopulation)) * 100.0
/ MIN(cps.LatestRecordedPopulation) AS GrowthRate
FROM dbo.CityPopulationStatistics AS cps
WHERE cps.LatestRecordedPopulation IS NOT NULL
AND cps.LatestRecordedPopulation <> 0
GROUP BY cps.CityName, cps.StateProvinceCode
),
InterestingCities
AS
(
SELECT DISTINCT pc.CityName,
pc.StateProvinceCode,
pc.PopulationIn2016,
FLOOR(pc.GrowthRate) AS GrowthRate
FROM PotentialCities AS pc
INNER JOIN Dimension.City AS c
ON pc.CityName = c.City
WHERE GrowthRate > 2.0
AND NOT EXISTS (SELECT 1 FROM Fact.Sale AS s WHERE s.[City Key] = c.[City Key])
)
SELECT TOP(100) CityName, StateProvinceCode, PopulationIn2016, GrowthRate
FROM InterestingCities
ORDER BY PopulationIn2016 DESC;
GO
-- Clean up if required
/*
DROP EXTERNAL TABLE dbo.CityPopulationStatistics;
GO
DROP EXTERNAL FILE FORMAT CommaDelimitedTextFileFormat;
GO
DROP EXTERNAL DATA SOURCE AzureStorage;
GO
*/
samples/databases/wide-world-importers/sample-scripts/polybase/README.md
-75
View File
@@ -1,75 +0,0 @@
# Sample Querying of External Data Source in WideWorldImportersDW
This script demonstrates the use of PolyBase to query an external data source.
Demographics data is available in Azure blob storage. This data is joined with sales data recorded in the local database to determine which would be good candidates for future expansion of the business.
### Contents
[About this sample](#about-this-sample)<br/>
[Before you begin](#before-you-begin)<br/>
[Running the sample](#run-this-sample)<br/>
[Sample details](#sample-details)<br/>
[Disclaimers](#disclaimers)<br/>
[Related links](#related-links)<br/>
<a name=about-this-sample></a>
## About this sample
<!-- Delete the ones that don't apply -->
1. **Applies to:** SQL Server 2016 (or higher), Azure SQL Database
1. **Key features:** PolyBase
1. **Workload:** Analytics
1. **Programming Language:** T-SQL
1. **Authors:** Greg Low, Jos de Bruijn
1. **Update history:** 26 May 2016 - initial revision
<a name=before-you-begin></a>
## Before you begin
To run this sample, you need the following prerequisites.
**Software prerequisites:**
<!-- Examples -->
1. SQL Server 2016 (or higher) with PolyBase, connected to the internet.
2. SQL Server Management Studio
3. The WideWorldImportersDW database (Full version).
<a name=run-this-sample></a>
## Running the sample
1. Execute the sample script.
2. Inspect external tables in the database.
3. Review query results.
## Sample details
The sample script performs a configuration and runs three queries:
1. An external table `dbo.CitePopulationStatistics` is created in the database, pointing to a data set in Azure blob storage.
2. The data in Azure storage is queried through Transact-SQL, showing all the data in the data source.
3. Cities with a significant growth rate (>= 20%) are identified.
4. Top cities for potential expansion are identified based on external data as well as sales data in the local database.
<a name=disclaimers></a>
## Disclaimers
The code included in this sample is not intended to be used for production purposes.
<a name=related-links></a>
## Related Links
<!-- Links to more articles. Remember to delete "en-us" from the link path. -->
For more information, see these articles:
- [Get started with PolyBase](https://msdn.microsoft.com/library/mt163689.aspx)
- [PolyBase: Gaining insights from HDFS and relational data in SQL Server 2016 (video)](https://channel9.msdn.com/Events/DataDriven/SQLServer2016/PolyBase)