diff --git a/samples/features/azure-arc-data-controller/README.md b/samples/features/azure-arc-data-controller/README.md index 8a68db32..6bbbf149 100644 --- a/samples/features/azure-arc-data-controller/README.md +++ b/samples/features/azure-arc-data-controller/README.md @@ -1,59 +1,4 @@ -# Azure Arc Data Controller clusters - -Installation instructions for SQL Server 2019 big data clusters can be found [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sql-server-ver15). +# Azure Arc Data Controller cluster ## Samples Setup - -**Before you begin**, load the sample data into your big data cluster. For instructions, see [Load sample data into a SQL Server 2019 big data cluster](https://docs.microsoft.com/en-us/sql/big-data-cluster/tutorial-load-sample-data). - -## Executing the sample scripts -The scripts should be executed in a specific order to test the various features. Execute the scripts from each folder in below order: - -1. __[spark/data-loading/transform-csv-files.ipynb](spark/data-loading/transform-csv-files.ipynb)__ -1. __[data-virtualization/generic-odbc](data-virtualization/generic-odbc)__ -1. __[data-virtualization/hadoop](data-virtualization/hadoop)__ -1. __[data-virtualization/storage-pool](data-virtualization/storage-pool)__ -1. __[data-virtualization/oracle](data-virtualization/oracle)__ -1. __[data-pool](data-pool/)__ -1. __[machine-learning/sql/r](machine-learning/sql/r)__ -1. __[machine-learning/sql/python](machine-learning/sql/python)__ - -## __[data-pool](data-pool/)__ - -SQL Server 2019 big data cluster contains a data pool which consists of many SQL Server instances to store data & query in a scale-out manner. - -### Data ingestion using Spark -The sample script [data-pool/data-ingestion-spark.sql](data-pool/data-ingestion-spark.sql) shows how to perform data ingestion from Spark into data pool table(s). - -### Data ingestion using sql -The sample script [data-pool/data-ingestion-sql.sql](data-pool/data-ingestion-sql.sql) shows how to perform data ingestion from T-SQL into data pool table(s). - -## __[data-virtualization](data-virtualization/)__ - -SQL Server 2019 or SQL Server 2019 big data cluster can use PolyBase external tables to connect to other data sources. - -### External table over Generic ODBC data source -The [data-virtualization/generic-odbc](data-virtualization/generic-odbc) folder contains samples that demonstrate how to query data in MySQL & PostgreSQL using external tables and generic ODBC data source. The generic ODBC data soruce can be used only in SQL Server 2019 on Windows. - -### External table over Hadoop -The [data-virtualization/hadoop](data-virtualization/hadoop) folder contains samples that demonstrate how to query data in HDFS using external tables. This demonstrates the functionality available from SQL Server 2016 using the HADOOP data source. - -### External table over Oracle -The [data-virtualization/oracle](data-virtualization/oracle) folder contains samples that demonstrate how to query data in Oracle using external tables. - -### External table over Storage Pool -SQL Server 2019 big data cluster contains a storage pool consisting of HDFS, Spark and SQL Server instances. The [data-virtualization/storage-pool](data-virtualization/storage-pool) folder contains samples that demonstrate how to query data in HDFS inside SQL Server 2019 big data cluster. - -## __[deployment](deployment/)__ - -The [deployment](deployment) folder contains the scripts for deploying a Kubernetes cluster for SQL Server 2019 big data cluster. - -## __[machine-learning](machine-learning/)__ - -SQL Server 2016 added support executing R scripts from T-SQL. SQL Server 2017 added support for executing Python scripts from T-SQL. SQL Server 2019 adds support for executing Java code from T-SQL. SQL Server 2019 big data cluster adds support for executing Spark code inside the big data cluster. - -### SQL Server Machine Learning Services -The [machine-learning\sql](machine-learning\sql) folder contains the sample SQL scripts that show how to invoke R, Python, and Java code from T-SQL. - -### Spark Machine Learning -The [machine-learning\spark](machine-learning\spark) folder contains the Spark samples. +Follow the instrutions here: https://raw.githubusercontent.com/ananto-msft/sql-server-samples/master/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md diff --git a/samples/features/azure-arc-data-controller/deployment/README.md b/samples/features/azure-arc-data-controller/deployment/README.md index fe5ec277..70ca9f19 100644 --- a/samples/features/azure-arc-data-controller/deployment/README.md +++ b/samples/features/azure-arc-data-controller/deployment/README.md @@ -1,16 +1,8 @@ -# Creating a Kubernetes cluster for SQL Server 2019 big data cluster +# Creating a Kubernetes cluster for Azure Arc Data Controller cluster -SQL Server 2019 big data cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes clusters using different environments. +Azure Arc Data Controller cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes clusters using different environments. ## __[Deploy a Kubernetes cluster using kubeadm](kubeadm/)__ -Use the scripts in the **kubeadm** folder to deploy a Kubernetes cluster over one or more Linux machines (physical or virtualized) using `kubeadm` utility. - -## __[Deploy a SQL Server big data cluster on Azure Kubernetes Service (AKS)](aks/)__ - -Using the sample Python script in **aks** folder, you will deploy a Kubernetes cluster in Azure using AKS and a SQL Server big data cluster using on top of it. - -## __[Push SQL Server big data cluster images to your own private Docker repository](offline/)__ - -Using the sample Python script in **offline** folder, you will push the necessary images required for the deployment to your own repository. \ No newline at end of file +Use the scripts in the **kubeadm** folder to deploy a Kubernetes cluster over one or more Linux machines (physical or virtualized) using `kubeadm` utility. \ No newline at end of file diff --git a/samples/features/azure-arc-data-controller/deployment/kubeadm/README.md b/samples/features/azure-arc-data-controller/deployment/kubeadm/README.md index d129e144..6a482e79 100644 --- a/samples/features/azure-arc-data-controller/deployment/kubeadm/README.md +++ b/samples/features/azure-arc-data-controller/deployment/kubeadm/README.md @@ -7,8 +7,4 @@ This folder contains scripts that provide a template for deploying a Kubernetes ## __[ubuntu-single-node-vm](ubuntu-single-node-vm/)__ -This folder contains a sample script that can be used to create a single-node Kubernetes cluster on a Linux machine and deploy SQL Server big data cluster. - -## __[ubuntu-single-node-vm-ad](ubuntu-single-node-vm-ad/)__ - -This folder contains a sample script that can be used to create a single-node Kubernetes cluster on a Linux machine and deploy SQL Server big data cluster with Active Directory integration. +This folder contains a sample script that can be used to create a single-node Kubernetes cluster on a Linux machine and deploy Azure Arc Data Controller cluster. diff --git a/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md b/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md index adb926cd..ba028a52 100644 --- a/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md +++ b/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md @@ -1,7 +1,7 @@ -# Deploy a SQL Server big data cluster on single node Kubernetes cluster (kubeadm) +# Deploy a Azure Arc Data Controller on single node Kubernetes cluster (kubeadm) -Using this sample bash script, you will deploy a single node Kubernetes cluster using kubeadm and a SQL Server big data cluster on top of it. The script must be run from the VM you are planning to use for your kubeadm deployment. +Using this sample bash script, you will deploy a single node Kubernetes cluster using kubeadm and a Azure Arc Data Controller on top of it. The script must be run from the VM you are planning to use for your kubeadm deployment. ## Pre-requisites @@ -20,34 +20,28 @@ sudo systemctl reboot 1. Use checkpoint or snapshot capability in your hyper visor so that you can rollback the virtual machine to a clean state. -## Instructions to deploy SQL Server big data cluster +## Instructions to deploy Azure Arc Data Controller 1. Download the script on the VM you are planning to use for the deployment ``` bash -curl --output setup-bdc.sh https://raw.githubusercontent.com/microsoft/sql-server-samples/master/samples/features/sql-big-data-cluster/deployment/kubeadm/ubuntu-single-node-vm/setup-bdc.sh +curl --output setup-controller.sh https://raw.githubusercontent.com/ananto-msft/sql-server-samples/master/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/setup-controller.sh ``` 2. Make the script executable ``` bash -chmod +x setup-bdc.sh +chmod +x setup-controller.sh ``` 3. Run the script (make sure you are running with sudo) ``` bash -sudo ./setup-bdc.sh +sudo ./setup-controller.sh ``` -4. Refresh alias setup for azdata - -``` bash -source ~/.bashrc -``` - -When prompted, provide your input for the password that will be used for all external endpoints: controller, SQL Server master and gateway. The password should be sufficiently complex based on existing rules for SQL Server password. The controller username is defaulted to *admin*. +When prompted, provide your input for the password that will be used for all external endpoints: controller, SQL Server master and gateway. The password should be sufficiently complex based on existing rules for SQL Server password. The controller username is defaulted to *controlleradmin*. ## Cleanup -1. The [cleanup-bdc.sh](cleanup-bdc.sh/) script is provided as convenience to reset the environment in case of errors. However, we recommend that you use a virtual machine for testing purposes and use the snapshot capability in your hyper-visor to rollback the virtual machine to a clean state. +1. The [cleanup-controller.sh](cleanup-controller.sh/) script is provided as convenience to reset the environment in case of errors. However, we recommend that you use a virtual machine for testing purposes and use the snapshot capability in your hyper-visor to rollback the virtual machine to a clean state. diff --git a/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu/README.md b/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu/README.md index 79d73ea7..ad17a2a2 100644 --- a/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu/README.md +++ b/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu/README.md @@ -12,8 +12,6 @@ In this example, we will deploy Kubernetes over multiple Linux machines (physica ### Useful resources -[Deploy SQL Server 2019 big data cluster on Kubernetes](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions) - [Creating a cluster using kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/) [Troubleshooting kubeadm](https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/) @@ -25,7 +23,3 @@ In this example, we will deploy Kubernetes over multiple Linux machines (physica 1. After successful initialization of the Kubernetes master, follow the kubeadm join commands output by the setup script on each agent machine 1. Execute [setup-volumes-agent.sh](setup-volumes-agent.sh/) script on each agent machine to create volumes for local storage 1. Execute ***kubectl apply -f local-storage-provisioner.yaml*** against the Kubernetes cluster to create the local storage provisioner. This will create a Storage Class named "local-storage". -1. Now, you can deploy the SQL Server 2019 big data cluster following instructions [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions). -Simply type in "local-storage" twice (once for data, once for logs) when facing the following prompt by azdata : - -`Kubernetes Storage Class - Config Path: spec.storage.data.className - Description: This indicates the name of the Kubernetes Storage Class to use. You must pre-provision the storage class and the persistent volumes or you can use a built in storage class if the platform you are deploying provides this capability. - Please provide a value:`