This is an index for all of the demo goodness on the site. Please also check out My YouTube Channel for video demos and davedoesdemos.com for other info.
Data Lake Documentation and Opinions This repo contains my opinions on best practices and other documentation around a data lake. Please consider this incomplete, and read it alongside other documentation from other sources. It will be updated as and when my thoughts change and I discuss topics with my team.
| SubCategory | Demo Instructions | YouTube Video | Description |
|---|---|---|---|
| Azure Data Factory | DevOps with ADF | Video | This demo shows how to use DevOps techniques with Azure Data Factory. |
| Azure Databricks | DevOps with Databricks | Video | This demo shows how to use DevOps techniques with Azure Databricks Notebooks. |
| Testing | Intro to testing | Video | This demo introduces the process of testing in Azure DevOps |
| Testing | Data testing | No Video Yet | This demo introduces data platform testing in Azure DevOps |
| Python Testing | Python testing | No Video Yet | This demo introduces python testing in Azure DevOps |
| Category | SubCategory | Demo Instructions | YouTube Video | Description |
|---|---|---|---|---|
| Azure Data Factory | Ingest | Google Big Query Ingest | Video | This demo shows how to get Google BigQuery data into an Azure data lake using Azure Data Factory. |
| Azure Data Factory | Ingest | SFTPIngest | Video | This demo shows how to ingest data from an SFTP source to your Azure Data Lake |
| Azure Data Factory | External Data | Quickwins-WeatherData | Video | This quick win demo is a very short and concise way to get weather data into your data lake to use for analytics. Here we use openweathermap.org but this example would work with any API source of data. |
| Azure Data Factory | Data Flows | Preview ADF Data Flows | No Video Yet | This demo takes you through the preview mapping data flows functionality in Azure Data Factory. Sample data is included to make this a quick and easy one to try. |
| Azure Data Factory | DevOps | Quickwins-ADFGit | No Video Yet | This demo shows how to use Git repositories to version control and merge code from dev to prod in Data Factory |
| Logic Apps | Data Capture | Quickwins-FormsData | Video | This demo shows how you can use a simple HTML form stored in Blob storage to submit data and store it as JSON (or any other format) in Logic Apps |
| Logic Apps | Azure API | Scale a Power BI Embedded Instance using Logic Apps | No Video Yet | This demo shows how you can use a Logic App to scale a Power BI Embedded instance. This technique can be used for any Azure API call |
| Cosmos DB | Gremlin | CosmosDBGremlinPowerBI | Video | Build a graph database in Cosmos DB with the Gremlin model and then surface that graph data into Power BI |
| Power BI | Streaming Data | Quickwins-PowerBIStreaming | No Video Yet | This very quick demo shows how to set up a streaming data source in PowerBI and generate some data with a Logic App to test it. |
| Cognitive Services | Translation | Book Translator | No Video Yet | This C# project is a book translator. It takes in a book in XML format and translates using the Azure cognitive services to a new language |
| Azure Functions | Streaming Data | CSV from Blob to Event Hubs | Video | This project is a function app that collects CSVs from Blob and injects the rows one at a time into an Event Hub instance with optional delay |
| Stream Analytics Jobs | Streaming Data | Global data distribution | Video | This demo shows how to use Stream Analytics to distribute data from Event Hubs into SQL Servers globally with minimal latency using the CSV function app as a source |
| Storage | Tiering | Logic App to archive new data | Video | This demo shows how to programmatically move data immediately to archive tier when it's ingested into a storage account using a Logic App. |
| Storage | Tiering | Function App to archive new data | No Video Yet | This demo shows how to programmatically move data immediately to archive tier when it's ingested into a storage account using a Function. |
| Storage | Tiering | Function App to change tier by prefix or folder | No Video Yet | This demo shows how to programmatically move data to a different tier based on container, prefix (folder) and a tier parameter. |
| Log Analytics | Data Ingest | Function App to ingest files as data into Log Analytics | No Video Yet | This demo shows how to ingest text data (such as unsupported logs) into Log Analytics when they land on a storage account |
| Storage | Data Upload Website | Web interface to upload file data to Blob storage with drag and drop | Video | This demo shows how to use HTML and Javascript to upload files to Azure using a drag and drop website hosted in a storage account |
| Logic Apps | GitHub | Long term Github traffic statistics | video | This demo shows how to get long term Github traffic statistics and insights using Logic Apps and the API to create reports beyond the 14 days Gihub provides |
| IoT Hub | Events | IoT Hub Event processor | No video yet | This demo shows how to get values from a JSON array submitted to IoT Hub and process the elements in Event Hubs or SQL Server |
| Governance | Lineage | Simple Data Lineage | No video yet | This demo shows how to create data lineage inside your data set so that each row contains its own lineage information |
| Category | SubCategory | Demo Instructions | YouTube Video | Description |
|---|---|---|---|---|
| Azure OpenAI | App | Fashion App | Video | Introduction to the series |
| Category | SubCategory | Demo Instructions | YouTube Video | Description |
|---|---|---|---|---|
| Application | ConnectIQ | ConnectIQ App Code | No Video | The code repository for the ConnectIQ apps |
| Streaming Data | Intro | ConnectIQ Garmin watch demo app | Video | Introduction to the Garmin IoT Project |
| Streaming Data | Event Hubs | Watch demo infrastructure | Video | Create your ingest infrastructure and Power BI dashboard |
| Streaming Data | Predictive Model | Watch Demo HR check with machine learning | No Video Yet | Transform data and train a machine learning model |
ARM Template samples for various products