Spark SQL
4 posts about Spark SQL by 3 contributors.
Know Spark SQL? Share what you know with developers who are learning it.
- Articles
- 2
- Videos
- 2
- Contributors
- 3
Videos about Spark SQL
- Video · Databricks
Real-Time Streaming Data in Databricks [End to End Project]
??In this comprehensive end-to-end data engineering video, I demonstrated how to architect real-time streaming data solutions in Databricks, starting from weather data captured in VS Code, seamlessly integrated with Azure Event Hubs. The setup leverages the robust capabilities of the Medallion Architecture and the advanced features of Unity Catalog to ensure efficient data management and processing.In this video, you will learn how to: ?? Create Event Hubs and Share Access Policies ?? Generate weather streaming data using Python in VS Code ?? Send the events to Azure Event Hubs ?? Create Databricks Cluster and Install Maven Library to authenticate access Azure EventHub ?? Create unity catalog, schemas and volumes using medallion architecture ?? Read and write streams across the bronze, silver and gold layers ??Access Power BI Desktop in Marketplace Partners Connect tab ?? Build near real-time Power BI report using DirectQuery
- Video · Azure
Read, Combine and Analyse ADLS Gen2 CSV files using Azure Synapse Spark SQL
This video shows how to use Azure Synapse Analytics to read, combine, and analyze multiple CSV files residents in ADLS Gen2 using Spark SQL.
