Loading
Topic
Spark SQL
4 posts about Spark SQL by 3 contributors.
Know Spark SQL? Share what you know with developers who are learning it.
+ Follow Spark SQLBe the first to follow this
- Articles
- 2
- Videos
- 2
- Contributors
- 3
Everything about Spark SQL
- Video · Databricks
Real-Time Streaming Data in Databricks [End to End Project]
??In this comprehensive end-to-end data engineering video, I demonstrated how to architect real-time streaming data solutions in Databricks, starting from weather data captured in VS Code, seamlessly integrated with Azure Event Hubs. The setup leverages the robust capabilities of the Medallion Architecture and the advanced features of Unity Catalog to ensure efficient data management and processing.In this video, you will learn how to: ?? Create Event Hubs and Share Access Policies ?? Generate weather streaming data using Python in VS Code ?? Send the events to Azure Event Hubs ?? Create Databricks Cluster and Install Maven Library to authenticate access Azure EventHub ?? Create unity catalog, schemas and volumes using medallion architecture ?? Read and write streams across the bronze, silver and gold layers ??Access Power BI Desktop in Marketplace Partners Connect tab ?? Build near real-time Power BI report using DirectQuery
