Example of how to use Kafka and Spark to handle streaming submissions of urls.
-
Updated
Oct 4, 2021 - Python
Example of how to use Kafka and Spark to handle streaming submissions of urls.
A transformation pipeline for Delta Lake using AWS SDK for Pandas
Developed an end-to-end real time sentiment analysis system using Delta Lake Medallion architecture, with Hugging Face Transformer Model, Spark, and MLflow integration.
Distributed Systems - Principles and Paradigms
Spark Structured Streaming application transferring Avro data from Kafka with Schema Registry to Delta Lake
Data pipeline that processes Formula1 data with Azure Databricks, DeltaLake, and Azure Data Factory
Example of local pyspark setup including DeltaLake for unit-testing
Коллекция кейсов на базе платформы Databricks
Completed the SQL Basics for Data Science Specialization from the University of California, Davis, gaining proficiency in Data Analysis, SQL, Apache Spark, and Delta Lake.
Building an Azure Data Lake for Bike Share Data Analytics
🚦 Project of Data Warehouse in star architecture based on UK traffic data
A Delta Table pipeline in Rust, triggered by Azure Functions responding to blob storage events in a specific container subfolder. The pipeline processes CSV files, updating or creating Delta Tables as needed, using merges for row changes.
Setting up a data lakehouse with delta lake using docker
Add a description, image, and links to the delta-lake topic page so that developers can more easily learn about it.
To associate your repository with the delta-lake topic, visit your repo's landing page and select "manage topics."