A series of Plugins for Apache Airflow (https://airflow.incubator.apache.org/)
-
Updated
Feb 10, 2021 - Python
A series of Plugins for Apache Airflow (https://airflow.incubator.apache.org/)
A flexible data integration tool to help nonprofits connect to their data collection tools and ERP systems
A repository containing all the data engineering and data analysis projects
Ready to go Apache Airflow stack for docker
Data Engineering using Apache Airflow
Projects related to ETL data pipelines, Kubernetes and Go REST API development
An EL pipeline built with Apache Airflow that downloads a file from the web uploads it to Google Cloud Storage, and creates an external table in BigQuery for data storage and analysis.
A comprehensive data engineering pipeline, orchestrates data workflows with Apache Airflow, Python, Kafka, Zookeeper, Spark, and Cassandra. Containerized using Docker: to deploy and scale effortlessly. This Etsy API Data Pipeline extracts, processes, and analyzes Etsy marketplace data—retrieving product listings, shop details, and reviews.
This project showcases the implementation of a data pipeline using Apache Airflow. Leveraging the OpenWeather API, it efficiently fetches real-time weather data and performs ETL processing. Results are seamlessly stored in AWS S3 buckets for further analysis. Moreover, the integration of Slack notifications ensures timely alerts to myself.
Small sample project to test Apache Airflow workflow engine.
Corrent: Experimental Airflow functional DAG API
Apache Airflow
An open source Tableau Prep Conductor alternative that combines Apache Airflow with Tableau’s Metadata and REST APIs to create a simple dockerized server for administering published Prep Flows.
An on-going application to track and therefore analyse personal activities and goal setting.
In this project, files were ingested to Google Cloud Storage and later to moved to BigQuery so as to perform some queries and the result moved back to Google Cloud Storage.
Entire ETL pipeline project with different tasks orchestration using Apache airflow. And data storage in PostgreSQL database using Python
Add a description, image, and links to the apache-airflow topic page so that developers can more easily learn about it.
To associate your repository with the apache-airflow topic, visit your repo's landing page and select "manage topics."