Monitoring Data Pipeline Design Example

Spark tutorial

An End-To-End data pipeline integration from Website Source to analytical dashboard in AWS using Python flask, ML models, DynamoDB and other AWS services.

Building a highly scalable data streaming pipeline in python

Udacity's Data Engineering Nanodegree project: Data Pipeline with Airflow.

Script to write GFW country stats XLS files for download

This repo includes the code which will help beginners pick up tensorflow dataset and build effective pipelines for their Machine Learning and Deep Learning experiments built for my medium post on the same.

Cyber Attack Prediction Model

Automated data pipeline for cryptocurrency exchange analytics. Extracts data from multiple sources, processes it, and visualizes insights via a near real-time dashboard. Built with Python and Docker, featuring modular ETL, comprehensive logging, and automated testing.

Prototype for easy data transfer between MySql, Bigquery and CSV

This project focuses on the Air Pressure system (APS), which creates pressurized air for truck brakes and gear changes. Positive datasets include component failures. APS component. Negative class trucks have non-APS-related component failures.

This Project aims to automate the process of infrastructure creation.

ETL pipeline and analysis of Walmart Dataset using python

data pipeline

This project creates and builds data models and deploy database for E-commerce store

Executed diverse ML and NLP initiatives, implementing robust data pipelines, thorough data analysis, and seamless Docker projects. Demonstrated versatility in handling complex tasks, ensuring successful project outcomes

Welcome to the MiniProjects Playground—an interactive space where learning meets doing! This repository is a collection of hands-on mini-projects that I've crafted after delving into various tech stacks and frameworks. From theory to application, each project is a testament to the practical side of coding.

Simple ETL tool for Salesforce