pipeDejavu: Hardware-aware Latency Predictable, Differentiable Search for Faster Config and Convergence of Distributed ML Pipeline Parallelism
-
Updated
May 9, 2023 - Jupyter Notebook
pipeDejavu: Hardware-aware Latency Predictable, Differentiable Search for Faster Config and Convergence of Distributed ML Pipeline Parallelism
Description of Framework for Efficient Fused-layer Cost Estimation, Legion (2021)
A fully distributed hyperparameter optimization tool for PyTorch DNNs
A simple graph partitioning algorithm written in Go. Designed for use for partitioning neural networks across multiple devices which has an added cost when crossing device boundaries.
WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
Adaptive Tensor Parallelism for Foundation Models
Official implementation of DynPartition: Automatic Optimal Pipeline Parallelism of Dynamic Neural Networks over Heterogeneous GPU Systems for Inference Tasks
Mesh TensorFlow: Model Parallelism Made Easier
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
Development of Project HPGO | Hybrid Parallelism Global Orchestration
distributed tensorflow (model parallelism) example repository
A decentralized and distributed framework for training DNNs
performance test of MNIST hand writings usign MXNet + TF
PyTorch implementation of 3D U-Net with model parallel in 2GPU for large model
Torch Automatic Distributed Neural Network (TorchAD-NN) training library. Built on top of TorchMPI, this module automatically parallelizes neural network training.
The project is focused on parallelising pre-processing, measuring and machine learning in the cloud, as well as the evaluation and analysis of the cloud performance.
Serving distributed deep learning models with model parallel swapping.
An MPI-based distributed model parallelism technique for MLP
Fast and easy distributed model training examples.
Add a description, image, and links to the model-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the model-parallelism topic, visit your repo's landing page and select "manage topics."