A fully distributed hyperparameter optimization tool for PyTorch DNNs
-
Updated
Jan 12, 2022 - Python
A fully distributed hyperparameter optimization tool for PyTorch DNNs
Mesh TensorFlow: Model Parallelism Made Easier
performance test of MNIST hand writings usign MXNet + TF
A simple graph partitioning algorithm written in Go. Designed for use for partitioning neural networks across multiple devices which has an added cost when crossing device boundaries.
An MPI-based distributed model parallelism technique for MLP
Torch Automatic Distributed Neural Network (TorchAD-NN) training library. Built on top of TorchMPI, this module automatically parallelizes neural network training.
The project is focused on parallelising pre-processing, measuring and machine learning in the cloud, as well as the evaluation and analysis of the cloud performance.
distributed tensorflow (model parallelism) example repository
Description of Framework for Efficient Fused-layer Cost Estimation, Legion (2021)
pipeDejavu: Hardware-aware Latency Predictable, Differentiable Search for Faster Config and Convergence of Distributed ML Pipeline Parallelism
Development of Project HPGO | Hybrid Parallelism Global Orchestration
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
Serving distributed deep learning models with model parallel swapping.
A decentralized and distributed framework for training DNNs
Official implementation of DynPartition: Automatic Optimal Pipeline Parallelism of Dynamic Neural Networks over Heterogeneous GPU Systems for Inference Tasks
Fast and easy distributed model training examples.
Adaptive Tensor Parallelism for Foundation Models
PyTorch implementation of 3D U-Net with model parallel in 2GPU for large model
WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
Add a description, image, and links to the model-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the model-parallelism topic, visit your repo's landing page and select "manage topics."