Torch Automatic Distributed Neural Network (TorchAD-NN) training library. Built on top of TorchMPI, this module automatically parallelizes neural network training.
-
Updated
Feb 28, 2018 - Lua
Torch Automatic Distributed Neural Network (TorchAD-NN) training library. Built on top of TorchMPI, this module automatically parallelizes neural network training.
Mesh TensorFlow: Model Parallelism Made Easier
distributed tensorflow (model parallelism) example repository
A decentralized and distributed framework for training DNNs
performance test of MNIST hand writings usign MXNet + TF
An MPI-based distributed model parallelism technique for MLP
A simple graph partitioning algorithm written in Go. Designed for use for partitioning neural networks across multiple devices which has an added cost when crossing device boundaries.
PyTorch implementation of 3D U-Net with model parallel in 2GPU for large model
A GPipe implementation in PyTorch
Development of Project HPGO | Hybrid Parallelism Global Orchestration
Slicing a PyTorch Tensor Into Parallel Shards
A fully distributed hyperparameter optimization tool for PyTorch DNNs
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
The project is focused on parallelising pre-processing, measuring and machine learning in the cloud, as well as the evaluation and analysis of the cloud performance.
WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
Adaptive Tensor Parallelism for Foundation Models
Description of Framework for Efficient Fused-layer Cost Estimation, Legion (2021)
Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
Official implementation of DynPartition: Automatic Optimal Pipeline Parallelism of Dynamic Neural Networks over Heterogeneous GPU Systems for Inference Tasks
Add a description, image, and links to the model-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the model-parallelism topic, visit your repo's landing page and select "manage topics."