Skip to content

Experimental HPC accelerated deep Learning research, a next-gen R&D AI project with Scala API. πŸš€

Notifications You must be signed in to change notification settings

binarybrainiacs/bitfusion

Repository files navigation

πŸ—οΈ Building BitFusion

This README outlines the process of building BitFusion, a high-performance computing framework, utilizing various essential libraries and tools.

βš™οΈ Prerequisites

Prior to building BitFusion, ensure your development environment is equipped with the following:

Note: Compatibility and specific versions of these components may vary. Consult the official documentation for the most up-to-date requirements.

πŸ“¦ Dependency Management

The following dependencies are crucial for BitFusion's functionality. It's recommended to install them in a specific order for optimal compatibility.

Core Dependencies

  1. Torch (PyTorch): BitFusion leverages PyTorch for deep learning capabilities. Install PyTorch with CUDA support. (https://pytorch.org/get-started/locally/)
  2. cuDNN (CUDA Deep Neural Network library): For optimized deep learning operations, install the cuDNN library compatible with your CUDA version. (https://developer.nvidia.com/cudnn)
  3. OpenBLAS: OpenBLAS provides high-performance BLAS (Basic Linear Algebra Subprograms) routines. (https://www.openblas.net/)

Additional Dependencies

  1. netcdf-cxx4 (netCDF-CXX4): NetCDF-CXX4 is a C++ library for reading and writing netCDF files. (https://www.unidata.ucar.edu/software/netcdf/)
  2. jsoncpp (JsonCpp): JsonCpp enables efficient JSON parsing and manipulation. (https://github.com/open-source-parsers/jsoncpp)
  3. MSMPI (Microsoft MPI): Microsoft MPI provides support for parallel programming using the Message Passing Interface (MPI) standard. (https://docs.microsoft.com/en-us/message-passing-interface/microsoft-mpi)
  4. glog (Google Logging Library): glog offers a robust logging framework for debugging and monitoring. (https://github.com/google/glog)
  5. Highway: Highway is a high-performance library for SIMD (Single Instruction Multiple Data) operations. (https://github.com/google/highway)
  6. Boost C++ Libraries: Boost provides a wide range of reusable C++ libraries for diverse functionalities. (https://www.boost.org/)
  7. Mojo: Mojo is a network communication library designed for high-performance applications. (https://www.modular.com/max/mojo)
  8. pybind11: pybind11 facilitates seamless integration between Python and C++ code. (https://github.com/pybind/pybind11)
  9. NCLL (NCCL): NCCL provides high-performance communication for NVIDIA GPUs. (https://github.com/NVIDIA/nccl)
  10. Gtest (Google Test): Gtest is a powerful testing framework for C++ code. (https://github.com/google/googletest)
  11. cutlass: Cutlass is a library for generating high-performance CUDA kernels. (https://github.com/NVIDIA/cutlass)
  12. TensorRT: TensorRT provides an inference engine for deploying deep learning models with optimized performance. (https://developer.nvidia.com/tensorrt-download)

Installation:

  • Ensure that all dependencies are installed on your system and their respective libraries are available in the system PATH.
  • Consult individual library documentation for specific installation procedures and requirements for your operating system.

πŸ”¨ Building BitFusion

  1. Open Visual Studio 2022: Launch Visual Studio 2022.
  2. Open Solution File: Navigate to the BitFusion project directory and open the bitfusion.sln solution file.
  3. Select Build Configuration: In the Visual Studio menu, choose the desired build configuration (e.g., Debug or Release).
  4. Build Solution: Utilize the shortcut Ctrl + Shift + B to build the entire solution. Alternatively, use the "Build" menu and select "Build Solution".

Troubleshooting: If you encounter any build errors, review the Visual Studio output window for specific messages and consult the respective dependency documentation for potential solutions.

πŸ§ͺ Running Tests

Once the project has been built successfully, execute unit tests to verify its functionality:

  1. Navigate to the Tests Directory: Open the tests folder within the BitFusion project directory.
  2. Run Tests: Locate the appropriate test executable (e.g., bitfusion_tests.exe) and execute it.

Note: Ensure the required test dependencies are installed and the test environment is correctly configured.

πŸ“š Documentation

Detailed documentation regarding BitFusion's functionalities, APIs, and other aspects can be found in the docs directory.

🀝 Contributing

We encourage contributions from the community to improve and extend BitFusion's capabilities. If you're interested in contributing, please refer to the Contribution Guidelines for more information.

πŸ“„ License

BitFusion is licensed under the Apache License 2.0. Please review the license for details on usage and distribution rights.