Skip to content
@bentoml

BentoML

The easiest way to run AI Inference in the cloud
README.md

Welcome to BentoML 👋 Twitter Follow Slack

BentoML

What is BentoML? 👩‍🍳

BentoML is an open-source model serving library for building model inference APIs and multi-model serving systems with any open-source or custom AI models. It comes with everything you need for serving optimization, model packaging, and simplifies production deployment via ☁️ BentoCloud.

Get in touch 💬

👉 Join our Slack community!

👀 Follow us on X @bentomlai and LinkedIn

📖 Read our blog

Pinned Loading

  1. BentoML BentoML Public

    The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

    Python 6.8k 770

  2. OpenLLM OpenLLM Public

    Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

    Python 9.3k 595

Repositories

Showing 10 of 79 repositories
  • BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    bentoml/BentoVLLM’s past year of commit activity
    Python 39 9 3 0 Updated Jul 16, 2024
  • helm-charts Public
    bentoml/helm-charts’s past year of commit activity
    0 0 0 0 Updated Jul 16, 2024
  • yatai-image-builder Public

    🐳 Build OCI images for Bentos in k8s

    bentoml/yatai-image-builder’s past year of commit activity
    Go 14 9 4 5 Updated Jul 16, 2024
  • BentoML Public

    The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

    bentoml/BentoML’s past year of commit activity
    Python 6,805 Apache-2.0 770 152 13 Updated Jul 16, 2024
  • OpenLLM Public

    Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

    bentoml/OpenLLM’s past year of commit activity
    Python 9,317 Apache-2.0 595 24 2 Updated Jul 15, 2024
  • openllm-models Public
    bentoml/openllm-models’s past year of commit activity
    HTML 0 1 0 1 Updated Jul 13, 2024
  • byoc Public
    bentoml/byoc’s past year of commit activity
    Shell 0 0 0 0 Updated Jul 10, 2024
  • BentoInfinity Public
    bentoml/BentoInfinity’s past year of commit activity
    Python 0 0 0 0 Updated Jul 9, 2024
  • BentoLMDeploy Public

    Self-host LLMs with LMDeploy and BentoML

    bentoml/BentoLMDeploy’s past year of commit activity
    Python 12 1 1 0 Updated Jul 8, 2024
  • BentoTRTLLM Public
    bentoml/BentoTRTLLM’s past year of commit activity
    Python 2 1 0 1 Updated Jul 4, 2024
View all repositories

Sponsoring

  • @pdm-project

Top languages

Loading…
GitHub Sponsor