Mlflow Extensions

The goal of this project is to make deploying any large language model, or multi modal large language models a simple three-step process.

1. Download the model from hf or any other source.
2. Register the model with mlflow.
3. Deploy the model using the mlflow serving infrastructure. (e.g. Databricks)

Framework Support / Roadmap

• vLLM
• SGLang
• Ray Serving [WIP]

This project will take those optimized model serving frameworks and deploy them to the following deployment targets.

Deployment Clouds

• AWS
• Azure
• GCP

Deployment Targets

• Databricks Model Serving
• Databricks Job Cluster
• Databricks Interactive Clusters

Deployment Modes

• EzDeployLite will ship a prebuilt configuration to databricks jobs. (dev/testing)
• EzDeploy will ship a prebuilt configuration to databricks model serving. (production)

Disclaimer

mlflow-extensions is not developed, endorsed not supported by Databricks. It is provided as-is; no warranty is derived from using this package. For more details, please refer to the license.