What it does
MLflow is an open-source platform for building and running AI applications. For LLM apps and agents it provides tracing built on OpenTelemetry that works with any LLM provider and agent framework, systematic evaluation with more than 50 built-in metrics and LLM judges, and a prompt registry with versioning and lineage plus automatic prompt optimisation. An AI gateway puts one OpenAI-compatible API in front of LLM providers, with request routing, rate limits, fallbacks and cost control. An Agent Server deploys agents behind a FastAPI-based endpoint with request validation, streaming and built-in tracing. For classic machine learning, MLflow covers experiment tracking, hyperparameter tuning, model evaluation, a model registry and deployment tools. Getting started takes a command such as uvx mlflow server, or Docker, followed by a line of autologging code in Python. Supported languages include Python, TypeScript/JavaScript, Java and R. The project is backed by the Linux Foundation and licensed under Apache 2.0, and a hosted demo is linked from the site.
Capabilities
- Tracing of LLM apps and agents built on OpenTelemetry
- 50+ built-in evaluation metrics and LLM judges
- Prompt registry with versioning, lineage and optimisation
- AI gateway with an OpenAI-compatible interface, routing and fallbacks
- Agent Server for deploying agents as FastAPI endpoints
- Experiment tracking and a model registry for classic ML
When to use it
- You need traces, quality metrics and cost visibility for an LLM app or agent in production
- You want to compare prompt or model changes with repeatable evaluations
- You want one place to track experiments and register models for traditional ML
- You prefer an open-source tool you run yourself that is not tied to one provider
