The open source AI engineering platform for agents, LLMs, and ML models.
MLflow publishes skills that help AI agents onboard to MLflow, instrument Python and TypeScript code with tracing, and connect to SageMaker Managed MLflow backends. Its skills also cover the evaluation side of agent work, including building scorers and judges, debugging failures from traces, analyzing multi-turn chat sessions, and fixing agent behavior through regression tests.
Skills
13 skills · 89 stars · Apache-2.0
Agent Evaluation Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality…
Analyzing Mlflow Session Analyzes an MLflow session — a sequence of traces from a multi-turn chat conversation or interact…
Analyzing Mlflow Trace Analyzes a single MLflow trace to answer a user query about it.
Build A Scorer Help the user choose, create, iterate, or align MLflow judges and scorers for an agent, RAG app,…
Debug Agent Root-causes AI agent failures from MLflow traces without requiring a code change, regression suit…
Fix Agent Issue Fix or change an AI agent's behavior using MLflow traces and regression tests.
Instrumenting With Mlflow Tracing Instruments Python and TypeScript code with MLflow Tracing for observability.
Mlflow Agent Master dispatcher for all MLflow workflows.
Mlflow Onboarding Onboards users to MLflow by determining their use case (GenAI agents/apps or traditional ML/deep…
Querying Mlflow Metrics Fetches aggregated trace metrics (token usage, latency, trace counts, quality evaluations) from M…
Retrieving Mlflow Traces Retrieves MLflow traces using CLI or Python API.
Sagemaker Mlflow Connect to SageMaker Managed MLflow (mlflow-app or mlflow-tracking-server ARN) as an MLflow backe…
Searching Mlflow Docs Searches and retrieves MLflow documentation from the official docs site.