Unified AI engineering and evaluation platform to accelerate development and improvement of AI apps and agents
Arize AI publishes skills that help AI agents optimize and debug LLM prompts from production traces, manage Prompt Hub templates through the ax CLI, set up LLM-as-judge and code evaluators, and run error analysis on agent traffic in Phoenix. Its catalog also covers the Phoenix TypeScript packages, including OpenTelemetry registration, package documentation, and the vendored SQLite extension build, with a focus on evaluation and observability workflows.
Phoenix
40 skills · 12,179 stars · NOASSERTION
Create Task Create a new Harbor task for evaluating agents.
Datasets Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how…
Daytona Creates and manages isolated cloud sandboxes (secure code execution environments with dedicated r…
Evaluators Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output.
Experiments Run, read, and compare dataset-backed experiments to find evidence that a prompt or pipeline is i…
Harbor Exec Use when working with Harbor's `harbor exec` CLI workflow: compiling files, directories, or globs…
Issue Triage Run open-source triage on Arize-ai/phoenix issues — classify type, check sufficiency, apply compo…
Mintlify Set up, write, and publish documentation sites with Mintlify.
Phoenix Cli Debug LLM applications using the Phoenix CLI (`px`, from @arizeai/phoenix-cli).
Phoenix Cli Development Design and implementation guide for the Phoenix CLI (`px`).
Phoenix Client Development Development guide for the @arizeai/phoenix-client TypeScript SDK — run and resume experiments, ma…
Phoenix Design Design system conventions for the Phoenix frontend — layout, dialogs, error display, BEM CSS clas…
Phoenix Docs Gap Audit Audit documentation gaps across the Phoenix repo by analyzing recent commits to main (default: la…
Phoenix Error Analysis Find out what is going wrong in LLM or agent traffic by reading sampled Phoenix traces, spans, or…
Phoenix Evals Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript…
Phoenix Evals New Metric Add a new built-in (pre-built) LLM classification evaluator to the Phoenix evals libraries, and o…
Phoenix Frontend Frontend development guidelines for the Phoenix AI observability platform.
Phoenix Github Manage GitHub issues, labels, project boards, sprint operations, and roadmap health for the Arize…
Phoenix Graphql Write efficient GraphQL queries against the Phoenix API.
Phoenix Harbor Configure and interpret the Phoenix plugin for Harbor agent evaluations.
Phoenix Integration Snippets Generates onboarding code snippets for Phoenix tracing integrations and wires them into the proje…
Phoenix Llms Txt Maintain the Phoenix llms.txt documentation index at docs/phoenix/llms.txt — the machine-readable…
Phoenix Otel Development Guide for the phoenix-otel TypeScript package — OTel registration, stack-based global provider ma…
Phoenix Playwright Tests Write Playwright E2E tests for the Phoenix AI observability platform.
Phoenix Pr Description Writes and maintains the body of a pull request opened by a coding agent on Arize-ai/phoenix, inc…
Phoenix Pr Screenshot Screenshot a running Phoenix feature and attach images to a GitHub PR.
Phoenix Pxi Playwright Write, extend, and debug PXI Playwright E2E tests for Phoenix.
Phoenix Release Notes Create Phoenix release documentation grounded in actual code changes.
Phoenix Release Please Bump the next release-please version for a Phoenix Python package (arize-phoenix, arize-phoenix-c…
Phoenix Rest Api REST API development for Phoenix.
Phoenix Server Backend development guide for the Phoenix AI observability platform (Strawberry GraphQL, SQLAlche…
Phoenix Skills Audit Audit recent changes to Phoenix's user-facing surfaces (Python clients, TypeScript clients, CLI,…
Phoenix Sqlean Maintaining packages/phoenix-sqlean, the vendored fork of nalgeon/sqlean.py published as arize-ph…
Phoenix Storybook Conventions for creating, modifying, and reviewing production-faithful Storybook stories in the P…
Phoenix Tracing OpenInference semantic conventions and instrumentation for Phoenix AI observability.
Phoenix Typescript TypeScript conventions and patterns for any TypeScript code in the Phoenix monorepo — including j…
Phoenix Typescript Package Docs Maintain the bundled TypeScript package docs that ship inside Phoenix npm packages.
Playground Author, edit, or iterate on prompts in the Phoenix prompt playground, including running experimen…
Rewardkit Write Harbor task verifiers using Reward Kit.
Typescript Tooling Migration Migrate or upgrade TypeScript tooling in the Phoenix monorepo.
Arize Skills
18 skills · 72 stars · MIT
Arize Admin Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions…
Arize Ai Provider Integration Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials us…
Arize Annotation Creates and manages annotation configs (categorical, continuous, freeform label schemas) and anno…
Arize Compliance Audit INVOKE THIS SKILL when auditing an AI agent or LLM app for regulatory compliance.
Arize Dataset Creates, manages, and queries Arize datasets and examples.
Arize Evaluator Handles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators…
Arize Experiment Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance.
Arize Instrumentation Adds Arize AX tracing to an LLM application for the first time.
Arize Instrumentation Health Audits instrumentation health of existing Arize traces.
Arize Link Generates deep links to the Arize UI for projects, traces, spans, sessions, datasets, experiments…
Arize Migrate Braintrust Migrates LLM observability from Braintrust into Arize AX — historical trace import via OTLP (BT o…
Arize Migrate Langfuse Migrates LLM observability from Langfuse into Arize AX — historical trace import via OTLP (LF out…
Arize Migrate Langsmith Migrates LLM observability from LangSmith into Arize AX — historical trace import via OTLP (LS ou…
Arize Phoenix Migration Migrate existing Phoenix (PX) traces, datasets, experiments, and stored evaluation results into A…
Arize Prompt Optimization Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotat…
Arize Prompts INVOKE THIS SKILL for Arize Prompt Hub and `ax prompts` workflows: author or import templates and…
Arize Span Routing Use when one Python service must send each agent's, tenant's, team's, or request's spans to its c…
Arize Trace Downloads, exports, and inspects existing Arize traces and spans to understand what an LLM app is…