Comet builds the infrastructure teams use to develop, evaluate, and improve modern AI applications.
Comet is the author of a set of Opik skills that help an AI agent instrument applications with tracing, build and run LLM evaluations, turn failing traces into regression tests, compare candidate and baseline runs, and decide whether a release should ship. The skills also cover diagnosing traces by severity, running LLM-as-judge rules on production traffic, and reference material for the Opik SDK across Python, TypeScript and REST. The common thread is the full development loop for LLM applications, from first trace to live monitoring.
Opik Skills
10 skills · 8 stars · Apache-2.0
Opik Reference for the Opik SDK — tracing, span types, framework integrations, threads, and the prompt…
Opik Compare Run a candidate against the baseline over an Opik test suite and read the numbers back — which ca…
Opik Diagnose Surface the Opik traces worth a developer's attention, ranked by signal — Diagnostics issues firs…
Opik Evaluate Build an LLM evaluation and run it against the app, returning an Opik experiment with scores and…
Opik Explain Root-cause a specific Opik trace, or a pattern across traces, and return a grounded explanation.
Opik Instrument Add Opik tracing to an existing app and verify a real trace lands.
Opik Online Eval Take a judge live on production traffic — create an Opik online evaluation rule (LLM-as-judge or…
Opik Optimize Improve a prompt with the Opik Agent Optimizer — resolve the prompt, a dataset, and a metric, pic…
Opik Test Turn a failing Opik trace (or a described failure) into a repeatable regression check — a test-su…
Opik Verify Decide ship or hold for a candidate from the compare skill's numbers, against an explicit release…