https://hamel.dev
Hamel Husain writes skills that help AI agents evaluate and debug LLM pipelines, covering error analysis of traces, auditing existing eval systems, writing LLM-as-judge prompts and calibrating those judges against human labels. His most popular skills also include building browser-based annotation interfaces for reviewing traces and collecting structured feedback. What stands out is a consistent focus on checking whether evaluation itself can be trusted before relying on its results.
Evals Skills
5 skills · 1,674 stars · MIT
Build Review Interface Build a custom browser-based annotation interface tailored to your data for reviewing LLM traces…
Error Analysis Help the user systematically identify and categorize failure modes in an LLM pipeline by reading…
Eval Audit Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated judges, vani…
Validate Evaluator Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction.
Write Judge Prompt Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle.