Skill Store 33

Workflow Llm Eval

Runs LLM-based workflow eval 12 by checking the analysis.md Appendix hardware reference values (plat

by amd · MIT

The skill

Runs LLM-based workflow eval 12 by checking the analysis.md Appendix hardware reference values (platform, memory bandwidth, MAF) for plausibility and writing a one-row results CSV. Use when the user asks for a workflow LLM eval, eval 12, or Appendix hardware reference scoring. Scripted evals 9 to 11 and 13 to 14 are in workflow_scripted_evals.py and are not covered here.

<!-- Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.

See LICENSE for license information. -->

name: workflow-llm-eval description: >- Runs LLM-based workflow eval 12 only: checks the analysis.md Appendix for plausible hardware reference values (platform, memory BW, MAF) and writes a one-row results CSV. Use when the user asks for a workflow LLM eval, eval 12, or Appendix hardware reference scoring. Scripted evals 9–11, 13–14 live in workflow_scripted_evals.py.

Workflow LLM eval

Semantic check of ## Appendix hardware facts in analysis.md (Gaia-style weighted correctness + completeness). Produces a single CSV row for workflow_eval_12.

Full procedure

See reference.md for inputs, files to read, scoring table, pass/fail thresholds, and CSV column contract.

Skill location

TraceLens/Agent/Analysis/skills/analysis-orchestrator/evals/skills/workflow-llm-eval/

Use this skill

Connect your agent to the Skill Store MCP server and ask for it by ref: amd/workflow-llm-eval-57698201359.

Download SKILL.md · Markdown version

Details

Publisher
amd
Category
Software Development › Testing & QA
License
MIT
Updated
2026-10-16

References

More from amd

  • Hyperloom Workload Optimizer Autonomously optimizes end-to-end LLM inference throughput on AMD Instinct GPUs and reports a val…
  • Lemonade Router Builder Turns a natural-language description of routing intent into a valid Lemonade `collection.router`…
  • Local Ai App Integration Integrates local AI capabilities into applications using Embeddable Lemonade.
  • Local Ai Use Makes this agent generate images, transcribe audio, and synthesize speech on the user's own machi…

Related skills

  • Quality Llm Eval Runs LLM-based quality evals 2 and 3 on generated analysis.md and category_findings files, comparing
  • Evaluating Llms Harness Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).
  • Exploring Llm Evaluations Investigate AI observability evaluations — `hog` (deterministic code-based), `llm_judge` (LLM-pro…
  • Llm Evaluation Implement comprehensive evaluation strategies for LLM applications using automated metrics, human…
  • Arize Evaluator Handles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators…