BestPromptFinder

Audit My AI Eval Suite Before I Trust Its Scores

Quality, usefulness and confidence are AI evaluations out of 100 — not user ratings. Community results come from visitors' "worked / didn't work" votes below. How scoring works →

The prompt

Review my evaluation setup for an AI feature and find ways it could report a good score that isn't real. What the feature does: [e.g. answers support questions from our docs] Eval code or config: [paste, or give file paths if you can read the repo] Recent results: [e.g. 0.9 pass rate] Check for: 1. Judges or graders that never actually ran (errors swallowed, defaults counted as passes, empty outputs scored). 2. Test cases that leak the answer, duplicate each other, or are far easier than real traffic. 3. Metrics that reward length, format or keywords instead of correctness. 4. Missing failure cases: refusals, wrong-but-confident answers, empty retrieval. 5. Thresholds or averages that hide a few bad categories. For each issue: where it is, how it inflates the score, and a small fix. Then suggest 5 sanity tests, such as feeding a deliberately wrong answer and checking it fails. Don't change code until I agree. Mark anything you can't verify from what I gave you as [VERIFY].

Why this prompt

• Hand-picked, editorially reviewed prompt
• Written for Claude, ChatGPT, Gemini, Claude Code

Source

Created by BestPromptFinder — editorial prompt, not user-generated. See our content & source policy.

Related Coding prompts