# LLM-as-Judge

Source: https://customlabs.io/glossary/llm-as-judge/
Updated: 2026-09-13

Evaluation

# LLM-as-Judge

LLM-as-judge uses another LLM to score a model's outputs against a rubric, at scale.

It is useful for grading subjective qualities like tone, relevance, or faithfulness to a source.

It inherits the judging model's own biases and blind spots, so it needs calibration.

It is usually calibrated against a smaller human-labeled sample rather than trusted blind.

Treat it as one signal in an eval suite, not the whole suite.

[Insight: Evals before you ship](https://customlabs.io/insights/evals-before-you-ship/)[Field guide: Failure modes](https://customlabs.io/failure-modes/)

[← Back to the full glossary](https://customlabs.io/glossary/)

## Related terms

[Eval Suite (Evals) An eval suite is a versioned set of test cases that measures whether outputs are good.](https://customlabs.io/glossary/eval-suite/)[Observability Observability captures traces of every prompt, retrieval, tool call, and response for debugging.](https://customlabs.io/glossary/observability/)

## More in Evaluation

[Hallucination A hallucination is a confident, fluent output that is factually wrong or unsupported.](https://customlabs.io/glossary/hallucination/)[Prompt Injection Prompt injection is untrusted input crafted to override a model's system prompt or task.](https://customlabs.io/glossary/prompt-injection/)[Shadow Deployment A shadow deployment runs a candidate release against real production traffic, offline, before release.](https://customlabs.io/glossary/shadow-deployment/)