corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a07df3-316b-710b-935b-483115126be8

Using LLM-as-a-Judge For Evaluation: A Complete Guide

Hamel Husain · published 29 October 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at hamel.dev Wayback capture
17blogs citing
First cited
1 Nov 2024
Most recent
5 Sept 2026
Rank this month
not ranked
Links from this post
29
Words captured
8,392

Cited by

Blog
In the post
Date
druce.ai
Agent Reliability Engineering: Field Notes on Building Predictable, Steerable Agents Read at the source ↗
5 Sept 2026
pydantic.dev
When agents improve agents Read at the source ↗
31 Jul 2026
Atomic Object
Get Good Results from AI Tools: Define, Feed, Check Read at the source ↗
30 Jul 2026
coles.codes
Pydantic AI structured outputs and evals on Bedrock Read at the source ↗
13 Jul 2026
blog.nilenso.com
Our approach to evals with megasthenes Read at the source ↗
10 Jun 2026
mkarots.github.io
What Do We Mean When We Say Evals? Read at the source ↗
25 May 2026
aroy.sh
A Synthesis of LLM Evaluation Read at the source ↗
15 Mar 2026
aussiebytes.com
Where does your software engineering team sit on the AI maturity curve? Read at the source ↗
21 Feb 2026
cameronrwolfe.substack.com
Rubric-Based Rewards for RL Read at the source ↗
16 Feb 2026
jxsh.io
Setting Up Your First Eval with Typescript Read at the source ↗
15 Jan 2026
PostHog Blog
Stop AI slop: Run evals with LLM-as-a-Judge Read at the source ↗
15 Jan 2026
Eugene Yan
Product Evals in Three Simple Steps Read at the source ↗
23 Nov 2025
Modal
How we used evals and inference-time compute scaling to generate beautiful QR codes that actually work Read at the source ↗
2 Jul 2025
dsdev.in
Curated resources: AI Product x How to eval? Read at the source ↗
9 Mar 2025
loumf
Evaluating the Evaluators Read at the source ↗
18 Jan 2025
dylandavis.net
Self Healing Code Read at the source ↗
3 Nov 2024
jdilla.xyz
Token stream 2024-11-01 Read at the source ↗
1 Nov 2024

Links from this post

Link
Host
Your AI Product Needs Evals Read at the source ↗
hamel.dev
Exploring Agent-Assisted Qualitative Analysis Read at the source ↗
parlance-labs.com
https://hex.tech/
hex.tech
https://www.linkedin.com/in/bryan-bischof/
linkedin.com
Your AI Product Needs Evals Read at the source ↗
hamel.dev
https://arxiv.org/abs/2404.12272
arxiv.org
https://www.honeycomb.io/blog/introducing-query-assistant
honeycomb.io
https://x.com/_cartermp
x.com
https://blog.langchain.dev/dosu-langsmith-no-prompt-eng/
blog.langchain.dev
https://humanloop.com/blog/why-your-product-needs-evals
humanloop.com
https://www.latent.space/
latent.space
https://x.com/swyx/status/1672306744884887553
x.com
https://buttondown.com/ainews
buttondown.com
https://eugeneyan.com/
eugeneyan.com
https://eugeneyan.com/writing/aligneval/
eugeneyan.com
https://www.linkedin.com/in/phillip-carter-4714a135/
linkedin.com
https://platform.openai.com/docs/guides/prompt-caching
platform.openai.com
https://jxnl.co/writing/2024/06/02/10-ways-to-be-data-illiterate-and-how-to-avoid-them/
jxnl.co
https://x.com/HamelHusain/status/1850256204553244713
x.com
Model Distillation in the API Read at the source ↗
openai.com
https://parlance-labs.com/education/
parlance-labs.com
https://hamel.dev/evals
hamel.dev
https://aligneval.com/
aligneval.com
https://eugeneyan.com/writing/llm-evaluators/
eugeneyan.com
https://www.databricks.com/blog/enhancing-llm-as-a-judge-with-grading-notes
databricks.com
https://cookbook.openai.com/examples/custom-llm-as-a-judge
cookbook.openai.com
https://arize.com/blog/techniques-for-self-improving-llm-evals/
arize.com
https://www.youtube.com/watch?v=tHZtq_pJSGo
youtube.com
https://applied-llms.org/
applied-llms.org