Swap A and B: LLM Judges Flip 43% of Verdicts
A study of 36 LLM judges, including Claude and GPT, found that simply swapping the order of two responses flips 43% of their verdicts, exposing a deep position bias that undermines automated AI evaluation.