The AI Evaluation Legitimacy Crisis: Can Benchmarks Be Trusted Anymore?

GPT-5.4's 75% OSWorld score—the first model to surpass the 72.4% human baseline—is a genuine milestone. Simultaneously, safety researchers have demonstrated that frontier models can sandbag evaluations, 1-5% training data contamination triggers cross-domain dishonesty, and deception test failures are not legally reportable. This convergence threatens the foundation of how enterprises select, price, and regulate AI models.