Dead Benchmarks
Vania ChowStress-testing the benchmarks the AI industry runs on.
| Platform |  | Pricing | Only free issues | Publishes | Weekly |
|---|---|---|---|---|---|
| Issues | 5 | Founded | a month ago | Last Issue | 5 days ago |
| Active | |||||
Read this Newsletter
deadbenchmarks.substack.comLatest Issues
The Working Capital Crisis Inside the AI Transition
I. The Mismatch
Recurring revenue has been the golden child of software economics, where customers signed annual contracts, paid in advance, and vendors recognized revenue ratably while collecting cash on day zero. Deferred revenue pre-...
The AI Industry Knows Its Most-Used Benchmark Is Broken. It Uses It Anyway.
The known broken thing
In June 2024, a team at the University of Edinburgh published a paper titled “Are We Done With MMLU?” They re-annotated three thousand questions from the benchmark by hand. They found that 6.5% of the items had wro...
Can You Fool an LLM Judge?
The AI industry has a grading problem, and its favorite solution is to hand the red pen to another AI. That’s LLM-as-judge evaluation. In our previous piece, we used this method to test whether constitutional AI actually improves moral reas...
Does Constitutional AI Work?
In January 2026, Anthropic published an updated constitution for Claude — a 23,000-word document outlining the principles, values, and behavioral guidelines that shape how the model reasons about the world. It establishes a clear priority h...
We Don't Have Safety Ratings for AI — And We're Already Letting It Drive
Before a car reaches a dealership, it gets crash-tested. Before a drug reaches a pharmacy, it passes clinical trials. Before a building opens to the public, it gets a structural inspection. These systems exist because the consequences of fa...
Key Facts
Audience Metrics
Subscribers, engagement, traffic and sponsorship for Dead Benchmarks.
| Subscribers | Engagement | 66 | Monthly Web Visits | ||
|---|---|---|---|---|---|
| Accepts Sponsors | Estimated Cost per Ad | ||||
Explore More
Authors
The writers behind this newsletter.
Computer Science @ Stanford University | Research @ Free Systems
Frequently Asked Questions
How can I access the email archive for Dead Benchmarks?
You can find recent issues that have been published by Dead Benchmarks on Reletter by scrolling up to where it says Latest Issues. Tap on the link for any of the most recent emails or hit More Issues to see older ones.
How many subscribers does Dead Benchmarks have?
To see how many people subscribe to Dead Benchmarks, simply upgrade your Reletter account. We provide readership numbers and lots of other stats for this newsletter so you can decide if it's worth reaching out to.
How can I advertise in Dead Benchmarks?
Newsletter advertising can be extremely effective when it's done right. Before you pitch Dead Benchmarks as a potential sponsor or partner, make sure that you've done your research and checked its newsletter stats with Reletter.
Then, personalize one of our winning pitching templates and send it to the right person using the contact info provided.
How much does it cost to sponsor a publication like Dead Benchmarks?
Newsletter ad rates (or CPM) vary depending on many factors, including industry, number of subscribers, open rate, ad placement and more.
To find out how much an ad will cost, contact Dead Benchmarks using the contact information provided and ask for a copy of their media kit.
How can I find newsletters related to Dead Benchmarks?
Scroll up to where it says Related Newsletters to see other publications like Dead Benchmarks. You can also search our email newsletter directory to discover other newsletters that cover the topics you're interested in.
How do I contact Dead Benchmarks?
Reletter provides this newsletter's website URL above, where you will often find their contact information. We also provide links to associated social media accounts and pitching templates so you can reach out fast.