▶ ai·
Open-world evaluations for measuring frontier AI capabilities
Introducing CRUX, a new project for evaluating AI on long, messy tasks
items20 latest
▶ ai·
New Paper: Towards a science of AI agent reliability
Quantifying the capability-reliability gap
▶ ai·
AI Won’t Automatically Make Legal Services Cheaper
Applying the AI as Normal Technology framework to legal services
▶ ai·
Fact checking Moravec's paradox
This famous aphorism is neither true nor useful
▶ ai·
A guide to understanding AI as normal technology
And a big change for this newsletter
▶ ai·
Could AI slow science?
Confronting the production-progress paradox
▶ ai·
AGI is not a milestone
There is no capability threshold that will lead to sudden impacts
▶ ai·
AI as Normal Technology
A new paper that we will expand into our next book
▶ ai·
Is AI progress slowing down?
Making sense of recent technology trends and claims
▶ ai·
We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.
Technology Isn’t the Problem—or the Solution.
▶ ai·
Does the UK’s liver transplant matching algorithm systematically exclude younger patients?
Seemingly minor technical decisions can have life-or-death effects
▶ ai·
FAQ about the book and our writing process
What's in the book and how we wrote it
▶ ai·
Can AI automate computational reproducibility?
A new benchmark to measure the impact of AI on improving science
▶ ai·
Start reading the AI Snake Oil book online
The book was published September 2024
▶ ai·
AI companies are pivoting from creating gods to building products. Good.
Turning models into products runs into five challenges
▶ ai·
AI existential risk probabilities are too unreliable to inform policy
How speculation gets laundered through pseudo-quantification
▶ ai·
New paper: AI agents that matter
Rethinking AI agent benchmarking and evaluation
▶ ai·
AI scaling myths
Scaling will run out. The question is when.
▶ ai·
Scientists should use AI as a tool, not an oracle
How AI hype leads to flawed research that fuels more hype
▶ ai·
AI leaderboards are no longer useful. It's time to switch to Pareto curves.
What spending $2,000 can tell us about evaluating AI agents