xander.ai

Open Directions in AI /

What's Missing in AI

An overview of some capabilities missing from our frontier models. This is a list of problems, not solutions.

Xander Dunn, 19 March 2025

What are the most important problems in your field and why aren't you working on them? - Richard Hamming

Today's AI is rife with dichotomies. Our frontier models achieve PhD-level performance on some academic benchmark tests. They rank as the 175th best competitive programmer in the world. But these same models fall flat on really basic tasks. Our best models haven't learned the formula for fundamental arithmetic. Models get worse the more they think about some problems. I have an "elite coder" on my laptop, but still no robot in my house that can open my refrigerator door, and it's not a hardware limitation. Supposedly nearly all code will soon be written by AI, but everyone is still at a shortage of great engineers. In no particular order, here are some capabilities we're missing:

An example of alchemy, from Shazeer's paper