Gary Marcus sat for two interviews last week - one at Web Summit, one at Bug Bash 2026 - and made a case that's uncomfortable for anyone betting big on current AI scaling laws. His argument isn't that large language models don't work. It's that the industry is pouring billions into a strategy that hits physical and mathematical limits we're pretending don't exist.
The crux: hyperscaling assumes more compute and more data produce proportionally better models forever. Marcus argues that assumption is breaking. Models are already trained on most of the available text on the internet. Synthetic data generation creates feedback loops that degrade quality. And throwing more GPUs at the problem costs exponentially more for diminishing returns.
The Hyperscaling Bet
Every major AI lab is building bigger models. OpenAI, Google, Anthropic, Meta - the playbook is identical. Raise capital, buy more compute, train larger models, charge for API access. The bet is that scale unlocks emergent capabilities that justify the investment. And for a while, it worked. GPT-3 to GPT-4 was a genuine leap. But GPT-4 to GPT-4.5? The improvements are narrower. The cost is higher. The curve is flattening.
Marcus points to the data wall. Language models need text - lots of it - to learn patterns. The internet provided that for free. But we've scraped most of the high-quality public text already. What's left is either duplicates, low-quality content, or copyrighted material that triggers legal battles. Synthetic data - AI-generated text - fills the gap, but it introduces subtle errors that compound. Train a model on its own output and quality drifts. It's a photocopy of a photocopy problem.
Then there's the compute cost. Training runs now cost tens of millions of dollars. Inference at scale costs more. The economics only work if the models deliver step-change improvements. If the curve flattens, the maths stops working. That's the hyperscaling risk Marcus highlights: companies are locked into a capital-intensive strategy with no exit plan if scaling laws break.
The Case for World Models and Neurosymbolic AI
Marcus isn't arguing we abandon neural networks. He's arguing we stop treating them as the only tool. Language models are pattern matchers. They're brilliant at that. But they don't understand the world - they predict text. When you ask GPT to reason about physics or causality, it's guessing based on training data, not calculating from first principles.
World models are different. They encode how things actually work - objects, physics, cause and effect. A world model knows that if you drop a ball, gravity pulls it down. A language model knows that sentences about dropping balls often include the word "fall". The difference matters when you want reliability.
Neurosymbolic AI combines neural networks with symbolic reasoning. Neural nets handle pattern recognition. Symbolic systems handle logic, rules, and verification. Together, they cover more ground than either alone. It's not a new idea - researchers have been working on this for years - but it fell out of fashion when deep learning delivered such fast wins. Marcus argues it's time to revisit it.
The advantage: neurosymbolic systems can be verified. You can formally prove certain behaviours. That matters in software engineering, medical applications, autonomous systems - anywhere mistakes have real costs. Language models are probabilistic. They're likely to be right, but you can't guarantee correctness. For high-stakes tasks, that's a problem.
What This Means for Builders
If Marcus is right - if hyperscaling hits limits sooner than the market expects - then the next wave of AI progress doesn't come from bigger models. It comes from smarter architectures. Hybrid systems that combine learning with reasoning. Smaller, specialised models that do one thing reliably. Tools that verify their own output instead of guessing.
For developers, that's a different kind of opportunity. You're not competing on compute scale - you can't outspend OpenAI. You're competing on design. Building systems that use AI where it's strong and traditional software where it's not. That's feasible for small teams. It doesn't require billion-dollar training runs.
The software verification angle is worth paying attention to. Language models in the loop mean non-deterministic behaviour in production systems. That makes debugging hard and guarantees impossible. If you're building something that needs reliability - medical tools, financial systems, infrastructure - you need verification layers that current LLMs don't provide. Neurosymbolic approaches might give you that.
The Uncomfortable Question
The uncomfortable bit isn't whether Marcus is right. It's what happens to the companies and capital locked into hyperscaling if he is. Billions in infrastructure spend. Thousands of engineers optimising training pipelines. Entire business models built on API margins from ever-larger models. If the curve flattens, that's not a minor correction - it's a rethink of the entire strategy.
Marcus isn't saying AGI is impossible or that AI progress stops. He's saying the current path has obvious limits and we're ignoring them because the momentum is too strong to question. That's worth listening to, even if - especially if - you're betting big on the other side.
The full interviews are worth your time if you want the detail. Whether you agree or not, the questions are the right ones to be asking.