This month I did not read as many papers as I should have, I have quite a big backlog to catch up on. Nevertheless, I still found a few interesting ones that I wanted to share. The theme that emerged for me this month was a contrast between two different scales of thinking about AI. On the one hand, there’s a more broad, scenario-based approach to forecasting how AI might change the world in the next few years. On the other hand, there’s a more zoomed-in, empirical approach to understanding how specific AI systems behave and how we can train and evaluate them.

Papers I Read in November 2025

Papers I Found Interesting in November 2025

AI 2027

Most writing about AI futures has a predictable failure mode: it waves at big changes without specifying what actually happens. “AGI will arrive!” “Society will transform!” But when? Who does what? Under what constraints?

AI 2027 does something different. It commits to a timeline. Not just “AGI soon” but “here’s what happens in March 2027, here’s what happens in July, here’s who races whom and why.” That makes it useful in a way most forecasts aren’t—because the moment you get specific, people can point to exactly where they disagree.

The project came out of Daniel Kokotajlo’s team (he’s the former OpenAI researcher who quit dramatically and wrote “What 2026 Looks Like” back in 2021, which turned out remarkably accurate). They ran tabletop exercises with researchers from the major AI labs, fed in trend extrapolations, and tried to answer: if coding agents get good enough to automate AI research itself, what happens next?

Their answer: gradual improvement through 2025 and 2026, then agents start meaningfully contributing to AI R&D in 2027, which triggers an intelligence explosion that blows through human-level capabilities by mid-2027 and hits superintelligence by early 2028. The US government wakes up somewhere in early 2027—probably after seeing what AI can do to cyberwarfare—and starts pulling AI companies under tighter control.

What I liked most wasn’t whether the scenario is “right” (they openly say their median has already shifted from 2027 to 2028, and other team members think it’ll take longer). It’s that the scenario format forces you to think through mechanisms. Is your disagreement about compute trends? About how fast research automation compounds? About whether the government can coordinate or whether companies race? These become legible arguments instead of vibes.

The other useful move: they’re explicit that this isn’t a manifesto. It’s “our best guess informed by exercises and data, and we want you to write alternatives.” That framing matters because it invites the right response—not “do I like this?” but “do I think it’s likely, and why not?”

My takeaway is that scenario forecasts sit in a useful middle layer between vibes and formal models. They’re not proof. But they expose hidden assumptions about incentives and coordination before you find out the hard way that you were wrong.

The Vulnerable World Hypothesis (Bostrom, 2019)

Bostrom’s core question is deceptively simple: what if progress makes it easier to destroy civilization than to defend it?

He uses a memorable urn metaphor. Imagine humanity drawing balls from an urn where each ball represents a technology. Most balls are white (beneficial tech). Some are gray (mixed effects). And a few might be black—technologies that, once discovered and widely accessible, reliably end civilization. The scary part isn’t that black balls must exist. It’s that we might not notice we’re approaching one until it’s too late to build defenses.

The hypothesis is less about any specific doomsday device and more about a structural problem: when destructive capability becomes cheap, scalable, and hard to monitor, normal social arrangements stop working. If it gets cheap enough for someone, somewhere to blow everything up—whether through malice, mistake, or desperation—then the default condition (lots of independent actors, imperfect enforcement) becomes unstable.

What makes this feel relevant now: advanced AI could be a general-purpose accelerator for exactly those “cheap, scalable, hard-to-monitor” capabilities. You don’t need a single black ball if you have a tool that massively lowers the barrier to designing dangerous things, scaling them, or coordinating around them.

Reading this alongside AI 2027 made the connection obvious. Even if you don’t buy the specific 2027 timeline, the “vulnerable world” frame gives you a way to think about the governance problem without assuming specific catastrophes. It’s more about the fragility of “normal life” under certain technological conditions and less about predicting which exact disaster will strike.

Bostrom walks through potential solutions and rejects most of them. Slowing down? Only works if everyone slows down. Better monitoring? Not sufficient on its own. He ends up pointing toward two controversial options: either massively amplified global surveillance (preventive policing at scale) or much stronger global governance. Neither is appealing. But that’s kind of the point—if the hypothesis is true, we’re facing choices we’d rather not make.

I personally doubt that the goverments of the world will come together and implement the kind of global governance Bostrom suggests—at least not until after a major catastrophe which forces their hand. The core issue is that you need unanimous agreement to slow down or regulate dangerous technologies, and that is very hard to achieve when different actors have different incentives.