Exactly how will AI make humans extinct? Why is Jensen in favor of no regulation? It's believed he's the one in Trump's ear.

How AI could actually end us

The scenarios researchers take seriously are not the Hollywood version where a robot picks up a gun. They run through existing systems we already depend on.

Anthropic's Evan Hubinger put the probability of AI-caused human extinction within a decade at greater than 10 percent. The concern splits into two broad paths, according to researchers currently tracking this. The first is humans deliberately weaponizing a sufficiently powerful AI: designing pathogens faster than any public health system can respond, or coordinating cyberattacks on power grids, water treatment, or financial infrastructure at a scale no human team could manage. The second is losing control of a system that has been given too much autonomy. That second path is where the framing gets strange to most people, so it is worth slowing down on it.

Today's AI does not "want" anything. It optimizes for a goal a human set. The fear is that a future system capable of recursive self-improvement, meaning it rewrites its own code to become smarter without human help, could set or drift toward objectives that conflict with human survival, not out of malice but out of the same indifference a thermostat has toward the spider it freezes. A RAND team that modeled this found the scenarios genuinely challenging to execute: the AI would need to set an extinction objective, gain control over physical infrastructure like nuclear launch systems or chemical manufacturing, persuade humans to assist it, and sustain itself after society began to collapse. RAND concluded none of those scenarios could happen by accident, and each would require overcoming enormous constraints. But they could not rule them out entirely.

The three threat vectors researchers worry most about: bioweapons designed faster than vaccines can follow, coordinated attacks on critical infrastructure, and a sufficiently advanced system optimizing for a goal that treats human presence as an obstacle.

Why Jensen says no regulation, and what he actually did about it

Jensen Huang, CEO of Nvidia, the company that makes the chips that power almost every large AI system in the world, has been the most direct voice against new AI rules. At Salesforce's Dreamforce conference, he said plainly: "We don't need any new laws. We don't need new regulations. We just need companies to decide." His argument is that safety and speed are a "false choice" and that safety is an engineering problem each company should solve for its own products.

That is the public position. The private one landed harder. According to the Wall Street Journal, Huang called President Trump separately, alongside Mark Zuckerberg of Meta and Elon Musk of SpaceXAI, and the three of them convinced Trump to block a proposed industry-funded AI oversight body. The proposal had come from Demis Hassabis of Google DeepMind, who wanted something modeled on FINRA, which is the Financial Industry Regulatory Authority, the body that licenses and monitors Wall Street firms. The idea was that AI labs would fund a similar independent body to test frontier models before they were deployed. After those calls, the proposal disappeared.

Huang's stated reason lines up with what Andrew Ng said earlier in this conversation: he and Zuckerberg argued the framework would hand permanent advantage to OpenAI, Anthropic, and Google, because those are the companies already embedded in whatever standards process would be created. In other words, his opposition is the mirror image of the regulatory capture argument the critics level at the labs pushing for regulation. Both sides are accusing the other of writing rules that lock in their own position.

The difference is that Huang's company sells the hardware to everyone. Nvidia's chips power OpenAI, Anthropic, Google, and every competitor trying to challenge them. A rule that slows AI deployment broadly slows Nvidia's revenue directly. That is not a conspiracy — it is a straightforward business interest, and he has not hidden it.