The paper, “Superintelligence Strategy,” posits that rapid advances in AI are poised to reshape nearly every aspect of society, but that governments see them as a means to military dominance, which will drive a “bitter race” to maximize AI capabilities.
The paper claims that the development of a “superintelligent” AI surpassing humans in nearly every domain would be the most precarious technological advancement since the atomic bomb.
The other authors are Dan Hendrycks, director of the Center for AI Safety, and Alexandr Wang, founder and CEO of Scale AI.
https://www.theregister.com/AMP/2025/03/06/schmidt_ai_superintelligence/
Main points:
Only a system where military rivals are assured self-destruction from SAGI will keep us safe (like Mutually Assured Destruction with nukes)
SAGI will benefit the economy and military who achieve it first. They would also have the ability to destroy, delay or sabotage the AI efforts of the competition if they achieve SAGI first.
The authors of the paper warn against a ‘Manhatten Project: of AI’ where the US government tries to form Super AGI in a secret facility using AI chips because it will lead to a arms race and development before we have AI safeguards in place. (ALREADY HAPPENING..?)
AI chips are incredibly dangerous and will need to be very strictly monitored and keep under surveillance to ensure no terrorists get them. Think bio weapons or nukes- very dangerous in the wrong hands!
AI agents will be able to make financial transactions and legal signatures for you. They will need to be bound to humans laws and will require an ID tied to you so that of they do something criminal, you get punished.
There is no clear way to control or limit a superintelligence. This is a big problem. They offer some ideas in the paper, but they are all theoretical and none of them are likely to contain SAGI once it is created.
This is an arms race between China and the US. Whoever gets SAGI first will likely rule the planet. Utterly.
AI is now a serious target for war, data Centers and the internet may have just become the most important target.
Tiawan is a HUGE problem for the whole world. When china takes it there will be no AI chips for the west unless they build chip plants in America. (Trump just made a HUGE deal to get TSCM to build a AI chip plant in America- this was the biggest deal for American security you didn’t hear about)
AI agents will likely commit crimes and the biggest threat to humans from AI is a ‘recursive intelligence event’ where AI find a way to improve its own intelligence in a loop- this could cause sudden superintelligence that humans have no idea how to cope with (first AI accident or disaster..?)
AI cyberattcks will be catastrophic if they occur.
The paper is hosted on Google DOCS:
https://drive.google.com/file/d/1JVPc3ObMP1L2a53T5LA1xxKXM6DAwEiC/view
It’s a long, mildly terrifying read.
h/t News Hound