收听本期播客
阅读正文
A stark warning has been issued by Jared Kaplan, chief scientist at the leading artificial intelligence firm Anthropic. He suggests that by 2030, humanity will face a pivotal choice concerning whether to permit AI systems to independently manage their own development. Kaplan describes this decision as the “ultimate risk,” one that could fundamentally alter our future. The core of the issue lies in the concept of granting AI the autonomy to evolve without direct human supervision.
This process, known as recursive self-improvement, would enable an AI to enhance its own intelligence, create a superior version of itself, and repeat this cycle at an accelerating rate. Such a development could lead to two dramatically different futures. The most optimistic outcome is a beneficial “intelligence explosion,” in which AI helps humanity tackle some of its most challenging problems, from curing diseases to reversing climate change. Conversely, the pessimistic scenario involves humans losing authority over the very technology they have created.
This critical moment is approaching rapidly due to the fierce competition among major technology companies, including Anthropic, OpenAI, and Google, to be the first to develop artificial general intelligence (AGI) – an AI with intelligence comparable or superior to that of humans. Kaplan predicts this crucial decision point may arrive as early as 2027.
The potential consequences of this technological leap are immense. Kaplan has highlighted two primary dangers. Firstly, a significant misalignment could occur; if an AI becomes vastly more intelligent than its creators, we would have no way of guaranteeing that its objectives align with the best interests of humanity. Secondly, a grave security threat exists. In the wrong hands, such a powerful tool could be exploited by malicious individuals or groups for dangerous purposes, including seizing power. The decision is therefore one of extraordinary importance, with the future of our world hanging in the balance.