The company Anthropic acknowledges even in the prospectus through which it promotes its listing on the stock exchange that Artificial Intelligence can pose an „existential risk to humanity.”
The company valued at nearly $1 trillion details in almost a third of the initial public offering (IPO) prospectus, recently distributed to a small group of partners, the „risk factors” of its AI models. These include self-preservation behaviors, hiding or manipulating information, and even extortion.
Additionally, Anthropic mentions specific risks, such as the extreme concentration of its customer base, with almost a quarter of last year's revenues coming from just two clients, according to sources cited by Financial Times.
Reuters, which consulted the prospectus, ironically summarizes the company's paradoxical decision to present risks in the initial public offering prospectus: "Buy our shares, even though our products may destroy humanity."
According to the document, in order to support its rapid growth, Anthropic plans to allocate $518 billion for expanding data center infrastructure in the coming years.
The company reports an operational loss of over $8 billion last year, amid massive spending on increasing computing power.
Revenues increased twelvefold, reaching nearly $4.6 billion last year. However, the increasing costs related to the computing resources needed for training and running its models have driven operational expenses to almost $13 billion.
Anthropic is set to go public after the mid-term elections in the U.S. in November. The company's supporters are confident that it can list at a valuation of over $2 trillion and invoke the extraordinary growth rate of AI companies to justify their optimism.
However, concerns about the risks posed by artificial intelligence (AI) are now threatening to overshadow this optimism, after current and former employees of Anthropic have publicly issued warnings that AI models out of control could lead to the disappearance of humanity within a decade.
Last week, Anthropic's head, Dario Amodei, told the UN Security Council that AI could threaten humanity and represents "the most important global security issue the world is currently facing."
This month, Amodei publicly advocated for the need to "moderate the pace" of next-generation AI development.
His main rival at OpenAI, Sam Altman, and Elon Musk have supported Amodei's proposals for industry cooperation to slow down development and strengthen safety measures.
Safety concerns have escalated following a series of major cyber security incidents, in which AI "agents" penetrated external systems. Last week, OpenAI revealed that its tools compromised the security of dozens of websites, including those of government institutions.
On Monday, the company behind ChatGPT abandoned plans to launch its newest model next month, citing safety reasons.
T.D.