Anthropic has added a warning to its IPO filing that its use of AI to build AI may soon cross a threshold it set itself. In an amendment to its S-1 filed on Monday 2 November, the company says its internal estimates and expectations for future models “indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI’”, TechCrunch reported. The point at which AI substantially speeds up the training of its successors is often called recursive self-improvement, or RSI.

What the filing says

The new text sits in the Risk Factors section. Anthropic says its growing use of AI in its own research “may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures”, and that it has “in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred”.

It also sets limits. “We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data,” the filing says. Anthropic adds that it does not expect to be able to use Claude safely “to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding”, and that it is considering changes to its Responsible Scaling Policy “to keep humans in control of frontier model development”.

Disclosure or marketing

TechCrunch notes that an S-1 works as both disclosure and marketing, and that a claim to be close to a self-improving model could serve either purpose. Risk factors are written to protect the issuer if things go wrong, so their wording is cautious by design. What is new is that a frontier lab has put a near-term capability threshold in a document that carries legal liability.

The same amendment updated the financials to 30 September. Third-quarter revenue reached $17bn, up from $11.5bn the quarter before, while compute rose to 58% of revenue from 48%, according to TechCrunch’s reading of the filing. The company spent $10bn on compute in the quarter, including $5bn under short-term commitments that it says cost significantly more than its multi-year contracts.

What it means for founders and investors

  • Safety policy becomes a listed company’s risk. If Anthropic lists, decisions under its Responsible Scaling Policy, such as slowing a release, become events that public shareholders price. Investors should read the policy as part of the business, not an appendix.
  • Faster research cycles shorten product life. If labs are close to using AI to speed up their own development, model generations may arrive faster. Start-ups building on a specific model’s limits should plan for those limits to move.
  • Regulators will read the same sentence. A public statement that AI-accelerated research may be imminent gives lawmakers a reference point. Founders in regulated sectors should expect it to be quoted back to them. This is analysis, not investment advice.

Sources

Photo: Diagram of an artificial neural network, by Loxaxs, CC0, via Wikimedia Commons.