Anthropic Warns of Recursive Self-Improvement Risk in S-1/A Filing
In the risk disclosures of its S-1/A prospectus supplement updated on Monday, Anthropic stated that the company may soon reach the state of recursive self-improvement as defined in its Responsible Scaling Policy. The filing said the company has already used AI to develop new models and achieve some capability jumps, but the team did not notice at the time, and may in the future only notice after an acceleration in R&D has already occurred. Anthropic also stated that none of its existing models have crossed this threshold, and committed to retaining human involvement in frontier model R&D for the foreseeable future, and is considering revising its Responsible Scaling Policy.
#Safety / alignment#Anthropic#Industry news