Anthropic Prospectus Discloses Models Already Close to Having RSI Capabilities

Article body

Although market attention has mostly focused on the financial figures disclosed in Anthropic's newly revised S-1A filing (revenue has soared, but so has spending), the AI lab has dropped another small bombshell in the fine print—the company is very close to producing a model with RSI capabilities.

Recursive self-improvement (RSI) refers to the process in which an AI system uses the results of its own improvements as input, continuously optimizing its own capabilities and modifying its own architecture, forming a self-improvement loop. It is regarded as a key path toward achieving artificial general intelligence (AGI). Simply put, it means an AI system can design and develop its subsequent versions without human intervention.

Although RSI has not yet been achieved, as AI systems become increasingly complex and pervasive on the internet, foundation model builders may lose control of the underlying technology or fail to account for potential unintended consequences. Several companies, including Anthropic and OpenAI, have previously warned that this would make it easier for humans to lose control of these systems.

To AI doomers, RSI represents an irreversible catastrophe: AI builds itself strong enough to exceed human control and is indifferent to the damage it causes in pursuit of its goals. But to AI supporters, RSI will be a tremendous tool for boosting productivity, scientific discovery and more.

Judging from Anthropic's wording, it seems to fall somewhere in between.

The company noted in the aforementioned filing that it makes extensive use of AI in research and development, but also cautioned: "Using AI to help build more capable AI could also carry considerable safety risks, especially if it sharply accelerates the pace of progress." Such risks are monitored under the company's "Responsible Scaling Policy" (RSP).

"The policy sets out our plans for how we will monitor and mitigate risk as we advance what is known as 'recursive self-improvement,'" the filing added.

Earlier this year, when the term RSI became popular, many people regarded it as merely a buzzword, or at best a goal that is difficult to achieve, define or prove.

But on Monday, Anthropic added new statements to its "Risk Factors" section, saying: 1) it has already used AI to develop new models, but at the time it did not realize these models had made a leap in capability; 2) it may soon meet its internal definition of RSI:

In addition, we increasingly use AI systems to accelerate our own research and development, which could lead to model capabilities improving faster than our ability to develop, validate and implement corresponding safety, alignment and safeguard measures. As a result, we failed to notice this in the past, and may also notice it only after the pace of R&D has significantly accelerated in the future.

For example, our internal evaluations and expectations for future models (rather than current models) indicate that we may soon reach the definition of "significant acceleration of development driven by AI" in our "Responsible Scaling Policy." We have not yet identified any current model as exceeding this threshold, and we will continue to evaluate the relevant data.

However, Anthropic also said in its filing that no model currently crosses this line, and pledged to continue keeping humans involved in the R&D process for the foreseeable future.

"We expect that, even if technically feasible, we would not be able to safely use Claude to fully autonomously design and build the next generation of frontier models without effective human oversight or understanding. Therefore, we are considering updating our RSP to ensure humans always remain in control of the development of Anthropic's frontier models," the filing said.

(Source: Cailian Press) Mid-article embedded advertisement