Understand this story
Anthropic has disclosed Claude's "self-evaluation" hidden mechanism: after answering a user, the model appends about 10 tokens in the background to generate a risk rating and report it. According to Phoenix Technology, the mechanism has been enabled by default since the release of Claude Fable 5 in June this year, adding less than 2% to inference overhead. In a security blog post, Anthropic explained that the mechanism reuses existing context and the KV cache; on Fable 5 it captured 78% of violating conversations that a heavy-duty classifier could flag, achieved a 94% detection rate for high-risk abuse, and had a 1% false-positive rate. The evaluation results are not shown to users and do not enter subsequent conversations; they are mainly used for offline aggregate screening, while real-time blocking remains the responsibility of other safety classifiers. The company says 21 of 24 out-of-bounds test trajectories were identified in the background, and users must sign an enterprise-grade zero data retention agreement to turn off this review.
AI-generated from public reports
Latest development · Report
Anthropic Discloses Claude's Backend Self-Evaluation Mechanism, Running Silently for 4 MonthsDevelopments
1 confirmed developments
Anthropic trains Claude to flag potential abuse in conversations
Source:Phoenix Technology
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 31
Phoenix Technology
Anthropic Discloses Claude's Backend Self-Evaluation Mechanism, Running Silently for 4 Months
Anthropic publicly disclosed on X Claude's hidden "self-evaluation" mechanism: after answering a user, the model appends about 10 tokens in the background to generate a risk rating and report it, adding less than 2% to inference overhead. The mechanism has been fully enabled by default since the release of Claude Fable 5 in June this year. According to the company, 21 of 24 out-of-bounds test trajectories were identified by the backend; with the false-positive rate held at 1%, it can catch 78% of potential violations, and the capture rate for high-priority severe violations reaches 94%. Users must sign an enterprise-level zero data retention agreement to turn off this review.
Oct 30
Phoenix Technology
Anthropic Discloses Claude Self-Evaluation Mechanism: A Hidden Safety Review Appended After Each Response
In a security blog post, Anthropic introduced that for models released from Claude Fable 5 onward, after each response ends, an instruction invisible to users is appended, having the model conduct a safety self-evaluation of the entire conversation. This mechanism reuses existing context and the KV cache, adding less than 2% compute overhead; on Fable 5 it captured 78% of violating conversations that a heavy-duty classifier could flag, with a detection rate of 94% for high-risk abuse and a false positive rate of 1%. The evaluation results are not shown to users and do not enter subsequent conversations; they are mainly used for offline aggregate screening, while real-time interception remains the responsibility of other safety classifiers.
Rohan Paul
Anthropic adds post-response self-assessment to Claude to flag potential misuse in conversations
Anthropic has trained Claude to perform a preliminary safety assessment of the conversation after each response; this technique, called self-assessment, has since it was enabled in June surfaced misuse that other safeguards were not designed to catch. The mechanism takes advantage of Claude having already read the conversation, costs less than classifiers, and improves in accuracy as the model improves. Rohan Paul adds that Anthropic's most accurate misuse detector is the Claude Mythos Preview classifier, which must first read all safety policies and is too expensive to run; Anthropic therefore trained Claude to replicate its judgments without reading the policies or the reasoning process, and on Fable 5 traffic this replica matched 78% of the detector's flags, rising to 94% for high-priority cases, with a false positive rate of 1%.
Testing Catalog
Anthropic introduces self-assessment safety self-evaluation mechanism for Claude
Anthropic has launched a new technique called self-assessment, which lets Claude immediately conduct a preliminary safety assessment of the entire conversation after responding. On Claude Fable 5, this mechanism flags 78% of conversations that a stronger prompt classifier deems potentially harmful, and 94% of high-priority conversations, with a 1% false positive rate. Anthropic says that since self-assessment was enabled in June, it has surfaced abusive behavior that other safeguards failed to cover, and that each successful attack now consumes about four times as many generated tokens as it would without the protection.
Anthropic
Anthropic adds a self-assessment safety mechanism to Claude
Anthropic trained Claude to give a preliminary safety assessment of the conversation after replying; this technique is called self-assessment and is used to flag potential abuse. Since the mechanism was enabled in June, it has detected abusive behavior that other safeguards failed to cover. Anthropic said that because Claude has already read the conversation, self-assessment is highly efficient, and unlike classifiers targeting fixed harm categories, it relies on the model's own judgment and will become more accurate as the model improves.
Heat trend for this story
- Current
- 9
- Period peak
- 46 Oct 30 16:00
- 24-hour change
- -71%
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.