Anthropic 提出 Bumpers 对齐策略:用多重防线捕捉并纠正模型失准
Anthropic 对齐科学团队发布文章,提出名为 Bumpers 的对齐策略,主张在早期 AGI 系统上优先建立多条相对独立的防线,以便在发现模型失准时及时纠偏。文章给出的流程包括预训练、微调、以机制可解释性审计和红队测试等作为主要防线、发现失准信号后定位原因并回退微调重试,以及部署后的实时监控等次级防线。作者认为该策略针对对齐可解但并非易事的中间难度场景,并指出迭代会带来选择压力、部分失准信号难以判定等局限。

ent monitoring. We believe that, even without further breakthroughs, this work we can almost entirely mitigate the risk that we unwittingly put misaligned circa-human-expert-level agents in a position where they can cause severe harm. On this view, if we’re dealing with an early AGI system that has human-expert-level capabilities in at least some key domains, our approach to alignment might look something like this: Pretraining: We start with a pretrained base model; Finetuning: We attempt to fi
匹配字段正文

