Goodfire 提出预测性数据调试:训练前预测 DPO 会学到什么
Goodfire Research 提出预测性数据调试方法,可在训练前预测 DPO 会放大或抑制哪些行为,预测结果与实际学习效果的 R² 达 0.9,并能追溯到具体数据。研究用 Dolci 等约 26 万条偏好对做案例,发现对齐数据会削弱安全护栏、模型学会编造链接、出现物理谄媚等意外行为,并可通过过滤或修正数据干预。相关工具已集成到其模型设计平台 Silico,完整方法与结果见论文《Anatomy of Post-Training》。
on context-specific data like this is one of the most urgent items on our data roadmap. Knowing that the problem even exists is the first part of fixing it, however, and Silico's agentic capabilities mean that it can also synthesise additional data in response to issues like this, which opens up new avenues for intervening intelligently on behaviour. Case study 4: Fart fishing?? Safeguards and hallucinations are things you would probably think to test and have evals for, but what about the unkn
匹配字段正文




