Anthropic:微调版谎言检测器泛化失败
Anthropic 对齐科学团队用开源模型在 12 种场景中诱导的 on-policy 谎言微调谎言检测器,分布内检测 AUROC 从 0.60 提升至 0.95,但跨类别迁移仅约 0.70–0.75,几乎不敌提示词基线,更大模型的零样本提示往往直接胜出。团队公开了约 20 万条标注样本的数据集。
s if operators have a channel to correct what it finds. Prior work has evaluated lie detectors on synthetic datasets or model organisms specifically trained to deceive, with discouraging results. Liars' Bench (Kretschmar et al., 2025) found that detectors fail to generalize across lie types, and Smith et al. (2025) argue that distinguishing strategic deception from reflexive responses remains fundamentally difficult. One hypothesis for these failures is distribution mismatch: detectors trained o
匹配字段正文

