Arena.ai compares coding agent user criticism across 20,869 traces
Based on 20,869 traces from Agent Arena: Code, Arena.ai compared user criticism signals of five top models from Anthropic and OpenAI against other agents. The analysis found that frontier models differ most noticeably in three dimensions: 6.9 percentage points fewer complaints about failure-fix loops and 1.9 percentage points fewer about ignoring project context; 6.4 percentage points fewer design issues and 5.8 percentage points fewer issues with producing usable results; and 5.6, 4.9, and 4.5 percentage points fewer complaints about instruction following, task completion, and breaking existing code, respectively. There are currently no follow-up developments.
Based on 20,869 traces from Agent Arena: Code, Arena.ai compared user criticism signals of five top models from Anthropic and OpenAI against other agents. The analysis found that frontier models differ most noticeably in three dimensions: 6.9 percentage points fewer complaints about failure-fix loops and 1.9 percentage points fewer about ignoring project context; 6.4 percentage points fewer design issues and 5.8 percentage points fewer issues with producing usable results; and 5.6, 4.9, and 4.5
Matched fieldsSummary