NAll news

Wednesday4 items

Monday1 items

Sunday1 items

Saturday2 items

Thursday2 items

Wednesday2 items

Monday2 items

Saturday1 items

Friday2 items

Thursday1 items

Wednesday5 items

  1. Fireworks AIFirst-partyAnalysis
    AI score61

    Fireworks Evaluates DeepSeek V4 Pro Against Fable 5 and Kimi K3

    Fireworks tested DeepSeek V4 Pro 0813 on four benchmarks: SWE-Bench Verified, LiveCodeBench v6, Aider Polyglot, and Terminal-Bench 2.1. It scored 95.2% on SWE-Bench Verified, higher than Kimi K3's 92.6% and Fable 5's 85.4%, and 92.0% on LiveCodeBench v6. On cost, DeepSeek V4 Pro spent $0.309 per task solved on SWE-bench, versus $0.808 for Fable 5; on LiveCodeBench the figures were $0.040 and $0.225, respectively. Its weakness is the multilingual breadth of Aider Polyglot, where it scored only 48.9% on Java, versus 74.5% for Fable 5. After oracle routing on a combined set of 409 tasks, accuracy was 92.4%, higher than the 86.6% achieved by Fable 5 alone, at a cost of $0.279 versus $4.510 per task, with 86.3% of tasks routed to V4 Pro.

    Same story · 2 reports

    #Evaluations / benchmarks#English text is being prepared.#Agents