memdoor reproduces the GVS5H multi-agent coding method with a workflow
The author rebuilt GVS5H's multi-agent coding pipeline on memdoor, running 5 rounds on the first 10 LiveCodeBench Hard problems with Qwen3.8-27B, achieving an average score of 9.8, compared with 9.6 for the author's original harness and 8.0 for Fable 5 alone. 49 of 50 runs passed, at a total cost of $35.79 and 15.3M output tokens. The author notes that only 10 problems were tested rather than the paper's 100, that the problems date from March 29 to April 6, 2025, and that training data contamination cannot be ruled out, and disclosed the specific causes of the two failures.
Why read this
AI score 65
The author reproduced the paper's multi-agent coding pipeline using the same model and released all run logs and failure cases, allowing inspection of method details and costs.