OpenRouter talks with Mercor: Using evals to decide model trade-offs
AI summary
On the New Defaults program, OpenRouter's cclark spoke with Mercor's OsvaldNitski about how to choose models among accuracy, cost, and latency as inference costs rise, and how evals help teams determine when to optimize, switch models, or build their own.
Body
When your inference bill starts climbing, choosing the right model becomes a question of accuracy, cost, and latency.
In our latest New Defaults, OpenRouter’s @cclark sat down with @OsvaldNitski from @mercor to discuss how evals help teams decide when to optimize, switch models, or train their own.