Search “Search”

Matched fieldsSummary

Matched fieldsSummary

Google's John Platt introduced Empirical Research Assistance (ERA), developed by his team. It uses Gemini to maintain an experiment tree, selects and mutates notebooks in a manner similar to Monte Carlo tree search, and automates the solving of scientific problems that can be written as scoring functions. It has already produced at least ten papers. ERA underwent a qualitative leap between Gemini 2.0 and 2.5, going from completely unusable to highly effective, and has been used in climate change

Matched fieldsSummary、Body

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

In part 10 of the "How to build great evals" series, Madhu Guru argues that evaluating AI agents cannot rely only on the final answer; the steps taken to reach that answer matter too: both may arrive at 42, but one trajectory is 4 clean tool calls, while the other is 17 calls, 3 repeated searches, and recovery from 2 errors. He recommends first defining the complete workflow, breaking down each step's task, designing an evaluation method for each step, and distinguishing medium-difficulty from h

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary