Research · Evaluation
Agent evaluation
Measuring a system before it changes anything. What exists is internal: eval sets and harnesses, run offline and in continuous integration. There is no public evaluation service and no benchmark result being claimed.
Internal tooling Active development
Development status
The authoritative, dated record is build status.
Evaluation
Maturity Internal offline tooling
Availability Internal
Router eval sets and harnesses, run offline and in CI, and acceptance tests across the run and Assurance foundations. No public evaluation service and no benchmark claims.
What exists
Eval sets and harnesses for the router, covering routing decisions, answer quality and a shadow mode that records what the router would have done without applying it, run offline and in continuous integration. Acceptance tests bind the agent-run and Assurance foundations to each other. Evaluating a multi-step agent run end to end is active work.
Telemetry keeps routing decisions for evaluating the router. Routing →
What it is not
There is no public evaluation service, no benchmark result being claimed, and no automatic loop that turns an evaluation into a better model. An evaluation says where something stands; it does not by itself make a model or a system better.
Direction
From the development direction, where verification comes before coordination, and both come before anything is allowed to act on its own.
- Evaluation Internal tooling — Widen evaluation from the router to multi-step agent runs, so a change can be measured before it reaches anyone.
Recent
Research · — A skill-routing layer, with its own evaluation set. A routing layer landed in the engine together with an eval set for measuring it. Its source default is off, and in that mode it changes nothing about a response. News →