Skip to main content

Verify a change

Your coding agent has changed some code and says it is faster. Verifying the change with Artemis means measuring that claim on your own hardware before you merge it.

What happens​

The change goes into an Artemis Branch, next to the Baseline, your unchanged code. Your Runner runs a Validation of each one: it builds the code, runs your tests, and runs the same benchmark on each, several times. Artemis then reports every Metric for both, with the spread across runs and the change against the Baseline.

The benchmark and tests are the Script saved in your Project settings, so your agent cannot adjust them to make its own change look better.

Reading the result​

A change is worth keeping when it builds, passes the tests, and improves the Metric you care about by more than the run-to-run noise, at a cost to the other Metrics you can accept. If the difference is inside the noise, the result is inconclusive. Run more repeats, or treat the change as no faster.

A gain is only as good as the benchmark's workload. If the workload is unlike real use, the gain may not show up in production.

How to ask for it​

In Claude Code with the Artemis skills, ask in plain words, for example "have Artemis check whether this change is faster on my runner". Other agents can do the same through the Artemis CLI. You need a Runner online and selected, and a saved Script with build, test and benchmark Commands.