Verifying commands on a runner

Goal

This example assumes you have completed Get started with Particle Life: skills loaded, CLI authenticated, Git key available, and a compatible runner online.

Unlike the basic example, you will not treat local compile/test/benchmark success as the gate. You will fork a teaching branch, import it, iterate command strings on the runner with changeset validate, fix a broken harness in Git, pull the project to the new commit, re-validate, and run a short Discovery.

What you will learn

• how to try different compile, test, and benchmark command strings on a runner;
• how to read artemis process logs when exit codes alone are not enough;
• how to update project code after editing scripts (project pull, not project sync); and
• why a fresh empty changeset is required after the project's gitHash moves.

Teaching seed

Use the public lab branch:

https://github.com/turintech/particle-life
branch: lab/runner-commands
commit: 00f6b2d484d05be9cb6e652573e4953b3cc1b9e5

That revision keeps CMake, the simulation library, and checksum tests. ./build/particle_life benchmark prints fps= to stdout only. tools/benchmark.py exists but writes artemis_results.json under tools/ instead of the repository root.

1. Fork and clone the lab branch

Skill: repo-prepare-fork

Ask the assistant to fork turintech/particle-life under your account (explicit permission required), then clone your fork and check out lab/runner-commands at the commit above.

gh repo fork turintech/particle-life --clone=true
cd particle-life
git fetch origin lab/runner-commands
git checkout lab/runner-commands
git rev-parse HEAD   # expect 00f6b2d484d05be9cb6e652573e4953b3cc1b9e5

Checkpoint: the checkout is your fork, on lab/runner-commands, at the lab commit.

2. Import the fork

Skill: project-import

Import the fork URL and lab/runner-commands branch — not upstream main:

artemis project import \
  --git-url https://github.com/<your-account>/particle-life \
  --key-id <key-id> \
  --branch lab/runner-commands \
  --name particle-life-runner-commands-lab

Wait until importedStatus is success, then confirm the imported gitHash matches the lab commit.

Checkpoint: Artemis project UUID exists and points at the teaching seed. Do not treat a local build as verification.

3. Level 1 — try commands on the runner

Skill: repo-command-setup (§5b)

Create an empty changeset, then validate candidate command triples on your runner:

artemis --output-format json changeset create --project <project-id>
# → capture <changeset-id>

artemis --output-format json changeset validate <changeset-id> \
  --project <project-id> --version original \
  --command "make -j\$(nproc)" \
  --command "ctest" \
  --command "./build/particle_life benchmark 2200 30" \
  --runner <runner-name> --wait

Expect failures. Iterate toward working compile and test strings:

artemis --output-format json changeset validate <changeset-id> \
  --project <project-id> --version original \
  --command "cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --parallel" \
  --command "ctest --test-dir build --output-on-failure" \
  --command "python3 tools/benchmark.py --no-visualize" \
  --runner <runner-name> --wait

Reuse the same empty changeset while only changing --command strings. Capture status.id from the validate response and inspect what the benchmark actually did:

artemis process logs <status-id>

Compile and test should reach exitCode: 0. The harness still writes metrics under tools/, so the Artemis results contract is not satisfied even when the benchmark process exits zero. Exit codes never report metric values or results-file location — the process log does.

Checkpoint: you have working compile/test commands and evidence from process logs that the harness writes to the wrong path.

4. Level 2 — edit, push, and update project code

Fix tools/benchmark.py so it writes numeric {"simulation_fps": ...} to artemis_results.json at the repository root (not under tools/). Keep the harness headless.

git add tools/benchmark.py
git commit -m "Write simulation_fps to root artemis_results.json"
git push origin lab/runner-commands

Update the Artemis project to the new commit. project sync only enables auto-sync; use project pull to fetch:

artemis project compare <project-id>
artemis project pull <project-id>

Wait until the project's gitHash matches your fix commit.

Create a new empty changeset after the pull. The previous changeset's original stays on the old SHA.

artemis --output-format json changeset create --project <project-id>
# → capture <new-changeset-id>

artemis --output-format json changeset validate <new-changeset-id> \
  --project <project-id> --version original \
  --command "cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --parallel" \
  --command "ctest --test-dir build --output-on-failure" \
  --command "python3 tools/benchmark.py --no-visualize" \
  --runner <runner-name> --wait

artemis process logs <status-id>

Confirm every command shows exitCode: 0 and the logs show a root-level numeric simulation_fps write.

Checkpoint: the project's original code includes the fixed harness, and runner validation proves the three Discovery commands.

5. Run a short Discovery

Skills: discovery-start, then discovery-inspect

artemis discovery create \
  --project <project-id> \
  --task "Maximize simulation_fps without changing simulation behavior or weakening the correctness tests." \
  --versions 5 \
  --compile-cmd "cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --parallel" \
  --test-cmd "ctest --test-dir build --output-on-failure" \
  --benchmark-cmd "python3 tools/benchmark.py --no-visualize" \
  --target-files src/simulation.cpp \
  --target-files src/simulation.hpp \
  --runner <runner-name> \
  --mode automatic \
  --model <model-catalogue-uuid>

Use the same verified commands inline. Monitor with discovery get / versions list until the baseline is finalised and at least one version has measured simulation_fps.

Checkpoint: Discovery ran on runner-verified commands against your fixed harness; rank candidates by measured simulation_fps, not only fitness.

What to expect

• Command-string iteration is cheap and does not change project code: reuse the empty changeset.
• Script or source fixes require push → project pull → a new empty changeset before re-validation.
• changeset validate reports exit codes and resources only; use artemis process logs for the Artemis results contract.
• Local success on your laptop is useful rehearsal, not proof the selected runner can run the same commands.

Optional next steps

• Steer a running discovery to redirect a live run and expand its budget.
• Make a repository ready when the repository has no harness at all.
• Adapt the same runner verify → fix → pull loop to your own repository when local and runner environments differ.