source: .claude/rules/test-record.md · generated — edit the markdown, not this file

The Test Record

The detail behind .claude/rules/git-conventions.md § The test suites run locally, and the PR records them, which keeps the rules every session needs. Read this file before running a local record or the browser gate, marking an app-repo PR ready, or writing a wait loop for a heavy run. A bare § reference below (§ Branching) is a section of git-conventions.md.

No app repo’s CI runs its test suite on a pull request: not forge-et-be’s suite (every Spock spec and the ArchUnit rules), and not the unit and component tests (Vitest) of forge-et-fe and forge-et-pwa. The suites were most of the organisation’s GitHub Actions minutes, and when those run out every workflow stops, deployments included. A suite runs in one place, never in two: CI runs them in each app repo’s Slow lane job only on demand (gh workflow run "PR validation" --ref <branch>, main included; on main, -f tests=false runs the Fast lane only), and every PR, a Dependabot PR included, a push to release/* and the weekly schedule skip it. The nightly full run — every suite of the three app repos, the browser gates included, plus every check the Fast lane runs on main except those that read a pull request and the wrapper validation (the frontends’ static checks and build, the backend’s guard scripts, the bundle and jar fixture checks, the record scripts’ self-tests; N-5 lists them) and a secret and dependency scan of every repo — runs on main every night at 02:00 Sri Lanka time on the lead’s Mac (.claude/lib/nightly/run-nightly.py, .forge/design/harness/test-run-cost.md N-3). The scheduled CI run is weekly (Sunday 19:30 UTC), runs the Fast lane only, and skips a week main did not move; it is kept to refresh main’s caches, which every PR restores. § Branching relies on the nightly full run after a clean merge. It is the only full run of the browser suites for a PR into main (.claude/CLAUDE.md § Quality Gates › Browser QA); before a frontend PR into main is marked ready, the developer runs only the browser tests its change selects, pnpm e2e:gate --selected, after the record (.forge/design/harness/fewer-nightly-reds.md R-1). It is the only run of every test: a PR’s record runs the tests its change needs (below). A problem it finds is fixed the next day, with the others that night found.

The developer records the tests once per PR, on its final commit, after the last push, and a script records it: it adds (or updates) one PR comment naming the commit and the test counts, and sets the Test record commit status on that commit, linking to the comment — the local record. The record runs the tests the change needs, chosen by the rule pnpm test:selected and python3 scripts/select-tests.py apply since the merge base with the PR’s base branch (harness design .forge/design/harness/test-run-cost.md S-1, S-3, S-4): the changed code’s tests and every test that depends on it, plus the tests that read files as text. It runs every test when the change touches shared code (each repo’s scripts/shared-code.json), on a PR into release/* (which the nightly full run never tests, and whose push runs no suite in CI), or with --full; the comment says which ran and why. forge-et-be’s record still runs the whole ./gradlew build — compile, static checks and the ArchUnit rules — with only the test set narrowed. Test record is a required check in all three app repos, so a PR whose head has none cannot merge, and a new commit starts without one. The script refuses a dirty working tree and a commit that is not the PR’s head on GitHub (the status needs a commit GitHub has, so push first), and records nothing when a test fails. Why: a required check on the head asks for one run, on whatever head is about to merge; running every test on every PR repeated what the nightly run checks on main every night (the lead, 2026-11-02).

The records and the browser gate refuse to start on an overloaded machine. Before running anything, each samples the CPU for five seconds and refuses while other work keeps it more than 0.65 busy, naming the busiest processes; ET_LOAD_LIMIT sets another limit. While it runs it records the machine’s load (in the record comment, the gate’s result, or on the terminal when it fails), which is what a flaky failure is triaged from (.forge/test-strategy.md § Flaky Tests). --ignore-load runs anyway and the record says so. Claude passes --ignore-load only when the developer says so in that session. Why: a run on an overloaded laptop fails with timeouts that are not in the code (BUG-194, BUG-195), and an agent that is refused would otherwise add the flag and repeat the problem (harness design .forge/design/harness/load-check-and-flaky-tests.md, D-1 and D-2).

One heavy run at a time on a machine, kept awake. Every heavy run on the machine takes one lock, et-heavy-run.lock in the user’s temp folder, whoever starts it: the browser gate and both records take it themselves, as does the nightly full run and a direct playwright test (its global setup, e2e/support/heavyRunLock.ts), and every other heavy command runs through python3 <harness>/.claude/lib/heavy-run.py -- <command> (<harness> the harness checkout; it takes the same lock, runs the command and releases the lock when the command ends, whatever its exit). A heavy command is a ./gradlew build, check or test (targeted runs included), a vitest run over more than one file, and anything run for a timing measurement. Run Gradle under it with --no-daemon, so nothing is left running when it ends. A backend a test run needs is started inside the same wrapper call as that run — one script that starts bootRun, waits for it, runs the suite and stops the backend, whatever the suite’s exit — because a bootRun wrapped on its own holds the lock for as long as the server runs, and the suite that needs it then waits for ever. A suite that takes the lock itself (the browser gate, a direct playwright test) may instead have its backend started unwrapped beside it, and stopped when it ends. A backend left running for a person’s use (a manual walk) is not wrapped; stop it before the next heavy run. A session never makes a lock of its own (a folder per ticket or per session) in place of this one: other sessions cannot know to look for it. A second run waits for the first, naming it, and starts when it ends; --no-wait refuses instead. A run started from inside a heavy run (a record the wrapper starts) sees ET_HEAVY_RUN_LOCK_HOLDER and runs without waiting. The nightly full run has the next turn: while it waits it writes et-heavy-run.next beside the lock, and every other run leaves a free lock to it and waits (BUG-521: on 2026-11-21 sessions passed the lock from one run to the next all through its hour, and the night ran nothing). So a session’s run started after 02:00 waits for the whole nightly run, about four hours; start long runs before then or after it ends. While one holds the lock it keeps the machine from sleeping (caffeinate -dims on macOS, SetThreadExecutionState on Windows). Start the second run rather than a wait loop of your own; a wait loop that is still needed (for another check) matches only the real process: it brackets one character of each pgrep -f alternative ('[e]2e:gate|[l]ocal-build-evidence'), and when the same shell starts the command after the loop, it anchors the pattern on the program that runs it (pgrep -f '[n]ode .*e2e:gate'). It never uses ps … | grep, which also matches the waiting shell’s own command line when that line names node before e2e:gate. A loop that waits on the lock itself waits while the lock file names a live process, never only while the file exists, because a run that ends without its teardown leaves the file behind: the file is JSON with the holder’s pid, so the test is [ -f "$LOCK" ] && kill -0 "$(node -e 'console.log(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).pid)' "$LOCK")". Check the loop once by hand while nothing is running (it must find nothing), and run it in the background. On a laptop keep the lid open, which no wake lock overrides, keep it on mains power for a run of more than an hour (pmset -g batt prints Now drawing from 'AC Power'; a critical battery sleeps the machine whatever its sleep settings, L-218), and on a Mac, before calling a stall a flake, check pmset -g log for a sleep in the run’s window (.forge/test-strategy.md § Flaky Tests). Why: two heavy runs together fail each other with timeouts, an idle sleep on battery stalled 20 browser tests in one gate (2026-10-22), and an unbracketed pgrep -f matches its own command line, so the wait never ends (harness design .forge/design/harness/test-run-cost.md M-5, M-2; .forge/lessons.md L-234, L-240), as does a loop on a lock file its holder left behind (.forge/specs/F32-d-list-views-accounts-leads-contacts-findings.md F-006). A heavy command run without the lock runs beside whatever holds it and slows every suite in it: on 2026-11-12 other sessions’ direct Gradle and Vitest runs kept the load average at 50–76 on 8 cores during the nightly run, and its backend build stopped at its 60-minute limit (.forge/lessons.md L-288); on 2026-11-18 a session’s verify backend and Gradle build ran beside another session’s full build, which took about 5 hours instead of about 85 minutes, while the sessions had invented per-ticket lock folders that only some of them checked (L-315).

A sub-agent waits for the lock in the foreground. It runs the wrapper (or the record) as an ordinary foreground command, or detached as # Long commands in the agent prompt template says, polling an exit file — detached whenever the wait and the run together may pass the Bash tool’s 10-minute limit; never a background wait loop that ends its turn, whose report then never comes (.forge/lessons.md L-309). On macOS and Linux a stop by signal (SIGTERM, SIGHUP) ends the wrapped command and releases the lock; on Windows treat every outside stop as a hard kill. A hard kill (kill -9, taskkill /F) does neither, so the next run removes the lock once its holder’s PID is gone while the command may still be running. Before stopping anything by hand, match the running processes to the lock’s pid and started, and tell any session that was waiting.

Load started on purpose (busy loops for a load reproduction) starts only after the run that needs it holds the lock, never on the same command line before a command that waits for it. Leave the nightly’s ports free overnight: the backend it starts for each browser gate listens on 18080, its CRM gate on 15173, its capture-app gate on 15174 and its stack on 56432 and 46666; a backend, dev server or preview left on one of them costs the night a suite. Before each gate the runner waits up to 30 minutes for 18080 and names the holder when it gives up. Why: on 2026-11-13 a session’s busy loops waited in front of its own Playwright run, which was queued for the night’s lock, and kept the CPU 0.98 busy, while its backend on 18080 stopped the CRM gate from running at all (.forge/lessons.md L-293).

A hand-run frontend browser check whose result is evidence runs with E2E_GATE=1. Outside the gate, Playwright reuses whatever server answers on port 5173, and on this shared machine that is often another worktree’s pnpm dev or hand-run spec, serving that worktree’s app code; with E2E_GATE=1 it starts this worktree’s own server on 15173 and never reuses one. Before quoting a result, the run must be one that served this worktree’s code (lsof -nP -iTCP:5173 -sTCP:LISTEN names the holder’s directory). Why: a morning of BUG-385 runs tested another worktree’s app on 5173, and a state that “passed” or “failed” said nothing about the change (.forge/lessons.md L-274).

The end of a PR in each repo (the table), and the rules for drafts, the secret scan, a push after the record, review, asking CI for the suite and a failed nightly run, are in git-conventions.md § The test suites run locally, and the PR records them. The detail of two of them:

  • Keep the runs down. Push fixes in batches rather than one commit at a time; merge main in only for a reason (§ Branching); while developing, run the tests the change needs — pnpm test:selected in the frontends and python3 scripts/select-tests.py in forge-et-be, which choose them by a rule and run everything when shared code changed (harness design .forge/design/harness/test-run-cost.md S-1, S-3) — or one file (pnpm exec vitest <file>, ./gradlew test --tests '<Spec>'). None of them is the record, which sets the check after the same selection on the final head. A frontend record that fails only on timeouts under its own load can run fewer Vitest workers: VITEST_MAX_WORKERS=4 pnpm test:record, a whole number (Vitest reads 50% as 50 workers).
  • A change that only documentation no test or build step reads needs no new run. The script sets the check without running the suite when nothing it reads changed since a commit that already has a delivery-team member’s record on the PR (the * line of the base branch’s .github/CODEOWNERS; scripts/fast-lane-scope.*, the Fast lane’s documentation-only rule), naming that commit; --full runs it anyway, and a PR into release/* always runs every test. A commit that changes nothing at all (an empty commit, pushed to start a lost CI run again) counts as documentation only here. On a PR into main, the same holds across a merge of main (test-run-cost.md W-5): the script walks back from the head along first parents to a recorded commit, and names it in the check. It steps past each merge whose second parent is on origin/main and whose own paths are only such documentation, or none. A merge’s own paths are the paths git’s automatic merge left in conflict and the paths where the commit differs from that automatic merge (git merge-tree --write-tree). It also steps past each commit that changes only such documentation, or nothing. A merge that resolved a conflict in code, or made any other change to code, runs the record as before. So does, in forge-et-be, a merge that brings a migration into a branch that adds or changes one of its own. A PR into main or release/* that changes only such documentation gets the check from CI, in a job of its own after the Fast lane. A Dependabot PR gets it from a delivery-team member’s local record on its final head, after any @dependabot rebase, like any other PR (test-run-cost.md N-5, 2026-11-13).

Generated from .claude/rules/test-record.md by .claude/lib/render-docs-html.js. Markdown is canonical.