The test-impact selector has to hand the C runner individual tests, and the
map behind it has to know what each test executes. The runner could do
neither: argv selects whole suites.
Selection. CBM_TEST_ONLY (comma-separated) and CBM_TEST_ONLY_FILE (one per
line; blank lines and # comments allowed) take suite:test tokens, and are
unioned when both are set. A suite no token names never executes its body.
Inside a named suite only the named tests run; the rest are counted as
deselected and the summary reports the count. Argv still selects suites and
the tokens narrow inside that selection, so the parallel harness hands every
per-suite process the same list with no change of its own.
Tokens are strict: one that names a suite this build does not register, or a
test its suite never ran, is printed and fails the run. An unreadable file, a
malformed token, and a selection that names no test stop the run before any
suite. Without a selection the output is byte-identical to before, and
--list-suites ignores the selection.
Coverage. TEST_COVERAGE=1 compiles clang source coverage into our own code
(the one-shot sources, the LSP unity object, the preprocessor; vendored
objects stay uninstrumented) and the runner's hooks into the test runner.
`make test-runner-cov` and `make cbm-cov` build the instrumented runner and
product binary into build/cov. With CBM_TEST_COVERAGE_DIR set the runner
writes <suite>/<test>.parent.profraw for itself around each test, points
LLVM_PROFILE_FILE at <suite>/<test>.%p.profraw for the children the test
spawns, and sends what runs between tests to _setup.<n>.profraw. No other
build compiles any of it.
tests/test_harness_test_only.sh pins the selection contract against the built
runner and runs as Step 2b of scripts/test.sh, before the suites.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel
3ecac479a6