mirror of
https://github.com/Fission-AI/OpenSpec.git
synced 2026-10-02 05:24:34 +08:00
skill-templates-parity.test.ts pins a SHA-256 per workflow template so an
unintended template edit fails loudly. The cost lands on every intended
edit: the pinned hashes go stale, and because all 37 live in two maps in
one file, two branches editing different templates collide there on rebase.
Resolving that means hand-editing 64-character hashes, which is where
transcription mistakes come from - and the test proves a hash matches its
source, never that the source is right, so a bad value regenerated over a
bad merge passes CI in silence.
Recompute every pinned hash from the built dist/ and rewrite the map in
place, reporting which entries moved. The skill-directory mapping comes
from getSkillTemplates(), the same helper the skills.sh generator uses, so
adding a workflow needs no second list here; function labels resolve
dynamically against the module exports, so there is no hard-coded list at
all.
"Nothing to update" has to mean it, so four things abort the run without
writing:
- dist/ missing or older than src/, which would pin hashes from a stale
build that the parity test - which reads src/ - then rejects
- a pinned label with no matching export, from a renamed or deleted
template
- a pinned hash whose line the patterns do not recognise, counted by
comparing 64-hex literals found against literals rewritten; the count
uses a deliberately broader pattern so it is a real cross-check rather
than a restatement of the same patterns
- a skill the registry deploys that nothing pins, compared in the other
direction: pins-to-registry only sees pins that already exist
That last direction closes a hole that predates this script. A workflow
added to getSkillTemplates() but never pinned was invisible to the parity
test too, which compares only the entries it already lists - so it shipped
with no golden hash while everything reported success. skill-templates-
parity.test.ts now pins the registry itself, so CI catches it whether or
not anyone runs this script.
The rewriting lives in parity-hash-shared.mjs, following the split between
generate-skillssh.mjs and skillssh-shared.mjs, so those guards can be
exercised against fabricated input. Running the script for real from a test
would rewrite the repository's own parity test file mid-suite. Each case in
parity-hash-shared.test.ts was mutation-checked: removing the guard it
covers makes it fail.
The script cannot silently emit a wrong hash: the parity test recomputes
the same values independently and compares, so a drift between the two
copies of stableStringify fails the test. The test stays the authority.
Dev tooling only. scripts/ is not published (package.json files ships just
scripts/postinstall.js), no src/ is touched, and no runtime behaviour
changes - hence no changeset.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
116 lines
4.8 KiB
JavaScript
116 lines
4.8 KiB
JavaScript
#!/usr/bin/env node
|
|
|
|
/**
|
|
* Regenerate the golden hashes in `test/core/templates/skill-templates-parity.test.ts`.
|
|
*
|
|
* That test pins a SHA-256 per template so an unintended edit to any workflow
|
|
* template fails loudly. The flip side is that every *intended* edit leaves the
|
|
* pinned hashes stale, and two branches editing different templates collide on
|
|
* the same hash map — so a rebase means recomputing them by hand, which is
|
|
* where transcription mistakes creep in.
|
|
*
|
|
* This script recomputes every pinned hash from the built `dist/` and rewrites
|
|
* the map in place, reporting exactly which entries moved.
|
|
*
|
|
* Three things are hard errors rather than silent skips, because "nothing to
|
|
* update" has to mean it:
|
|
* - a `dist/` older than `src/`, which would pin hashes from a stale build
|
|
* that the parity test (which reads `src/`) then rejects
|
|
* - a pinned label with no matching export (a renamed or deleted template)
|
|
* - a pinned hash whose line the patterns do not recognise, which would
|
|
* otherwise be left stale while the run reported success
|
|
*
|
|
* The last two live in `parity-hash-shared.mjs` so they can be exercised against
|
|
* fabricated input; see `test/core/templates/parity-hash-shared.test.ts`.
|
|
*
|
|
* It cannot silently produce wrong hashes: the parity test recomputes them
|
|
* independently and compares. If `stableStringify` ever drifted from the test's
|
|
* copy, the test fails. Always run the test afterwards - that check, not this
|
|
* script, is the authority.
|
|
*
|
|
* Usage:
|
|
* pnpm build && pnpm regen:parity-hashes && pnpm vitest run test/core/templates/skill-templates-parity.test.ts
|
|
*/
|
|
|
|
import { createHash } from 'node:crypto';
|
|
import { readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs';
|
|
import { dirname, join } from 'node:path';
|
|
import { fileURLToPath, pathToFileURL } from 'node:url';
|
|
|
|
import { rewriteParityHashes, stableStringify } from './parity-hash-shared.mjs';
|
|
|
|
const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '..');
|
|
const distUrl = (p) => pathToFileURL(join(repoRoot, 'dist', p)).href;
|
|
|
|
/** Newest mtime under a directory, or -1 if it does not exist. */
|
|
function newestMtime(dir) {
|
|
let newest = -1;
|
|
let entries;
|
|
try {
|
|
entries = readdirSync(dir, { withFileTypes: true });
|
|
} catch {
|
|
return newest;
|
|
}
|
|
for (const entry of entries) {
|
|
if (entry.name === 'node_modules' || entry.name.startsWith('.')) continue;
|
|
const full = join(dir, entry.name);
|
|
const mtime = entry.isDirectory() ? newestMtime(full) : statSync(full).mtimeMs;
|
|
if (mtime > newest) newest = mtime;
|
|
}
|
|
return newest;
|
|
}
|
|
|
|
// Hashes are computed from dist/, but the parity test recomputes them from
|
|
// src/. Regenerating against a stale build therefore writes hashes the test
|
|
// then rejects, after reporting "nothing to update" - a false all-clear on the
|
|
// most common mistake there is, forgetting to build. Refuse to guess.
|
|
const srcMtime = newestMtime(join(repoRoot, 'src'));
|
|
const distMtime = newestMtime(join(repoRoot, 'dist'));
|
|
if (distMtime < 0) {
|
|
throw new Error('dist/ is missing. Run `pnpm build` first - hashes are computed from the build.');
|
|
}
|
|
if (srcMtime > distMtime) {
|
|
throw new Error(
|
|
'dist/ is older than src/, so the hashes would be computed from a stale build\n' +
|
|
'and the parity test - which reads src/ - would reject them. Run `pnpm build` first.'
|
|
);
|
|
}
|
|
|
|
const templates = await import(distUrl('core/templates/skill-templates.js'));
|
|
const { getSkillTemplates, generateSkillContent } = await import(
|
|
distUrl('core/shared/skill-generation.js')
|
|
);
|
|
|
|
const TEST_FILE = join(repoRoot, 'test/core/templates/skill-templates-parity.test.ts');
|
|
|
|
const sha256 = (value) => createHash('sha256').update(value).digest('hex');
|
|
|
|
// The generated-content hashes are keyed by skill directory. Read that mapping
|
|
// from the same production helper the skills.sh generator uses, so a new
|
|
// workflow never needs a second list kept in sync here.
|
|
const PARITY_BASELINE = 'PARITY-BASELINE';
|
|
const contentByDir = new Map(
|
|
getSkillTemplates().map(({ dirName, template }) => [
|
|
dirName,
|
|
sha256(generateSkillContent(template, PARITY_BASELINE)),
|
|
])
|
|
);
|
|
|
|
const { source, moved } = rewriteParityHashes(readFileSync(TEST_FILE, 'utf-8'), {
|
|
resolveFunctionHash: (name) =>
|
|
typeof templates[name] === 'function' ? sha256(stableStringify(templates[name]())) : undefined,
|
|
resolveContentHash: (dirName) => contentByDir.get(dirName),
|
|
knownContentKeys: contentByDir.keys(),
|
|
sourceLabel: TEST_FILE,
|
|
});
|
|
|
|
writeFileSync(TEST_FILE, source);
|
|
|
|
if (moved.length === 0) {
|
|
console.log('Parity hashes already match the build - nothing to update.');
|
|
} else {
|
|
console.log(`Updated ${moved.length} parity hash(es):`);
|
|
for (const name of moved) console.log(` ${name}`);
|
|
}
|
|
console.log('\nNow run: pnpm vitest run test/core/templates/skill-templates-parity.test.ts');
|