fix(validate): warn when tracked tasks have no checkboxes (#1774)

* fix(validate): warn when tracked tasks have no checkboxes

Progress counts checkboxes and nothing else, so a tasks.md written as
plain bullets or a numbered list is worse than an empty one: `openspec
list` and `openspec status` report "No tasks", and `openspec archive`
has no incomplete task to warn about. The file reads as finished to the
tool and unfinished to a human.

`openspec validate` now warns when every task file the change's schema
tracks contains list items but not one checkbox, pointing at the first
offending line. Reported per change, not per file, so a checklist
alongside a prose file stays silent, and only files an artifact actually
declares are linted - a bare tasks.md no schema tracks is left alone.

Closes #354

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): honor fence delimiter length and width

CommonMark closes a fence only on the same character, a run at least as
long as the opener's, and no info string. Comparing the first character
alone let an inner ``` end an outer ```` block, exposing the bullets of
a nested code sample as a task list. The delimiter pattern also loses
its end anchor: `.` does not match `\r`, so an anchored info-string
group matched nothing in a CRLF file and blinded the scan to fences.

Adds the nested-fence, annotated-closer, tilde/backtick, longer-closer
and CRLF cases, plus an e2e change whose nested task files are all
bullets, asserting both reported paths stay POSIX-separated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): scan rendered content and complete evidence only

Hardening pass over the checkbox warning.

The scan for the offending line now skips YAML front matter and HTML
comment blocks alongside fenced code. A list under `tags:` is metadata
about the file rather than the work it tracks, and a commented-out list
is not work either; each exclusion can only silence a warning, never
drop a real task, which is the opposite trade from the task parser. An
unterminated `---` opener rewinds to the top, because that is a thematic
break and everything below it is still content. Only a comment opening
its own line hides that line, so the template's `## 1. <!-- Task Group
Name -->` heading cannot swallow the checklist beneath it.

A tracked file that exists but cannot be read now withdraws the warning
entirely: "no file here holds a checkbox" is a claim about the whole
tracked set, and the checkboxes may be in exactly the file that would
not open. `validate --archived` stays the surface that reports an
unreadable task file loudly (#205).

The message leads with the consequence rather than an accusation, since
a file may legitimately carry a bulleted note and no tasks yet.

New coverage: every packaged tasks template is asserted checkbox-shaped
(the guard fails if a template loses its boxes), a schema tracking tasks
by artifact id with no `apply` block, the deprecated `change validate`
text output, and an unreadable tracked file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): match CommonMark on fence indent and front matter

Two block-scanning defects, both of which hid list items.

A fence indented four spaces is an indented code block, not an opener.
Accepting it left the scan inside a block that never began, so every
list below it went unseen. Fence recognition now stops at three spaces.

`----` is a thematic break, not a YAML front-matter delimiter. Matching
three-or-more dashes let one open a block that swallowed the list under
it until the next `---`. Front matter is now exactly three dashes.

Two test defects alongside them. The deprecated-command test claimed to
assert the reported line, but the text renderer prints no line for any
issue; it now asserts the level and path prefix that surface actually
emits, with the line left to the JSON assertion that already covers it.
The unreadable-file fixture would have passed for the wrong reason had
the mode not taken, since the checkbox it hides would have silenced the
warning by itself; the read failure is now asserted first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): name task files relative to a canonical change dir

Windows CI caught the report naming a task file
`../../../../../../../../runneradmin/AppData/.../tasks.md` instead of
`tasks.md`. `resolveArtifactOutputs` hands back real paths while
`changeDir` carries whatever spelling the caller resolved, and a short
8.3 alias against its expanded form is a difference in spelling, not in
location, so the relative path escaped the change. A symlinked project
directory reproduces it off Windows.

Canonicalizing both sides recovers the relationship. A path that still
escapes falls back to the file name, so no report can leak an absolute
filesystem path. Numbering issues are named through the same helper and
gain the same fix.

The deprecated-command test now asserts the `[WARNING] tasks.md:` prefix
that exposed this, and the Windows job is its regression guard: the
mismatch cannot be staged on POSIX, where the spawned CLI's cwd is
already physical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): keep indented code out of the uncheckboxed-task scan

alfred-openspec on #1774: the scan said it reads rendered content, but
LIST_ITEM began with \s* and so reported top-level four-space-indented code
such as '    - example output' as an uncheckboxed task list. Under --strict
that false positive failed validation on a correct file.

A list-shaped line is now taken only below four visual columns of indent, the
same cut the fence logic already applies, with tabs counting as four. Genuine
nested lists are untouched: this scan reports the first list item it finds and
a nested item always sits under a shallower parent, so the parent is reported
exactly as before. A list-shaped line four columns deep with nothing shallower
above it is not nested under anything, which is what makes it code.

Regressions cover space-indented, tab-indented and numbered code samples, and
pin both the nested-list case (parent still reported) and three-space indent
(not code). Verified the guard bites: removing the column test fails them.

Also moves the documentation to its canonical home. docs-lab/README.md says the
old docs/ tree is legacy and must stay untouched, so the docs/concepts.md line
is dropped and the warning is documented under 'openspec validate' in
docs-lab/reference/cli.md, beside the archive merge findings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(validate): skip BOM-prefixed front matter and cap ordered markers at nine digits

A prose-only task file that opened with a UTF-8 BOM before its front
matter was warned about, because the opener never matched and the
tags list was scanned. A number longer than nine digits followed by a
period also matched as a list item, which CommonMark does not allow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changeset): drop the em dash

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(tasks): use a multi-character marker for the unrecognised-checkbox case

A single-character marker such as `[~]` becomes a task once #1773 lands,
which would flip this expectation. `[ab]` is not a task under either
parser, so the test keeps asserting that checkbox-looking list items that
count as no task still warn, whichever PR merges first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Clay Good
2026-09-16 15:47:04 +00:00
committed by GitHub
co-authored by Claude Opus 5
parent b928165276
commit 09984b8242
6 changed files with 885 additions and 44 deletions
+7
View File
@@ -0,0 +1,7 @@
---
"@fission-ai/openspec": patch
---
### Bug Fixes
- **Task lists without checkboxes are now caught**: a `tasks.md` written as plain bullets or a numbered list counts as zero tasks, so `openspec list` and `openspec status` reported "No tasks" and `openspec archive` had no unfinished work to warn about. `openspec validate` now warns when a change's tracked task files contain list items but no checkbox at all, and points at the first offending line.
+12
View File
@@ -667,6 +667,18 @@ Bulk runs print one status line per item, followed by any findings, and end with
Totals: 2 passed, 0 failed (2 items)
```
**Task checkbox findings**
Progress counts checkboxes and nothing else, so a task file written as plain bullets reads as zero tasks: `openspec list` and `openspec status` report no work, and `openspec archive` has nothing to flag as incomplete. Validate reports a `WARNING` on each tracked task file that lists work without a checkbox:
```text
⚠ [WARNING] tasks.md: This change counts as 0 tasks: no line in its tracked task files is a checkbox, so "openspec list" and "openspec status" report no work and "openspec archive" has nothing to flag as incomplete. Write each task as "- [ ] 1.1 Description".
```
The warning fires only when the change's whole tracked set holds no checkbox at all. One file of prose beside a real checklist is not reported, and a change mid-authoring keeps its progress the moment a single checkbox exists. `--strict` turns the warning into a failure. The line number is in the `--json` report.
Fenced blocks, HTML comments, YAML front matter and indented code are not scanned, so a pasted terminal sample is never mistaken for a task list.
**Archive merge findings**
For changes, validate runs archive's merge builder against the current main specs without writing files. It reports merge conflicts, such as a missing `MODIFIED` target or a conflicting `ADDED` requirement, as `INFO`:
+204
View File
@@ -0,0 +1,204 @@
import { parseTaskLines } from '../../utils/task-progress.js';
export interface TaskCheckboxDocument {
path: string;
content: string;
}
export interface TaskCheckboxIssue {
path: string;
line: number;
message: string;
}
/**
* A list item that carries text but no checkbox: `- item`, `* item`, `+ item`,
* `1. item`, `1) item`. Checkbox lines match this too, so callers must rule the
* document set out on checkbox count first.
*
* Matches at any indent; the caller decides how deep is too deep. See
* `CODE_BLOCK_COLUMN`.
*
* A thematic break (`---`, `***`, `- - -`) is not a list item: the run has no
* text after it, and this pattern requires a non-space character. `* * *` is
* the one break spelled like a list of `*` items, and it is accepted as a list
* item rather than special-cased, because a file whose only list-shaped line is
* a horizontal rule still has zero tasks — the warning stays true, it just
* points at an odd line.
*
* An ordered marker runs at most nine digits, as CommonMark and the scenario
* bullet pattern in `specs-apply.ts` both require.
*/
const LIST_ITEM = /^\s*(?:[-*+]|\d{1,9}[.)])\s+\S/;
/**
* The indent at which a top-level line stops being content and becomes an
* indented code block, per CommonMark.
*
* The fence logic already treats four spaces as code; the list scan did not,
* so a sample of terminal output written as ` - example output` was
* reported as an uncheckboxed task list, and under `--strict` that false
* positive failed validation on a correct file.
*
* Genuine nested lists survive the cut, because this scan reports the *first*
* list item it finds and a nested item always sits under a shallower parent.
* That parent is what gets reported, exactly as before. A list-shaped line
* four columns deep with no shallower item above it is not nested under
* anything, which is precisely what makes it code.
*/
const CODE_BLOCK_COLUMN = 4;
/** Leading whitespace of a line in visual columns, a tab counting as four. */
function indentColumns(line: string): number {
let column = 0;
for (const char of line) {
if (char === ' ') column += 1;
else if (char === '\t') column += CODE_BLOCK_COLUMN - (column % CODE_BLOCK_COLUMN);
else break;
}
return column;
}
/**
* A fenced block delimiter: a run of three or more backticks or tildes, plus
* whatever follows it on the line (an info string on an opener, nothing on a
* valid closer).
*
* Indented by at most three spaces, as CommonMark requires: at four, the line is
* an indented code block rather than a fence, and treating it as an opener would
* leave the scan inside a block that never began and hide every list below it.
*
* Deliberately unanchored at the end: `.` does not match `\r`, so `(.*)$` would
* fail on every line of a CRLF file and blind the scan to fences entirely.
*/
const FENCE = /^ {0,3}(`{3,}|~{3,})(.*)/;
/**
* The YAML front-matter delimiter: exactly three dashes. A longer run is a
* thematic break, so `----` must not open a block that swallows the list under
* it until the next `---`.
*/
const FRONT_MATTER = /^-{3}\s*$/;
const COMMENT_OPEN = '<!--';
const COMMENT_CLOSE = '-->';
/**
* Reports tracked task files that list work as plain bullets or numbered items
* instead of checkboxes (#354).
*
* Progress counts checkboxes and nothing else, so a tasks file written as a
* bare list is worse than an empty one: `openspec list` prints "No tasks",
* `openspec status` treats the change as having no work left, and
* `openspec archive` has no incomplete task to warn about. The file looks
* finished to the tool and unfinished to the reader.
*
* Reported only when the change's *whole* tracked set has zero checkboxes.
* One nested tasks file of prose beside a real checklist is not the failure
* this catches, and a change that is mid-authoring keeps its progress the
* moment a single checkbox exists.
*
* The scan for the offending line looks at rendered content only: fenced
* blocks, HTML comments, YAML front matter and top-level indented code are
* skipped. Every one of those
* exclusions can only *silence* a warning, never drop a real task — that is the
* opposite trade from the task parser, where fence awareness would hide work
* that `archive` must still refuse, and it is why the parser stays literal
* while this scan does not.
*/
export function findMissingTaskCheckboxIssues(
documents: readonly TaskCheckboxDocument[]
): TaskCheckboxIssue[] {
if (documents.length === 0) return [];
if (documents.some((document) => parseTaskLines(document.content).length > 0)) return [];
const issues: TaskCheckboxIssue[] = [];
for (const document of documents) {
const line = findFirstListItemLine(document.content);
if (line === undefined) continue;
issues.push({
path: document.path,
line,
message:
'This change counts as 0 tasks: no line in its tracked task files is a checkbox, ' +
'so "openspec list" and "openspec status" report no work and "openspec archive" ' +
'has nothing to flag as incomplete. Write each task as "- [ ] 1.1 Description".',
});
}
return issues;
}
/** 1-based line of the first list item in rendered content, if any. */
function findFirstListItemLine(content: string): number | undefined {
const lines = content.split('\n');
let openFence: { marker: string; length: number } | undefined;
let inComment = false;
let index = skipFrontMatter(lines);
for (; index < lines.length; index++) {
const line = lines[index];
if (inComment) {
if (line.includes(COMMENT_CLOSE)) inComment = false;
continue;
}
const fence = line.match(FENCE);
if (fence) {
const marker = fence[1][0];
const length = fence[1].length;
if (openFence === undefined) {
openFence = { marker, length };
} else if (
marker === openFence.marker &&
length >= openFence.length &&
fence[2].trim() === ''
) {
// CommonMark: a closer matches its opener's character, runs at least as
// long, and carries no info string. A shorter or annotated run inside a
// block is content, so a ```` ``` ```` sample nested in a ```` ```` ````
// block does not end the block early and expose its bullets.
openFence = undefined;
}
continue;
}
if (openFence !== undefined) continue;
// Only a comment that opens the line hides it. `- [ ] 1.1 do it <!-- note`
// is a task line first, and the template's own `## 1. <!-- Task Group -->`
// must not swallow the checklist that follows it.
if (line.trimStart().startsWith(COMMENT_OPEN)) {
if (!line.includes(COMMENT_CLOSE, line.indexOf(COMMENT_OPEN) + COMMENT_OPEN.length)) {
inComment = true;
}
continue;
}
if (LIST_ITEM.test(line) && indentColumns(line) < CODE_BLOCK_COLUMN) return index + 1;
}
return undefined;
}
/**
* Index of the first line after a YAML front-matter block, or 0 when the
* document does not open with one. A list under `tags:` is metadata about the
* file, never the work it tracks, and pointing a "write checkboxes" warning at
* it would name the wrong line.
*/
function skipFrontMatter(lines: readonly string[]): number {
// A UTF-8 BOM, prepended by Windows editors and PowerShell redirects, would
// otherwise hide the opener and expose a `tags:` list to the scan.
if (lines.length === 0 || !FRONT_MATTER.test(lines[0].replace(/^/, '').trimEnd())) {
return 0;
}
for (let index = 1; index < lines.length; index++) {
if (FRONT_MATTER.test(lines[index].trimEnd())) return index + 1;
}
// An unterminated opener is a thematic break, not front matter: rewinding to
// the top keeps every list in the document visible to the scan.
return 0;
}
+115 -44
View File
@@ -34,6 +34,7 @@ import {
} from '../../utils/change-metadata.js';
import { resolveTaskFilesForChange } from '../../utils/task-progress.js';
import { findTaskNumberingIssues } from './task-numbering.js';
import { findMissingTaskCheckboxIssues } from './task-checkboxes.js';
import { findPurposePlaceholderIssue } from './purpose-placeholder.js';
import { getPackageSchemasDir, getSchemaDir } from '../artifact-graph/index.js';
@@ -479,67 +480,137 @@ export class Validator {
}
if (options.projectRoot) {
issues.push(...await this.collectTaskNumberingIssues(changeDir, options.projectRoot));
issues.push(...await this.collectTaskFileIssues(changeDir, options.projectRoot));
}
return this.createReport(issues);
}
private async collectTaskNumberingIssues(
/**
* Lints the change's task files.
*
* Two checks with deliberately different reach. Checkbox formatting is read
* from whatever the change's own schema declares as its tracked task output,
* because every schema's progress, apply and archive behavior is computed by
* counting checkboxes in exactly those files. Numbering stays scoped to the
* built-in `spec-driven` schema, whose template is the one that numbers tasks
* in the first place.
*
* The checkbox check never falls back to a bare top-level `tasks.md`: a file
* no artifact declares is not a tracked task list, and warning about its
* formatting would be a guess about a file the tool does not read.
*/
private async collectTaskFileIssues(
changeDir: string,
projectRoot: string
): Promise<ValidationIssue[]> {
let trackedFiles: string[];
try {
trackedFiles = resolveTaskFilesForChange(changeDir, projectRoot);
} catch {
return [];
}
const files = trackedFiles.length > 0 ? trackedFiles : [path.join(changeDir, 'tasks.md')];
const { documents, unreadable } = await this.readTaskDocuments(changeDir, files);
const toWarning = (issue: { path: string; line: number; message: string }): ValidationIssue => ({
level: 'WARNING',
path: issue.path,
line: issue.line,
message: issue.message,
});
const issues: ValidationIssue[] = [];
// "No file here holds a checkbox" is a claim about the whole tracked set, so
// a file that exists but could not be read withdraws it: the checkboxes may
// be in exactly that file. `validate --archived` is the surface that reports
// an unreadable task file loudly (#205); this one must not guess from it.
if (trackedFiles.length > 0 && unreadable === 0) {
issues.push(...findMissingTaskCheckboxIssues(documents).map(toWarning));
}
if (this.usesBuiltInSpecDrivenSchema(changeDir, projectRoot)) {
issues.push(...findTaskNumberingIssues(documents).map(toWarning));
}
return issues;
}
/**
* Reads task files into change-relative documents, counting the ones that
* exist but could not be read. A file that is simply absent is not counted:
* the resolver only returns files it found, so the remaining read failures
* are permissions and I/O, and a check that reasons over the whole set needs
* to know its evidence was incomplete.
*/
private async readTaskDocuments(
changeDir: string,
files: readonly string[]
): Promise<{ documents: Array<{ path: string; content: string }>; unreadable: number }> {
const documents: Array<{ path: string; content: string }> = [];
let unreadable = 0;
for (const file of files) {
let content: string;
try {
content = await fs.readFile(file, 'utf-8');
} catch (error: any) {
if (error?.code !== 'ENOENT') unreadable++;
continue;
}
documents.push({ path: this.taskDocumentPath(changeDir, file), content });
}
documents.sort((left, right) => left.path.localeCompare(right.path));
return { documents, unreadable };
}
/**
* Names a task file relative to its change, POSIX-separated.
*
* Both sides are canonicalized first. `resolveArtifactOutputs` hands back real
* paths, while `changeDir` carries whatever spelling the caller resolved, and
* the two can differ without being apart: on Windows a short 8.3 alias
* (`RUNNER~1`) against its expanded form turned `tasks.md` into a
* `../../../..`-prefixed absolute path in the report, and a symlinked project
* directory does the same elsewhere. Canonicalizing recovers the real
* relationship. The Windows CI job is the regression guard — the mismatch
* cannot be staged on POSIX, where the spawned CLI's `process.cwd()` is
* already physical.
*
* A path that still escapes would be a resolver bug rather than a spelling
* difference, but the report must never leak an absolute filesystem path, so
* the file name stands in.
*/
private taskDocumentPath(changeDir: string, file: string): string {
const relative = path.relative(
FileSystemUtils.canonicalizeExistingPath(changeDir),
FileSystemUtils.canonicalizeExistingPath(file)
);
const escapes = relative === '' || relative.startsWith('..') || path.isAbsolute(relative);
return escapes ? path.basename(file) : FileSystemUtils.toPosixPath(relative);
}
/**
* True when the change resolves to the package's own `spec-driven` schema. A
* project schema that merely reuses the name is not it, so checks written
* against the built-in template never fire on someone else's task format.
*/
private usesBuiltInSpecDrivenSchema(changeDir: string, projectRoot: string): boolean {
try {
const schemaName = resolveSchemaForChange(changeDir, undefined, projectRoot).replace(
/\.ya?ml$/,
''
);
if (schemaName !== 'spec-driven') return false;
const schemaDir = getSchemaDir(schemaName, projectRoot);
if (schemaDir === null) return false;
const builtInSchemaDir = path.join(getPackageSchemasDir(), 'spec-driven');
if (
schemaName !== 'spec-driven' ||
schemaDir === null ||
FileSystemUtils.canonicalizeExistingPath(schemaDir) !==
FileSystemUtils.canonicalizeExistingPath(builtInSchemaDir)
) {
return [];
}
return (
FileSystemUtils.canonicalizeExistingPath(schemaDir) ===
FileSystemUtils.canonicalizeExistingPath(builtInSchemaDir)
);
} catch {
return [];
return false;
}
let taskFiles: string[];
try {
taskFiles = resolveTaskFilesForChange(changeDir, projectRoot);
} catch {
return [];
}
if (taskFiles.length === 0) {
taskFiles = [path.join(changeDir, 'tasks.md')];
}
const documents: Array<{ path: string; content: string }> = [];
for (const taskFile of taskFiles) {
let content: string;
try {
content = await fs.readFile(taskFile, 'utf-8');
} catch {
continue;
}
documents.push({
path: FileSystemUtils.toPosixPath(path.relative(changeDir, taskFile)),
content,
});
}
documents.sort((left, right) => left.path.localeCompare(right.path));
return findTaskNumberingIssues(documents).map((issue) => ({
level: 'WARNING',
path: issue.path,
line: issue.line,
message: issue.message,
}));
}
/**
@@ -0,0 +1,291 @@
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { tmpdir } from 'os';
import { runCLI } from '../helpers/run-cli.js';
describe('openspec validate checks task checkbox formatting (#354)', () => {
let projectDir: string;
const write = async (relative: string, content: string) => {
const file = path.join(projectDir, relative);
await fs.mkdir(path.dirname(file), { recursive: true });
await fs.writeFile(file, content, 'utf-8');
};
const validDelta = [
'## ADDED Requirements',
'',
'### Requirement: Task lists SHALL be machine readable',
'The validator SHALL report task lists that progress cannot count.',
'',
'#### Scenario: Validate a bullet-only task list',
'- **WHEN** validation runs on a task file without checkboxes',
'- **THEN** the change is reported as counting zero tasks',
'',
].join('\n');
const globTasksSchema = [
'name: glob-tasks',
'version: 1',
'description: tasks artifact uses a nested glob',
'artifacts:',
' - id: proposal',
' generates: proposal.md',
' description: Proposal',
' template: proposal.md',
' requires: []',
' - id: tasks',
' generates: "**/tasks.md"',
' description: Nested tasks',
' template: tasks.md',
' requires: [proposal]',
'apply:',
' requires: [tasks]',
' tracks: "**/tasks.md"',
'',
].join('\n');
// No `apply` block: the tracked-tasks artifact is found by its `tasks` id,
// the same fallback progress counting uses.
const implicitTasksSchema = [
'name: implicit-tasks',
'version: 1',
'description: tasks artifact without an apply block',
'artifacts:',
' - id: proposal',
' generates: proposal.md',
' description: Proposal',
' template: proposal.md',
' requires: []',
' - id: tasks',
' generates: tasks.md',
' description: Tasks',
' template: tasks.md',
' requires: [proposal]',
'',
].join('\n');
const untrackedTasksSchema = [
'name: no-tasks-artifact',
'version: 1',
'description: schema without a tracked tasks artifact',
'artifacts:',
' - id: proposal',
' generates: proposal.md',
' description: Proposal',
' template: proposal.md',
' requires: []',
'',
].join('\n');
beforeAll(async () => {
projectDir = await fs.mkdtemp(path.join(tmpdir(), 'openspec-task-checkboxes-e2e-'));
await write('openspec/changes/bullet-tasks/specs/tasks/spec.md', validDelta);
await write(
'openspec/changes/bullet-tasks/tasks.md',
['# Tasks', '', '## 1. Implementation', '', '- Add the parser', '- Add the tests', ''].join(
'\n'
)
);
await write('openspec/changes/checkbox-tasks/specs/tasks/spec.md', validDelta);
await write(
'openspec/changes/checkbox-tasks/tasks.md',
['## 1. Implementation', '', '- [ ] 1.1 Add the parser', '- A supporting note', ''].join('\n')
);
await write('openspec/schemas/glob-tasks/schema.yaml', globTasksSchema);
await write('openspec/changes/nested-bullets/.openspec.yaml', 'schema: glob-tasks\n');
await write('openspec/changes/nested-bullets/specs/tasks/spec.md', validDelta);
await write('openspec/changes/nested-bullets/backend/tasks.md', '- build the api\n');
await write('openspec/changes/nested-bullets/frontend/tasks.md', '- [ ] 2.1 build the ui\n');
await write('openspec/changes/nested-all-bullets/.openspec.yaml', 'schema: glob-tasks\n');
await write('openspec/changes/nested-all-bullets/specs/tasks/spec.md', validDelta);
await write('openspec/changes/nested-all-bullets/backend/tasks.md', '- build the api\n');
await write('openspec/changes/nested-all-bullets/frontend/tasks.md', '- build the ui\n');
await write('openspec/schemas/implicit-tasks/schema.yaml', implicitTasksSchema);
await write('openspec/changes/implicit-tracking/.openspec.yaml', 'schema: implicit-tasks\n');
await write('openspec/changes/implicit-tracking/specs/tasks/spec.md', validDelta);
await write('openspec/changes/implicit-tracking/tasks.md', '- build the api\n');
await write('openspec/schemas/no-tasks-artifact/schema.yaml', untrackedTasksSchema);
await write('openspec/changes/untracked-tasks/.openspec.yaml', 'schema: no-tasks-artifact\n');
await write('openspec/changes/untracked-tasks/specs/tasks/spec.md', validDelta);
await write('openspec/changes/untracked-tasks/tasks.md', '- an untracked bullet\n');
});
afterAll(async () => {
await fs.rm(projectDir, { recursive: true, force: true });
});
it('reports a bullet-only task list and names the counting consequence', async () => {
const result = await runCLI(
['validate', '--type', 'change', 'bullet-tasks', '--strict', '--json'],
{ cwd: projectDir }
);
expect(result.exitCode).toBe(1);
const report = JSON.parse(result.stdout);
expect(report.items[0].issues).toEqual([
expect.objectContaining({
level: 'WARNING',
path: 'tasks.md',
line: 5,
message: expect.stringContaining('counts as 0 tasks'),
}),
]);
});
it('agrees with the progress the same change reports', async () => {
const result = await runCLI(['list', '--changes'], { cwd: projectDir });
expect(result.exitCode).toBe(0);
expect(result.stdout).toMatch(/bullet-tasks\s+No tasks/);
});
it('keeps the warning non-blocking without --strict', async () => {
const result = await runCLI(['validate', '--type', 'change', 'bullet-tasks', '--json'], {
cwd: projectDir,
});
expect(result.exitCode).toBe(0);
expect(JSON.parse(result.stdout).items[0].valid).toBe(true);
});
it('stays silent when the change has a real checklist', async () => {
const result = await runCLI(['validate', '--type', 'change', 'checkbox-tasks', '--strict'], {
cwd: projectDir,
});
expect(result.exitCode).toBe(0);
expect(result.stdout).toContain("Change 'checkbox-tasks' is valid");
});
it('checks the whole tracked set of a custom schema, not each file alone', async () => {
const result = await runCLI(
['validate', '--type', 'change', 'nested-bullets', '--strict', '--json'],
{ cwd: projectDir }
);
expect(result.exitCode).toBe(0);
const taskIssues = JSON.parse(result.stdout).items[0].issues.filter(
(issue: { path: string }) => issue.path.endsWith('tasks.md')
);
expect(taskIssues).toEqual([]);
});
it('reports each nested file with a POSIX path when none of them has a checkbox', async () => {
const result = await runCLI(
['validate', '--type', 'change', 'nested-all-bullets', '--strict', '--json'],
{ cwd: projectDir }
);
expect(result.exitCode).toBe(1);
// Paths are normalized, so this assertion fails on a Windows separator.
expect(JSON.parse(result.stdout).items[0].issues).toEqual([
expect.objectContaining({ level: 'WARNING', path: 'backend/tasks.md', line: 1 }),
expect.objectContaining({ level: 'WARNING', path: 'frontend/tasks.md', line: 1 }),
]);
});
it('ignores a tasks file no artifact tracks', async () => {
const result = await runCLI(
['validate', '--type', 'change', 'untracked-tasks', '--strict', '--json'],
{ cwd: projectDir }
);
expect(result.exitCode).toBe(0);
expect(JSON.parse(result.stdout).items[0].issues).toEqual([]);
});
it('follows the tracked-tasks artifact when a schema declares no apply block', async () => {
const result = await runCLI(
['validate', '--type', 'change', 'implicit-tracking', '--strict', '--json'],
{ cwd: projectDir }
);
expect(result.exitCode).toBe(1);
expect(JSON.parse(result.stdout).items[0].issues).toEqual([
expect.objectContaining({ level: 'WARNING', path: 'tasks.md', line: 1 }),
]);
});
it('surfaces the warning through the deprecated change validate command', async () => {
const result = await runCLI(['change', 'validate', 'bullet-tasks', '--strict'], {
cwd: projectDir,
});
expect(result.exitCode).toBe(1);
// The text renderer prints level, path and message; it carries no line for
// any issue, which is why this asserts what that surface actually emits.
// The line lives in the JSON report, asserted above.
expect(result.stderr).toContain('[WARNING] tasks.md:');
expect(result.stderr).toContain('counts as 0 tasks');
});
it.skipIf(process.platform === 'win32')(
'stays silent when a tracked file exists but cannot be read',
async () => {
// The claim is about the whole tracked set, and the checkboxes could be
// in exactly the file that would not open.
const dir = await fs.mkdtemp(path.join(tmpdir(), 'openspec-task-unreadable-e2e-'));
const writeIn = async (relative: string, content: string) => {
const file = path.join(dir, relative);
await fs.mkdir(path.dirname(file), { recursive: true });
await fs.writeFile(file, content, 'utf-8');
return file;
};
await writeIn('openspec/schemas/glob-tasks/schema.yaml', globTasksSchema);
await writeIn('openspec/changes/half-read/.openspec.yaml', 'schema: glob-tasks\n');
await writeIn('openspec/changes/half-read/specs/tasks/spec.md', validDelta);
await writeIn('openspec/changes/half-read/backend/tasks.md', '- build the api\n');
const locked = await writeIn(
'openspec/changes/half-read/frontend/tasks.md',
'- [ ] 2.1 build the ui\n'
);
await fs.chmod(locked, 0o000);
try {
// Without this the test would pass for the wrong reason: if the lock did
// not take (root, or a filesystem that ignores the mode), the checkbox
// in this very file would silence the warning on its own.
await expect(fs.readFile(locked, 'utf-8')).rejects.toThrow();
const result = await runCLI(
['validate', '--type', 'change', 'half-read', '--strict', '--json'],
{ cwd: dir }
);
expect(result.exitCode).toBe(0);
expect(JSON.parse(result.stdout).items[0].issues).toEqual([]);
} finally {
await fs.chmod(locked, 0o644);
await fs.rm(dir, { recursive: true, force: true });
}
}
);
it('applies the warning in bulk validation', async () => {
const result = await runCLI(['validate', '--changes', '--strict', '--json'], {
cwd: projectDir,
});
expect(result.exitCode).toBe(1);
const byId = Object.fromEntries(
JSON.parse(result.stdout).items.map((item: { id: string; valid: boolean }) => [
item.id,
item.valid,
])
);
expect(byId['bullet-tasks']).toBe(false);
expect(byId['checkbox-tasks']).toBe(true);
expect(byId['nested-bullets']).toBe(true);
expect(byId['nested-all-bullets']).toBe(false);
expect(byId['implicit-tracking']).toBe(false);
expect(byId['untracked-tasks']).toBe(true);
});
});
+256
View File
@@ -0,0 +1,256 @@
import { describe, expect, it } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import fg from 'fast-glob';
import { findMissingTaskCheckboxIssues } from '../../src/core/validation/task-checkboxes.js';
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');
const findInSingleFile = (content: string) =>
findMissingTaskCheckboxIssues([{ path: 'tasks.md', content }]).map(
({ path: _path, ...issue }) => issue
);
describe('findMissingTaskCheckboxIssues', () => {
it('reports a task list written as plain bullets', () => {
const issues = findInSingleFile(
['# Tasks', '', '## 1. Implementation', '', '- Add the parser', '- Add the tests', ''].join(
'\n'
)
);
expect(issues).toEqual([
{
line: 5,
message: expect.stringContaining('counts as 0 tasks'),
},
]);
});
it('reports a task list written as a numbered list', () => {
expect(findInSingleFile('# Tasks\n\n1. Add the parser\n2. Add the tests\n')).toEqual([
{ line: 3, message: expect.any(String) },
]);
expect(findInSingleFile('1) Add the parser\n')).toEqual([
{ line: 1, message: expect.any(String) },
]);
});
it('stays silent when checkboxes are present', () => {
expect(findInSingleFile('- [ ] 1.1 Add the parser\n- Supporting note\n')).toEqual([]);
expect(findInSingleFile('- [x] 1.1 Add the parser\n')).toEqual([]);
expect(findInSingleFile(' - [ ] 1.1.1 A nested task\n')).toEqual([]);
});
it('stays silent on prose and on an empty file', () => {
expect(findInSingleFile('# Tasks\n\nNothing planned yet.\n')).toEqual([]);
expect(findInSingleFile('')).toEqual([]);
expect(findMissingTaskCheckboxIssues([])).toEqual([]);
});
it('ignores list items inside fenced blocks', () => {
expect(
findInSingleFile(['# Tasks', '', '```md', '- an example bullet', '```', ''].join('\n'))
).toEqual([]);
expect(
findInSingleFile(
['~~~', '- fenced with tildes', '~~~', '', '- a real bullet', ''].join('\n')
)
).toEqual([{ line: 5, message: expect.any(String) }]);
});
it('closes a fence only on a matching, long enough, bare delimiter', () => {
// A three-marker sample nested inside a four-marker block: the inner run is
// content, so the bullets after it are still fenced.
expect(
findInSingleFile(
['````md', '```', '- an example bullet', '```', '````', ''].join('\n')
)
).toEqual([]);
// An annotated run is an opener's shape, never a closer's.
expect(
findInSingleFile(['```', '```js', '- an example bullet', '```', ''].join('\n'))
).toEqual([]);
// Tildes do not close a backtick fence.
expect(findInSingleFile(['```', '~~~', '- an example bullet', ''].join('\n'))).toEqual([]);
// A longer closing run still closes.
expect(
findInSingleFile(['```', 'sample', '`````', '', '- a real bullet', ''].join('\n'))
).toEqual([{ line: 5, message: expect.any(String) }]);
});
it('tracks fences in CRLF files', () => {
expect(
findInSingleFile(['```md', '- an example bullet', '```', '', '- a real bullet', ''].join('\r\n'))
).toEqual([{ line: 5, message: expect.any(String) }]);
});
it('only treats a fence indented up to three spaces as a fence', () => {
// Four spaces makes an indented code block, not an opener. Reading it as one
// would leave the scan inside a block that never began.
expect(
findInSingleFile([' ```', '', '- a real bullet', ''].join('\n'))
).toEqual([{ line: 3, message: expect.any(String) }]);
expect(
findInSingleFile([' ```', '- an example bullet', ' ```', ''].join('\n'))
).toEqual([]);
});
it('does not read top-level indented code as a task list', () => {
// Four spaces of indent is a code block, the same rule the fence logic
// already applies. A tasks file that pastes terminal output was reported
// as an uncheckboxed task list, and under --strict that failed validation
// on a correct file.
expect(
findInSingleFile(['## 1. Notes', '', 'Example output:', '', ' - example output', ''].join('\n'))
).toEqual([]);
expect(
findInSingleFile(['## 1. Notes', '', 'Example output:', '', '\t- tabbed output', ''].join('\n'))
).toEqual([]);
expect(
findInSingleFile(['## 1. Notes', '', 'Example output:', '', ' 1. numbered output', ''].join('\n'))
).toEqual([]);
});
it('still reports a genuine nested list, by naming its parent', () => {
// The cut is safe because this scan reports the first list item it finds,
// and a nested item always sits under a shallower parent. Three spaces is
// not code, so an indented-but-shallow list is still reported on its own.
expect(
findInSingleFile(['## 1. Work', '', '- Parent task', ' - Nested detail', ''].join('\n'))
).toEqual([{ line: 3, message: expect.any(String) }]);
expect(
findInSingleFile(['## 1. Work', '', ' - Three spaces is not code', ''].join('\n'))
).toEqual([{ line: 3, message: expect.any(String) }]);
});
it('does not treat a horizontal rule or emphasis as a list item', () => {
expect(findInSingleFile('# Tasks\n\n---\n\n***\n')).toEqual([]);
});
it('skips YAML front matter', () => {
expect(
findInSingleFile(
['---', 'tags:', ' - planning', ' - backend', '---', '', 'Nothing planned yet.', ''].join(
'\n'
)
)
).toEqual([]);
expect(
findInSingleFile(
['---', 'tags:', ' - planning', '---', '', '- a real bullet', ''].join('\n')
)
).toEqual([{ line: 6, message: expect.any(String) }]);
});
it('treats an unterminated front-matter opener as a thematic break', () => {
expect(findInSingleFile(['---', '', '- a real bullet', ''].join('\n'))).toEqual([
{ line: 3, message: expect.any(String) },
]);
});
it('does not read a longer dash run as front matter', () => {
// `----` is a thematic break. Reading it as an opener would hide every list
// between it and the next `---`.
expect(
findInSingleFile(['----', '', '- a real bullet', '', '---', ''].join('\n'))
).toEqual([{ line: 3, message: expect.any(String) }]);
});
it('skips front matter behind a UTF-8 byte order mark', () => {
// Windows editors and PowerShell redirects prepend a BOM. Without stripping
// it the opener never matched, and the `tags:` list failed `--strict`.
expect(
findInSingleFile(['---', 'tags:', ' - planning', '---', '', 'Nothing planned yet.', ''].join('\n'))
).toEqual([]);
});
it('does not read a number longer than nine digits as a list marker', () => {
// CommonMark caps an ordered marker at nine digits, as specs-apply does.
expect(findInSingleFile('1234567890. is a year range, not a task\n')).toEqual([]);
expect(findInSingleFile('123456789. still a list item\n')).toEqual([
{ line: 1, message: expect.any(String) },
]);
});
it('skips HTML comments without hiding the line that follows them', () => {
expect(
findInSingleFile(['<!--', '- a retired task', '-->', '', 'Nothing planned yet.', ''].join('\n'))
).toEqual([]);
expect(
findInSingleFile(['<!-- a note -->', '- a real bullet', ''].join('\n'))
).toEqual([{ line: 2, message: expect.any(String) }]);
expect(
findInSingleFile(['<!--', '- a retired task', '-->', '- a real bullet', ''].join('\n'))
).toEqual([{ line: 4, message: expect.any(String) }]);
});
it('accepts every packaged tasks template', async () => {
// An agent writing a task file follows these. If one ever loses its
// checkboxes, every change built from it starts life counting zero tasks.
const templates = await fg('schemas/*/templates/tasks.md', {
cwd: repoRoot,
absolute: true,
});
expect(templates.length).toBeGreaterThan(0);
for (const template of templates) {
const content = await fs.readFile(template, 'utf-8');
expect({
template: path.relative(repoRoot, template),
issues: findMissingTaskCheckboxIssues([{ path: 'tasks.md', content }]),
}).toEqual({ template: path.relative(repoRoot, template), issues: [] });
}
});
it('does not let the template heading comment swallow its checklist', () => {
// The scaffolded tasks.md: a heading carrying an inline comment, then real
// checkboxes. It must stay silent, and would not if an inline comment on a
// heading opened a block.
expect(
findInSingleFile(
[
'## 1. <!-- Task Group Name -->',
'',
'- [ ] 1.1 <!-- Task description -->',
'- [ ] 1.2 <!-- Task description -->',
'',
].join('\n')
)
).toEqual([]);
});
it('reports a list of unrecognised checkbox markers, which count as no task', () => {
// `- [ab] ...` looks like a checkbox, but a marker longer than one
// character is not one the task parser recognises, so the change really
// does count zero tasks and the warning is the only signal.
expect(findInSingleFile('- [ab] 1.1 in progress\n')).toEqual([
{ line: 1, message: expect.any(String) },
]);
});
it('reports every file only when the whole change has no checkbox', () => {
const withoutCheckboxes = [
{ path: 'backend/tasks.md', content: '- build the api\n' },
{ path: 'frontend/tasks.md', content: '- build the ui\n' },
];
expect(findMissingTaskCheckboxIssues(withoutCheckboxes).map((issue) => issue.path)).toEqual([
'backend/tasks.md',
'frontend/tasks.md',
]);
const oneRealChecklist = [
{ path: 'backend/tasks.md', content: '- [ ] 1.1 build the api\n' },
{ path: 'frontend/tasks.md', content: '- build the ui\n' },
];
expect(findMissingTaskCheckboxIssues(oneRealChecklist)).toEqual([]);
});
it('handles CRLF files', () => {
expect(findInSingleFile('# Tasks\r\n\r\n- Add the parser\r\n')).toEqual([
{ line: 3, message: expect.any(String) },
]);
expect(findInSingleFile('- [ ] 1.1 Add the parser\r\n')).toEqual([]);
});
});