Compare commits

...
Author SHA1 Message Date
Tabish Bidiwale 8ac50289f0 Add tests 2025-08-19 23:22:18 +10:00
Tabish Bidiwale 1f295cec52 feat: add unified validate command with interactive selection and bulk operations 2025-08-19 22:58:17 +10:00
Tabish Bidiwale ad8e213cf9 Merge pull request #42 from Fission-AI/feat/bulk-validation-and-interactive-selection
feat: add bulk validation and interactive selection for OpenSpec commands
2025-08-19 22:35:25 +10:00
11 changed files with 730 additions and 12 deletions
@@ -0,0 +1,81 @@
# Implementation Tasks
## 1. Change Command: Interactive Validation Selection
- [x] 1.1 Add `--no-interactive` flag to `change validate` in `src/cli/index.ts`
- [x] 1.2 Implement interactivity gate respecting TTY and `OPEN_SPEC_INTERACTIVE=0` in `src/commands/change.ts`
- [x] 1.3 When no `[change-name]` is provided and interactivity is allowed, prompt with a list of active changes (exclude `archive/`) and validate the selected one
- [x] 1.4 Preserve current non-interactive fallback: print available change IDs and hint, set `process.exitCode = 1`
- [x] 1.5 Tests: add coverage for interactive and non-interactive flows
- Added `test/commands/change.interactive-validate.test.ts`
## 2. Spec Command: Interactive Validation Selection
- [x] 2.1 Make `spec validate` accept optional `[spec-id]` in `src/commands/spec.ts` registration
- [x] 2.2 Add `--no-interactive` flag to `spec validate`
- [x] 2.3 Implement interactivity gate respecting TTY and `OPEN_SPEC_INTERACTIVE=0`
- [x] 2.4 When no `[spec-id]` provided and interactivity allowed, prompt to select from `openspec/specs/*/spec.md` and validate the selected spec
- [x] 2.5 Preserve current non-interactive fallback when no spec-id and no interactivity: print existing error and exit code non-zero
- [x] 2.6 Tests: add coverage for interactive and non-interactive flows
- Added `test/commands/spec.interactive-validate.test.ts`
## 3. New Top-level `validate` Command
- [x] 3.1 Add `validate` command in `src/cli/index.ts`
- Options: `--all`, `--changes`, `--specs`, `--type <change|spec>`, `--strict`, `--json`, `--no-interactive`
- Usage: `openspec validate [item-name]`
- [x] 3.2 Create `src/commands/validate.ts` implementing:
- [x] 3.2.1 Interactive selector when no args (choices: All, Changes, Specs, Specific item)
- [x] 3.2.2 Non-interactive fallback with helpful hint and exit code 1
- [x] 3.2.3 Direct item validation with automatic type detection
- [x] 3.2.4 Ambiguity error when name exists as both change and spec; suggest `--type` or subcommands
- [x] 3.2.5 Unknown item handling with nearest-match suggestions
- [x] 3.2.6 Bulk validation for `--all`, `--changes`, `--specs` (exclude `openspec/changes/archive/`)
- [x] 3.2.7 Respect `--strict` and `--json` options; JSON shape per spec
- [x] 3.2.8 Exit with code 1 if any validation fails
- [x] 3.2.9 Bounded concurrency (default 4–8) for bulk validation
- [x] 3.2.10 Progress indication during bulk runs (current item, running counts)
## 4. Utilities and Shared Helpers
- [x] 4.1 Add `src/utils/interactive.ts` with `isInteractive(stdin: NodeJS.ReadStream, noInteractiveFlag?: boolean): boolean`
- Considers: `process.stdin.isTTY`, `--no-interactive`, `OPEN_SPEC_INTERACTIVE=0`
- [x] 4.2 Add `src/utils/item-discovery.ts` with:
- `getActiveChangeIds(root = process.cwd()): Promise<string[]>` (exclude `archive/`)
- `getSpecIds(root = process.cwd()): Promise<string[]>` (folders with `spec.md`)
- [ ] 4.3 Optional: `src/utils/concurrency.ts` helper for bounded parallelism
- [x] 4.4 Reuse `src/core/validation/validator.ts` for item validation
## 5. JSON Output (Bulk Validation)
- [x] 5.1 Implement JSON schema:
- `items: Array<{ id: string, type: "change"|"spec", valid: boolean, issues: Issue[], durationMs: number }>`
- `summary: { totals: { items: number, passed: number, failed: number }, byType: { change?: { items: number, passed: number, failed: number }, spec?: { items: number, passed: number, failed: number } } }`
- `version: "1.0"`
- [x] 5.2 Ensure process exit code is 1 if any `items[].valid === false`
- [x] 5.3 Tests for JSON shape (keys, types, counts) and exit code behavior
- Added `test/commands/validate.test.ts`
## 6. Progress and UX
- [x] 6.1 Use `ora` or minimal console progress to show current item and running counts
- [x] 6.2 Keep output stable in `--json` mode (no extra logs to stdout; use stderr for progress if needed)
- [x] 6.3 Ensure responsiveness with concurrency limits
## 7. Tests
- [x] 7.1 Add top-level validate tests: `test/commands/validate.test.ts`
- Includes non-interactive hint, --all JSON, --specs with concurrency, ambiguity error
- [ ] 7.2 Add unit tests for `isInteractive` and item discovery helpers
- [x] 7.3 Extend existing change/spec command tests to cover interactive `validate`
- Added `test/commands/change.interactive-validate.test.ts`, `test/commands/spec.interactive-validate.test.ts`
## 8. CLI Help and Docs
- [x] 8.1 Update command descriptions/options in `src/cli/index.ts`
- [x] 8.2 Verify help output includes `validate` command and flags
- [x] 8.3 Ensure existing specs under `openspec/changes/bulk-validation-interactive-selection/specs/*` remain satisfied
## 9. Non-functional
- [x] 9.1 Code style and types: explicit types for exported APIs; avoid `any`
- [x] 9.2 No linter errors; stable formatting; avoid unrelated refactors
- [x] 9.3 Maintain existing behavior for unaffected commands
## 10. Acceptance Criteria Mapping
- [x] AC-1: `openspec change validate` interactive selection when no arg (TTY only; respects `--no-interactive`/env) — matches cli-change spec
- [x] AC-2: `openspec spec validate` interactive selection when no arg (TTY only; respects `--no-interactive`/env) — matches cli-spec spec
- [x] AC-3: New `openspec validate` supports interactive selection, bulk/filtered validation, JSON schema, progress, concurrency, exit codes — matches cli-validate spec
+26 -1
View File
@@ -9,6 +9,7 @@ import { ListCommand } from '../core/list.js';
import { ArchiveCommand } from '../core/archive.js';
import { registerSpecCommand } from '../commands/spec.js';
import { ChangeCommand } from '../commands/change.js';
import { ValidateCommand } from '../commands/validate.js';
const program = new Command();
@@ -146,7 +147,8 @@ changeCmd
.description('Validate a change proposal')
.option('--strict', 'Enable strict validation mode')
.option('--json', 'Output validation report as JSON')
.action(async (changeName?: string, options?: { strict?: boolean; json?: boolean }) => {
.option('--no-interactive', 'Disable interactive prompts')
.action(async (changeName?: string, options?: { strict?: boolean; json?: boolean; noInteractive?: boolean }) => {
try {
const changeCommand = new ChangeCommand();
await changeCommand.validate(changeName, options);
@@ -175,4 +177,27 @@ program
registerSpecCommand(program);
// Top-level validate command
program
.command('validate [item-name]')
.description('Validate changes and specs')
.option('--all', 'Validate all changes and specs')
.option('--changes', 'Validate all changes')
.option('--specs', 'Validate all specs')
.option('--type <type>', 'Specify item type when ambiguous: change|spec')
.option('--strict', 'Enable strict validation mode')
.option('--json', 'Output validation results as JSON')
.option('--concurrency <n>', 'Max concurrent validations (defaults to env OPENSPEC_CONCURRENCY or 6)')
.option('--no-interactive', 'Disable interactive prompts')
.action(async (itemName?: string, options?: { all?: boolean; changes?: boolean; specs?: boolean; type?: string; strict?: boolean; json?: boolean; noInteractive?: boolean; concurrency?: string }) => {
try {
const validateCommand = new ValidateCommand();
await validateCommand.execute(itemName, options);
} catch (error) {
console.log();
ora().fail(`Error: ${(error as Error).message}`);
process.exit(1);
}
});
program.parse();
+20 -8
View File
@@ -1,9 +1,12 @@
import { promises as fs } from 'fs';
import path from 'path';
import { select } from '@inquirer/prompts';
import { JsonConverter } from '../core/converters/json-converter.js';
import { Validator } from '../core/validation/validator.js';
import { ChangeParser } from '../core/parsers/change-parser.js';
import { Change } from '../core/schemas/index.js';
import { isInteractive } from '../utils/interactive.js';
import { getActiveChangeIds } from '../utils/item-discovery.js';
// Constants for better maintainability
const ARCHIVE_DIR = 'archive';
@@ -170,19 +173,28 @@ export class ChangeCommand {
}
}
async validate(changeName?: string, options?: { strict?: boolean; json?: boolean }): Promise<void> {
async validate(changeName?: string, options?: { strict?: boolean; json?: boolean; noInteractive?: boolean }): Promise<void> {
const changesPath = path.join(process.cwd(), 'openspec', 'changes');
if (!changeName) {
const changes = await this.getActiveChanges(changesPath);
if (changes.length === 0) {
console.error('No change specified. No active changes found.');
const canPrompt = isInteractive(options?.noInteractive);
const changes = await getActiveChangeIds();
if (canPrompt && changes.length > 0) {
const selected = await select({
message: 'Select a change to validate',
choices: changes.map(id => ({ name: id, value: id })),
});
changeName = selected;
} else {
console.error(`No change specified. Available IDs: ${changes.join(', ')}`);
if (changes.length === 0) {
console.error('No change specified. No active changes found.');
} else {
console.error(`No change specified. Available IDs: ${changes.join(', ')}`);
}
console.error('Hint: use "openspec change list" to view available changes.');
process.exitCode = 1;
return;
}
console.error('Hint: use "openspec change list" to view available changes.');
process.exitCode = 1;
return;
}
const proposalPath = path.join(changesPath, changeName, 'proposal.md');
+19 -2
View File
@@ -4,6 +4,9 @@ import { join } from 'path';
import { MarkdownParser } from '../core/parsers/markdown-parser.js';
import { Validator } from '../core/validation/validator.js';
import type { Spec } from '../core/schemas/index.js';
import { select } from '@inquirer/prompts';
import { isInteractive } from '../utils/interactive.js';
import { getSpecIds } from '../utils/item-discovery.js';
const SPECS_DIR = 'openspec/specs';
@@ -166,12 +169,26 @@ export function registerSpecCommand(rootProgram: typeof program) {
});
specCommand
.command('validate <spec-id>')
.command('validate [spec-id]')
.description('Validate a specification structure')
.option('--strict', 'Enable strict validation mode')
.option('--json', 'Output validation report as JSON')
.action(async (specId: string, options: { strict?: boolean; json?: boolean }) => {
.option('--no-interactive', 'Disable interactive prompts')
.action(async (specId: string | undefined, options: { strict?: boolean; json?: boolean; noInteractive?: boolean }) => {
try {
if (!specId) {
const canPrompt = isInteractive(options?.noInteractive);
const specIds = await getSpecIds();
if (canPrompt && specIds.length > 0) {
specId = await select({
message: 'Select a spec to validate',
choices: specIds.map(id => ({ name: id, value: id })),
});
} else {
throw new Error('Missing required argument <spec-id>');
}
}
const specPath = join(SPECS_DIR, specId, 'spec.md');
if (!existsSync(specPath)) {
+312
View File
@@ -0,0 +1,312 @@
import { select } from '@inquirer/prompts';
import ora from 'ora';
import path from 'path';
import { Validator } from '../core/validation/validator.js';
import { isInteractive } from '../utils/interactive.js';
import { getActiveChangeIds, getSpecIds } from '../utils/item-discovery.js';
type ItemType = 'change' | 'spec';
interface ExecuteOptions {
all?: boolean;
changes?: boolean;
specs?: boolean;
type?: string;
strict?: boolean;
json?: boolean;
noInteractive?: boolean;
concurrency?: string;
}
interface BulkItemResult {
id: string;
type: ItemType;
valid: boolean;
issues: { level: 'ERROR' | 'WARNING' | 'INFO'; path: string; message: string }[];
durationMs: number;
}
export class ValidateCommand {
async execute(itemName: string | undefined, options: ExecuteOptions = {}): Promise<void> {
const interactive = isInteractive(options.noInteractive);
// Handle bulk flags first
if (options.all || options.changes || options.specs) {
await this.runBulkValidation({
changes: !!options.all || !!options.changes,
specs: !!options.all || !!options.specs,
}, { strict: !!options.strict, json: !!options.json, concurrency: options.concurrency });
return;
}
// No item and no flags
if (!itemName) {
if (interactive) {
await this.runInteractiveSelector({ strict: !!options.strict, json: !!options.json, concurrency: options.concurrency });
return;
}
this.printNonInteractiveHint();
process.exitCode = 1;
return;
}
// Direct item validation with type detection or override
const typeOverride = this.normalizeType(options.type);
await this.validateDirectItem(itemName, { typeOverride, strict: !!options.strict, json: !!options.json });
}
private normalizeType(value?: string): ItemType | undefined {
if (!value) return undefined;
const v = value.toLowerCase();
if (v === 'change' || v === 'spec') return v;
return undefined;
}
private async runInteractiveSelector(opts: { strict: boolean; json: boolean; concurrency?: string }): Promise<void> {
const choice = await select({
message: 'What would you like to validate?',
choices: [
{ name: 'All (changes + specs)', value: 'all' },
{ name: 'All changes', value: 'changes' },
{ name: 'All specs', value: 'specs' },
{ name: 'Pick a specific change or spec', value: 'one' },
],
});
if (choice === 'all') return this.runBulkValidation({ changes: true, specs: true }, opts);
if (choice === 'changes') return this.runBulkValidation({ changes: true, specs: false }, opts);
if (choice === 'specs') return this.runBulkValidation({ changes: false, specs: true }, opts);
// one
const [changes, specs] = await Promise.all([getActiveChangeIds(), getSpecIds()]);
const items: { name: string; value: { type: ItemType; id: string } }[] = [];
items.push(...changes.map(id => ({ name: `change/${id}`, value: { type: 'change' as const, id } })));
items.push(...specs.map(id => ({ name: `spec/${id}`, value: { type: 'spec' as const, id } })));
if (items.length === 0) {
console.error('No items found to validate.');
process.exitCode = 1;
return;
}
const picked = await select<{ type: ItemType; id: string }>({ message: 'Pick an item', choices: items });
await this.validateByType(picked.type, picked.id, opts);
}
private printNonInteractiveHint(): void {
console.error('Nothing to validate. Try one of:');
console.error(' openspec validate --all');
console.error(' openspec validate --changes');
console.error(' openspec validate --specs');
console.error(' openspec validate <item-name>');
console.error('Or run in an interactive terminal.');
}
private async validateDirectItem(itemName: string, opts: { typeOverride?: ItemType; strict: boolean; json: boolean }): Promise<void> {
const [changes, specs] = await Promise.all([getActiveChangeIds(), getSpecIds()]);
const isChange = changes.includes(itemName);
const isSpec = specs.includes(itemName);
const type = opts.typeOverride ?? (isChange ? 'change' : isSpec ? 'spec' : undefined);
if (!type) {
console.error(`Unknown item '${itemName}'`);
const suggestions = nearestMatches(itemName, [...changes, ...specs]);
if (suggestions.length) console.error(`Did you mean: ${suggestions.join(', ')}?`);
process.exitCode = 1;
return;
}
if (!opts.typeOverride && isChange && isSpec) {
console.error(`Ambiguous item '${itemName}' matches both a change and a spec.`);
console.error('Pass --type change|spec, or use: openspec change validate / openspec spec validate');
process.exitCode = 1;
return;
}
await this.validateByType(type, itemName, opts);
}
private async validateByType(type: ItemType, id: string, opts: { strict: boolean; json: boolean }): Promise<void> {
const validator = new Validator(opts.strict);
if (type === 'change') {
const file = path.join(process.cwd(), 'openspec', 'changes', id, 'proposal.md');
const start = Date.now();
const report = await validator.validateChange(file);
const durationMs = Date.now() - start;
this.printReport('change', id, report, durationMs, opts.json);
process.exitCode = report.valid ? 0 : 1;
return;
}
const file = path.join(process.cwd(), 'openspec', 'specs', id, 'spec.md');
const start = Date.now();
const report = await validator.validateSpec(file);
const durationMs = Date.now() - start;
this.printReport('spec', id, report, durationMs, opts.json);
process.exitCode = report.valid ? 0 : 1;
}
private printReport(type: ItemType, id: string, report: { valid: boolean; issues: any[] }, durationMs: number, json: boolean): void {
if (json) {
const out = { items: [{ id, type, valid: report.valid, issues: report.issues, durationMs }], summary: { totals: { items: 1, passed: report.valid ? 1 : 0, failed: report.valid ? 0 : 1 }, byType: { [type]: { items: 1, passed: report.valid ? 1 : 0, failed: report.valid ? 0 : 1 } } }, version: '1.0' };
console.log(JSON.stringify(out, null, 2));
return;
}
if (report.valid) {
console.log(`${type === 'change' ? 'Change' : 'Specification'} '${id}' is valid`);
} else {
console.error(`${type === 'change' ? 'Change' : 'Specification'} '${id}' has issues`);
for (const issue of report.issues) {
const label = issue.level === 'ERROR' ? 'ERROR' : issue.level;
const prefix = issue.level === 'ERROR' ? '✗' : issue.level === 'WARNING' ? '⚠' : 'ℹ';
console.error(`${prefix} [${label}] ${issue.path}: ${issue.message}`);
}
}
}
private async runBulkValidation(scope: { changes: boolean; specs: boolean }, opts: { strict: boolean; json: boolean; concurrency?: string }): Promise<void> {
const spinner = !opts.json ? ora('Validating...').start() : undefined;
const [changeIds, specIds] = await Promise.all([
scope.changes ? getActiveChangeIds() : Promise.resolve<string[]>([]),
scope.specs ? getSpecIds() : Promise.resolve<string[]>([]),
]);
const DEFAULT_CONCURRENCY = 6;
const maxSuggestions = 5; // used by nearestMatches
const concurrency = normalizeConcurrency(opts.concurrency) ?? normalizeConcurrency(process.env.OPENSPEC_CONCURRENCY) ?? DEFAULT_CONCURRENCY;
const validator = new Validator(opts.strict);
const queue: Array<() => Promise<BulkItemResult>> = [];
for (const id of changeIds) {
queue.push(async () => {
const start = Date.now();
const file = path.join(process.cwd(), 'openspec', 'changes', id, 'proposal.md');
const report = await validator.validateChange(file);
const durationMs = Date.now() - start;
return { id, type: 'change' as const, valid: report.valid, issues: report.issues, durationMs };
});
}
for (const id of specIds) {
queue.push(async () => {
const start = Date.now();
const file = path.join(process.cwd(), 'openspec', 'specs', id, 'spec.md');
const report = await validator.validateSpec(file);
const durationMs = Date.now() - start;
return { id, type: 'spec' as const, valid: report.valid, issues: report.issues, durationMs };
});
}
const results: BulkItemResult[] = [];
let index = 0;
let running = 0;
let passed = 0;
let failed = 0;
await new Promise<void>((resolve) => {
const next = () => {
while (running < concurrency && index < queue.length) {
const currentIndex = index++;
const task = queue[currentIndex];
running++;
if (spinner) spinner.text = `Validating (${currentIndex + 1}/${queue.length})...`;
task()
.then(res => {
results.push(res);
if (res.valid) passed++; else failed++;
})
.catch((error: any) => {
const message = error?.message || 'Unknown error';
const res: BulkItemResult = { id: getPlannedId(currentIndex, changeIds, specIds) ?? 'unknown', type: getPlannedType(currentIndex, changeIds, specIds) ?? 'change', valid: false, issues: [{ level: 'ERROR', path: 'file', message }], durationMs: 0 };
results.push(res);
failed++;
})
.finally(() => {
running--;
if (index >= queue.length && running === 0) resolve();
else next();
});
}
};
next();
});
spinner?.stop();
results.sort((a, b) => a.id.localeCompare(b.id));
const summary = {
totals: { items: results.length, passed, failed },
byType: {
...(scope.changes ? { change: summarizeType(results, 'change') } : {}),
...(scope.specs ? { spec: summarizeType(results, 'spec') } : {}),
},
} as const;
if (opts.json) {
const out = { items: results, summary, version: '1.0' };
console.log(JSON.stringify(out, null, 2));
} else {
for (const res of results) {
if (res.valid) console.log(`✓ ${res.type}/${res.id}`);
else console.error(`✗ ${res.type}/${res.id}`);
}
console.log(`Totals: ${summary.totals.passed} passed, ${summary.totals.failed} failed (${summary.totals.items} items)`);
}
process.exitCode = failed > 0 ? 1 : 0;
}
}
function summarizeType(results: BulkItemResult[], type: ItemType) {
const filtered = results.filter(r => r.type === type);
const items = filtered.length;
const passed = filtered.filter(r => r.valid).length;
const failed = items - passed;
return { items, passed, failed };
}
function nearestMatches(input: string, candidates: string[], max: number = 5): string[] {
const scored = candidates.map(c => ({ c, d: levenshtein(input, c) }));
scored.sort((a, b) => a.d - b.d);
return scored.slice(0, max).map(s => s.c);
}
function levenshtein(a: string, b: string): number {
const m = a.length;
const n = b.length;
const dp: number[][] = Array.from({ length: m + 1 }, () => Array(n + 1).fill(0));
for (let i = 0; i <= m; i++) dp[i][0] = i;
for (let j = 0; j <= n; j++) dp[0][j] = j;
for (let i = 1; i <= m; i++) {
for (let j = 1; j <= n; j++) {
const cost = a[i - 1] === b[j - 1] ? 0 : 1;
dp[i][j] = Math.min(
dp[i - 1][j] + 1,
dp[i][j - 1] + 1,
dp[i - 1][j - 1] + cost
);
}
}
return dp[m][n];
}
function normalizeConcurrency(value?: string): number | undefined {
if (!value) return undefined;
const n = parseInt(value, 10);
if (Number.isNaN(n) || n <= 0) return undefined;
return n;
}
function getPlannedId(index: number, changeIds: string[], specIds: string[]): string | undefined {
const totalChanges = changeIds.length;
if (index < totalChanges) return changeIds[index];
const specIndex = index - totalChanges;
return specIds[specIndex];
}
function getPlannedType(index: number, changeIds: string[], specIds: string[]): ItemType | undefined {
const totalChanges = changeIds.length;
if (index < totalChanges) return 'change';
const specIndex = index - totalChanges;
if (specIndex >= 0 && specIndex < specIds.length) return 'spec';
return undefined;
}
+4 -1
View File
@@ -94,7 +94,8 @@ export class ChangeParser extends MarkdownParser {
spec: specName,
operation: 'ADDED' as DeltaOperation,
description: `Add requirement: ${req.text}`,
// Use plural form to satisfy validators that expect an array
// Provide both single and plural forms for compatibility
requirement: req,
requirements: [req],
});
});
@@ -109,6 +110,7 @@ export class ChangeParser extends MarkdownParser {
spec: specName,
operation: 'MODIFIED' as DeltaOperation,
description: `Modify requirement: ${req.text}`,
requirement: req,
requirements: [req],
});
});
@@ -123,6 +125,7 @@ export class ChangeParser extends MarkdownParser {
spec: specName,
operation: 'REMOVED' as DeltaOperation,
description: `Remove requirement: ${req.text}`,
requirement: req,
requirements: [req],
});
});
+7
View File
@@ -0,0 +1,7 @@
export function isInteractive(noInteractiveFlag?: boolean): boolean {
if (noInteractiveFlag) return false;
if (process.env.OPEN_SPEC_INTERACTIVE === '0') return false;
return !!process.stdin.isTTY;
}
+38
View File
@@ -0,0 +1,38 @@
import { promises as fs } from 'fs';
import path from 'path';
export async function getActiveChangeIds(root: string = process.cwd()): Promise<string[]> {
const changesPath = path.join(root, 'openspec', 'changes');
try {
const entries = await fs.readdir(changesPath, { withFileTypes: true });
return entries
.filter(entry => entry.isDirectory() && !entry.name.startsWith('.') && entry.name !== 'archive')
.map(entry => entry.name)
.sort();
} catch {
return [];
}
}
export async function getSpecIds(root: string = process.cwd()): Promise<string[]> {
const specsPath = path.join(root, 'openspec', 'specs');
const result: string[] = [];
try {
const entries = await fs.readdir(specsPath, { withFileTypes: true });
for (const entry of entries) {
if (!entry.isDirectory() || entry.name.startsWith('.')) continue;
const specFile = path.join(specsPath, entry.name, 'spec.md');
try {
await fs.access(specFile);
result.push(entry.name);
} catch {
// ignore
}
}
} catch {
// ignore
}
return result.sort();
}
@@ -0,0 +1,51 @@
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { execSync } from 'child_process';
// Note: We cannot truly simulate TTY prompts in this test runner easily.
// Instead, we verify non-interactive fallback behavior and basic invocation.
describe('change validate (interactive behavior)', () => {
const projectRoot = process.cwd();
const testDir = path.join(projectRoot, 'test-change-validate-tmp');
const changesDir = path.join(testDir, 'openspec', 'changes');
const bin = path.join(projectRoot, 'bin', 'openspec.js');
beforeAll(() => {
execSync('pnpm -s build', { stdio: 'pipe' });
});
beforeEach(async () => {
await fs.mkdir(changesDir, { recursive: true });
const content = `# Change: Demo\n\n## Why\nBecause reasons that are sufficiently long.\n\n## What Changes\n- **spec-x:** Add something`;
await fs.mkdir(path.join(changesDir, 'demo'), { recursive: true });
await fs.writeFile(path.join(changesDir, 'demo', 'proposal.md'), content, 'utf-8');
});
afterEach(async () => {
await fs.rm(testDir, { recursive: true, force: true });
});
it('prints list hint and exits non-zero when no arg and non-interactive', () => {
const originalCwd = process.cwd();
const originalEnv = { ...process.env };
try {
process.chdir(testDir);
process.env.OPEN_SPEC_INTERACTIVE = '0';
let err: any;
try {
execSync(`node ${bin} change validate`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.status).not.toBe(0);
expect(err.stderr.toString()).toContain('Available IDs:');
expect(err.stderr.toString()).toContain('openspec change list');
} finally {
process.chdir(originalCwd);
process.env = originalEnv;
}
});
});
@@ -0,0 +1,47 @@
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { execSync } from 'child_process';
describe('spec validate (interactive behavior)', () => {
const projectRoot = process.cwd();
const testDir = path.join(projectRoot, 'test-spec-validate-tmp');
const specsDir = path.join(testDir, 'openspec', 'specs');
const bin = path.join(projectRoot, 'bin', 'openspec.js');
beforeAll(() => {
execSync('pnpm -s build', { stdio: 'pipe' });
});
beforeEach(async () => {
await fs.mkdir(specsDir, { recursive: true });
const content = `## Purpose\nValid spec for interactive test.\n\n## Requirements\n\n### Requirement: X\nText`;
await fs.mkdir(path.join(specsDir, 's1'), { recursive: true });
await fs.writeFile(path.join(specsDir, 's1', 'spec.md'), content, 'utf-8');
});
afterEach(async () => {
await fs.rm(testDir, { recursive: true, force: true });
});
it('errors when no arg and non-interactive', () => {
const originalCwd = process.cwd();
const originalEnv = { ...process.env };
try {
process.chdir(testDir);
process.env.OPEN_SPEC_INTERACTIVE = '0';
let err: any;
try {
execSync(`node ${bin} spec validate`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.status).not.toBe(0);
expect(err.stderr.toString()).toContain('Missing required argument <spec-id>');
} finally {
process.chdir(originalCwd);
process.env = originalEnv;
}
});
});
+125
View File
@@ -0,0 +1,125 @@
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { execSync } from 'child_process';
describe('top-level validate command', () => {
const projectRoot = process.cwd();
const testDir = path.join(projectRoot, 'test-validate-command-tmp');
const changesDir = path.join(testDir, 'openspec', 'changes');
const specsDir = path.join(testDir, 'openspec', 'specs');
const bin = path.join(projectRoot, 'bin', 'openspec.js');
beforeAll(() => {
execSync('pnpm -s build', { stdio: 'pipe' });
});
beforeEach(async () => {
await fs.mkdir(changesDir, { recursive: true });
await fs.mkdir(specsDir, { recursive: true });
// Create a valid spec
const specContent = `## Purpose
Valid spec for testing.
## Requirements
### Requirement: Foo
Text
#### Scenario: Bar
Given A\nWhen B\nThen C`;
await fs.mkdir(path.join(specsDir, 'alpha'), { recursive: true });
await fs.writeFile(path.join(specsDir, 'alpha', 'spec.md'), specContent, 'utf-8');
// Create a simple change with bullets (parser supports this)
const changeContent = `# Test Change\n\n## Why\nBecause reasons that are sufficiently long for validation.\n\n## What Changes\n- **alpha:** Add something`;
await fs.mkdir(path.join(changesDir, 'c1'), { recursive: true });
await fs.writeFile(path.join(changesDir, 'c1', 'proposal.md'), changeContent, 'utf-8');
// Duplicate name for ambiguity test
await fs.mkdir(path.join(changesDir, 'dup'), { recursive: true });
await fs.writeFile(path.join(changesDir, 'dup', 'proposal.md'), changeContent, 'utf-8');
await fs.mkdir(path.join(specsDir, 'dup'), { recursive: true });
await fs.writeFile(path.join(specsDir, 'dup', 'spec.md'), specContent, 'utf-8');
});
afterEach(async () => {
await fs.rm(testDir, { recursive: true, force: true });
});
it('prints a helpful hint when no args in non-interactive mode', () => {
const originalCwd = process.cwd();
const originalEnv = { ...process.env };
try {
process.chdir(testDir);
process.env.OPEN_SPEC_INTERACTIVE = '0';
let err: any;
try {
execSync(`node ${bin} validate`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.status).not.toBe(0);
expect(err.stderr.toString()).toContain('Nothing to validate. Try one of:');
} finally {
process.chdir(originalCwd);
process.env = originalEnv;
}
});
it('validates all with --all and outputs JSON summary', () => {
const originalCwd = process.cwd();
try {
process.chdir(testDir);
let outStr = '';
try {
outStr = execSync(`node ${bin} validate --all --json`, { encoding: 'utf-8' });
} catch (e: any) {
// If exit code is non-zero (e.g., on failures), still parse stdout JSON
outStr = e.stdout?.toString?.() ?? '';
}
const json = JSON.parse(outStr);
expect(Array.isArray(json.items)).toBe(true);
expect(json.summary?.totals?.items).toBeDefined();
expect(json.version).toBe('1.0');
} finally {
process.chdir(originalCwd);
}
});
it('validates only specs with --specs and respects --concurrency', () => {
const originalCwd = process.cwd();
try {
process.chdir(testDir);
let outStr = '';
try {
outStr = execSync(`node ${bin} validate --specs --json --concurrency 1`, { encoding: 'utf-8' });
} catch (e: any) {
outStr = e.stdout?.toString?.() ?? '';
}
const json = JSON.parse(outStr);
// All items should be specs
expect(json.items.every((i: any) => i.type === 'spec')).toBe(true);
} finally {
process.chdir(originalCwd);
}
});
it('errors on ambiguous item names and suggests type override', () => {
const originalCwd = process.cwd();
try {
process.chdir(testDir);
let err: any;
try {
execSync(`node ${bin} validate dup`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.stderr.toString()).toContain('Ambiguous item');
expect(err.status).not.toBe(0);
} finally {
process.chdir(originalCwd);
}
});
});