Compare commits

...
20 changed files with 1428 additions and 12 deletions
+343
View File
@@ -0,0 +1,343 @@
# Comprehensive Retrospective: Creating an OpenSpec Change Proposal
## Executive Summary
This document consolidates learnings from creating the `bulk-validation-interactive-selection` change proposal for OpenSpec. The process revealed critical gaps in documentation, unhelpful error messages, and areas where the system could be more user-friendly. While OpenSpec's core functionality works correctly, the user experience for creating changes needs significant improvement.
## Table of Contents
1. [Errors Encountered](#errors-encountered)
2. [System Issues vs User Errors](#system-issues-vs-user-errors)
3. [Documentation Gaps Analysis](#documentation-gaps-analysis)
4. [Key Learnings](#key-learnings)
5. [Recommendations](#recommendations)
6. [Conclusion](#conclusion)
---
## Errors Encountered
### Error 1: Misunderstanding Delta Structure
**Error Message:** `✗ [ERROR] deltas: Change must have at least one delta`
**What Happened:**
Initially attempted to define deltas directly in the `proposal.md` file using markdown sections like:
```markdown
### Delta: Add validate-all command
**Type**: Feature addition
**Effort**: Small (< 100 lines)
```
**Root Cause:**
Fundamental misunderstanding of how OpenSpec processes deltas. Deltas are derived from spec files in the change's `specs/` directory, not from the proposal itself.
**Discovery Process:**
- Examined `ChangeParser` class in `/src/core/parsers/change-parser.ts`
- Found that `parseDeltaSpecs()` method looks for spec files in `specs/` subdirectory
- Learned that deltas are extracted by comparing spec files against existing specs
### Error 2: Missing Operation Prefix in Section Headers
**Error Message:** `✗ [ERROR] deltas: Change must have at least one delta`
**What Happened:**
Created new spec files with standard `## Requirements` headers instead of operation-prefixed headers.
**Root Cause:**
Failed to understand that ALL spec files in a change need operation prefixes (`ADDED`, `MODIFIED`, etc.) in their section headers, regardless of whether they're new specs or modifications.
**Discovery Process:**
- Created new specs with `## Requirements` → No deltas detected
- Changed to `## ADDED Requirements` → Deltas detected successfully!
- Realized creating new specs works perfectly fine once properly formatted
**Important Clarification:**
OpenSpec fully supports creating new specs. They appear as ADDED operations in the deltas. My initial analysis incorrectly suggested this was a limitation, but it was actually just a formatting issue.
### Error 3: Improper Scenario Formatting
**Error Message:**
```
✗ [ERROR] deltas.0.requirements.0.scenarios: Requirement must have at least one scenario
✗ [ERROR] deltas.1.requirements.0.scenarios: Requirement must have at least one scenario
```
**What Happened:**
Formatted scenarios as bullet lists under a bold "Scenarios:" label:
```markdown
**Scenarios:**
- **WHEN** executing command
- **THEN** expected behavior
```
**Root Cause:**
OpenSpec's parser expects scenarios to be defined as level 4 headers (`####`) with specific formatting:
```markdown
#### Scenario: Descriptive scenario name
- **WHEN** executing command
- **THEN** expected behavior
```
**Discovery Process:**
- Checked parsed JSON output: `npx openspec change show bulk-validation-interactive-selection --json`
- Saw `"scenarios": []` empty array despite having scenario content
- Examined working spec files and found the `#### Scenario:` header pattern
### Error 4: File Path Confusion
**Initial Confusion:**
Wasn't clear whether to create specs that would become part of the main `openspec/specs/` or just define them in the change.
**Resolution:**
Learned that changes can:
1. Create new specs (they start in `changes/{change-name}/specs/` and move to `openspec/specs/` when archived)
2. Modify existing specs (by creating a spec file with the same name as one in `openspec/specs/`)
3. The validation system detects both patterns and creates appropriate deltas
---
## System Issues vs User Errors
### System Issues / Bugs
#### 1. Unhelpful Error Messages ⚠️
**Issue:** `✗ [ERROR] deltas: Change must have at least one delta`
**Why This Is a System Problem:**
- Error message provides no guidance on HOW to create deltas
- Doesn't mention that deltas come from `specs/` subdirectory
- Doesn't explain the required section headers
- A better error would be: "No deltas found. Ensure your change has a specs/ directory with .md files containing sections like '## ADDED Requirements'"
#### 2. Silent Scenario Parsing Failures ⚠️
**Issue:** When scenarios were formatted incorrectly, they were silently ignored
**Why This Is a System Problem:**
- Parser silently returns empty scenarios array instead of warning
- No validation error explaining the format issue
- User gets "Requirement must have at least one scenario" without knowing their scenarios exist but aren't parsed
**Evidence:**
```json
{
"text": "The CLI SHALL provide a top-level `show` command with interactive selection.",
"scenarios": [] // Silent failure - scenarios existed but weren't parsed
}
```
#### 3. No Validation for Proposal Structure During Creation
**Issue:** System allows creating invalid proposals without early feedback
**Why This Is a System Problem:**
- No scaffolding or template commands
- No incremental validation as you build
- Must fully create the change before discovering structural issues
### User Errors (My Mistakes)
#### 1. Trying to Define Deltas in Proposal.md
- Incorrectly assumed deltas could be inline in the proposal
- System correctly expects deltas in separate spec files
#### 2. Using Wrong Section Headers
- Used `## Requirements` instead of `## ADDED Requirements`
- Convention is documented in existing changes, but I didn't examine carefully
#### 3. Wrong Scenario Format
- Used bullet lists instead of `#### Scenario:` headers
- Made assumptions instead of checking existing patterns
### Gray Areas
1. **Documentation gaps** - While examples exist, there's no comprehensive "How to Create a Change" guide
2. **Lack of tooling** - No scaffolding commands to create properly structured changes
---
## Documentation Gaps Analysis
### Critical Gaps in openspec/README.md
#### 1. Scenario Format - COMPLETELY MISSING ⚠️
**What README Shows:** No scenario examples at all
**What's Actually Required:**
```markdown
#### Scenario: Descriptive name
- **WHEN** condition
- **THEN** expected outcome
- **AND** additional outcomes
```
**Impact:** This was the biggest struggle. The README mentions requirements but never shows how to write scenarios. Without this, requirements fail validation.
#### 2. Complete Spec File Example - MISSING
**What README Shows:** Only fragments
**What's Actually Needed:** A complete working example showing:
- Full spec file structure
- Proper requirement format
- Scenario formatting
- All required elements
#### 3. Validation Commands - NOT MENTIONED
**Missing from README:**
- `npx openspec change validate <change-name>`
- `npx openspec change show <change-name> --json`
- The `--strict` flag for catching warnings
#### 4. Delta Detection Explanation - INCOMPLETE
**What's Missing:**
- WHERE the system looks for specs (specs/ subdirectory)
- THAT deltas are automatically extracted
- HOW to debug when deltas aren't detected
- WHAT error messages mean
### Misleading Documentation
#### "Store only the changes" - MISLEADING
**Line 145:** `# - Store only the changes (not complete future state)`
**Problem:** This suggests storing diffs or partial content. In reality, you need:
- Complete requirements in their final form
- Full scenario definitions
- The entire requirement text
### Documentation That Was Helpful
1. Delta section headers (`## ADDED Requirements`) - clearly documented
2. Directory structure - excellent visualization
3. When to create proposals - well defined
---
## Key Learnings
### 1. OpenSpec's Delta Detection Algorithm
The system follows this process:
1. Scans `openspec/changes/{change-name}/specs/` directory
2. For each spec file found, parses for delta sections (`ADDED`, `MODIFIED`, `REMOVED`, `RENAMED`)
3. Creates delta objects with operation type, affected spec, and requirements
4. Validates that at least one delta exists for the change to be valid
**Important:** Creating entirely new specs is fully supported! New specs use `## ADDED Requirements` and appear as ADDED operations in the deltas.
### 2. Spec File Structure Requirements
Valid spec files must follow this structure:
```markdown
# Spec Title
## [ADDED|MODIFIED|REMOVED|RENAMED] Requirements
### Requirement: Clear requirement statement
The requirement description using SHALL/SHOULD/MAY.
#### Scenario: Scenario name
- **WHEN** condition
- **THEN** expected outcome
- **AND** additional outcomes
```
### 3. Change Proposal Structure
A valid change must have:
- `## Why` section - explaining the motivation
- `## What Changes` section - summarizing the changes
- `specs/` directory with properly formatted spec files containing deltas
- Each delta must have at least one requirement with at least one scenario
### 4. Validation Commands Are Essential
```bash
# Basic validation
npx openspec change validate {change-name}
# Strict validation (recommended)
npx openspec change validate {change-name} --strict
# Debug delta detection
npx openspec change show {change-name} --json | jq '.deltas'
```
---
## Recommendations
### High Priority (System Bugs to Fix)
1. **Improve Error Messages**
- Add actionable guidance to error messages
- Example: "No deltas found. Check: 1) specs/ directory exists, 2) Files use ## ADDED Requirements headers, 3) Each requirement has #### Scenario: sections"
2. **Add Warnings for Malformed Content**
- Warn when scenarios exist but aren't properly formatted
- Show which line/file has the issue
3. **Add Delta Detection Debugging**
- Command like `openspec change debug-deltas {change-name}`
- Show which files were scanned, what was found, what was rejected
### Medium Priority (Documentation Improvements)
1. **Add Complete Working Example to README**
- Full change proposal with all files
- Properly formatted specs with scenarios
- Show the validation output
2. **Add Troubleshooting Section**
- Common errors and their solutions
- How to debug delta detection
- Scenario formatting requirements
3. **Add Validation Best Practices**
- When to use `--strict`
- How to use JSON output for debugging
- Common validation patterns
### Low Priority (Developer Experience)
1. **Add Scaffolding Command**
```bash
openspec change scaffold {change-name}
```
- Creates proper directory structure
- Includes template files with correct formatting
- Adds example scenarios
2. **Add Interactive Creation Wizard**
- Guide users through change creation
- Validate as they go
- Suggest fixes for common issues
3. **Add Auto-fix Capability**
- `--fix` flag to correct common formatting issues
- Convert bullet list scenarios to proper headers
- Add missing operation prefixes
---
## Conclusion
The OpenSpec system works correctly for its intended design, but the user experience for creating changes needs significant improvement. The core issues stem from:
### System Issues
- **Unhelpful error messages** that don't guide users to solutions
- **Silent parsing failures** that provide no feedback about malformed content
- **Lack of debugging tools** to understand what went wrong
### Documentation Issues
- **Critical formatting requirements missing** (especially scenario format)
- **No complete working examples** showing all required elements
- **Validation commands not documented** despite being essential
### User Issues
- **Incorrect assumptions** about how the system works
- **Not examining existing patterns** carefully enough
- **Trying to shortcut** instead of following established conventions
### The Path Forward
With better error messages, complete documentation, and basic tooling support, most of the errors encountered could be prevented. The system's delta-centric approach is powerful and ensures changes are atomic and trackable, but it needs to be more discoverable and user-friendly.
The most impactful improvements would be:
1. Adding scenario format documentation to the README
2. Improving error messages with actionable guidance
3. Creating a scaffolding command for new changes
These changes would transform OpenSpec from a system that works correctly but is hard to use, into one that actively helps developers succeed.
@@ -0,0 +1,20 @@
## Why
Users frequently need to view changes and specs but must know in advance whether they're looking at a change or spec. The current subcommand structure (`change show`, `spec show`) creates friction when:
- Users want to quickly view an item without remembering its type
- Exploring the codebase requires switching between different show commands
- Show commands without arguments return errors instead of helpful guidance
## What Changes
- Add new top-level `show` command for displaying changes or specs with intelligent selection
- Support direct item display: `openspec show <item>` with automatic type detection
- Interactive selection when no arguments provided
- Enhance existing `change show` and `spec show` to support interactive selection (backwards compatibility)
- Maintain all existing format options (--json, --deltas-only, --requirements, etc.)
## Impact
- New specs to create: cli-show
- Specs to enhance: cli-change, cli-spec (for backwards compatibility)
- Affected code: src/cli/index.ts, src/commands/show.ts (new), src/commands/spec.ts, src/commands/change.ts
@@ -0,0 +1,23 @@
# CLI Change Command Spec
## ADDED Requirements
### Requirement: Interactive show selection
The change show command SHALL support interactive selection when no change name is provided.
#### Scenario: Interactive change selection for show
- **WHEN** executing `openspec change show` without arguments
- **THEN** display an interactive list of available changes
- **AND** allow the user to select a change to show
- **AND** display the selected change content
- **AND** maintain all existing show options (--json, --deltas-only)
#### Scenario: Non-interactive fallback keeps current behavior
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
- **WHEN** executing `openspec change show` without a change name
- **THEN** do not prompt interactively
- **AND** print the existing hint including available change IDs
- **AND** set `process.exitCode = 1`
@@ -0,0 +1,83 @@
# CLI Show Command Spec
## ADDED Requirements
### Requirement: Top-level show command
The CLI SHALL provide a top-level `show` command for displaying changes and specs with intelligent selection.
#### Scenario: Interactive show selection
- **WHEN** executing `openspec show` without arguments
- **THEN** prompt user to select type (change or spec)
- **AND** display list of available items for selected type
- **AND** show the selected item's content
#### Scenario: Non-interactive environments do not prompt
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
- **WHEN** executing `openspec show` without arguments
- **THEN** do not prompt
- **AND** print a helpful hint with examples for `openspec show <item>` or `openspec change/spec show`
- **AND** exit with code 1
#### Scenario: Direct item display
- **WHEN** executing `openspec show <item-name>`
- **THEN** automatically detect if item is a change or spec
- **AND** display the item's content
- **AND** use appropriate formatting based on item type
#### Scenario: Type detection and ambiguity handling
- **WHEN** executing `openspec show <item-name>`
- **THEN** if `<item-name>` uniquely matches a change or a spec, show that item
- **AND** if it matches both, print an ambiguity error and suggest `--type change|spec` or using `openspec change show`/`openspec spec show`
- **AND** if it matches neither, print not-found with nearest-match suggestions
#### Scenario: Explicit type override
- **WHEN** executing `openspec show --type change <item>`
- **THEN** treat `<item>` as a change ID and show it (skipping auto-detection)
- **WHEN** executing `openspec show --type spec <item>`
- **THEN** treat `<item>` as a spec ID and show it (skipping auto-detection)
### Requirement: Output format options
The show command SHALL support various output formats consistent with existing commands.
#### Scenario: JSON output
- **WHEN** executing `openspec show <item> --json`
- **THEN** output the item in JSON format
- **AND** include parsed metadata and structure
- **AND** maintain format consistency with existing change/spec show commands
#### Scenario: Flag scoping and delegation
- **WHEN** showing a change or a spec via the top-level command
- **THEN** accept common flags such as `--json`
- **AND** pass through type-specific flags to the corresponding implementation
- Change-only flags: `--deltas-only` (alias `--requirements-only` deprecated)
- Spec-only flags: `--requirements`, `--no-scenarios`, `-r/--requirement`
- **AND** ignore irrelevant flags for the detected type with a warning
### Requirement: Interactivity controls
- The CLI SHALL respect `--no-interactive` to disable prompts.
- The CLI SHALL respect `OPEN_SPEC_INTERACTIVE=0` to disable prompts globally.
- Interactive prompts SHALL only be shown when stdin is a TTY and interactivity is not disabled.
#### Scenario: Change-specific options
- **WHEN** showing a change with `openspec show <change-name> --deltas-only`
- **THEN** display only the deltas in JSON format
- **AND** maintain compatibility with existing change show options
#### Scenario: Spec-specific options
- **WHEN** showing a spec with `openspec show <spec-id> --requirements`
- **THEN** display only requirements in JSON format
- **AND** support other spec options (--no-scenarios, -r)
- **AND** maintain compatibility with existing spec show options
@@ -0,0 +1,23 @@
# CLI Spec Command Spec
## ADDED Requirements
### Requirement: Interactive spec show
The spec show command SHALL support interactive selection when no spec-id is provided.
#### Scenario: Interactive spec selection for show
- **WHEN** executing `openspec spec show` without arguments
- **THEN** display an interactive list of available specs
- **AND** allow the user to select a spec to show
- **AND** display the selected spec content
- **AND** maintain all existing show options (--json, --requirements, --no-scenarios, -r)
#### Scenario: Non-interactive fallback keeps current behavior
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
- **WHEN** executing `openspec spec show` without a spec-id
- **THEN** do not prompt interactively
- **AND** print the existing error message for missing spec-id
- **AND** set non-zero exit code
@@ -0,0 +1,20 @@
## Why
Currently, users must validate changes and specs individually by specifying each ID. This creates friction when:
- Teams want to validate all changes/specs before a release
- Developers need to ensure consistency across multiple related changes
- Users run validation commands without arguments and receive errors instead of helpful guidance
- The subcommand structure requires users to know in advance whether they're validating a change or spec
## What Changes
- Add new top-level `validate` command with intuitive flags (--all, --changes, --specs)
- Enhance existing `change validate` and `spec validate` to support interactive selection (backwards compatibility)
- Interactive selection by default when no arguments provided
- Support direct item validation: `openspec validate <item>` with automatic type detection
## Impact
- New specs to create: cli-validate
- Specs to enhance: cli-change, cli-spec (for backwards compatibility)
- Affected code: src/cli/index.ts, src/commands/validate.ts (new), src/commands/spec.ts, src/commands/change.ts
@@ -0,0 +1,22 @@
# CLI Change Command Spec
## ADDED Requirements
### Requirement: Interactive validation selection
The change validate command SHALL support interactive selection when no change name is provided.
#### Scenario: Interactive change selection for validation
- **WHEN** executing `openspec change validate` without arguments
- **THEN** display an interactive list of available changes
- **AND** allow the user to select a change to validate
- **AND** validate the selected change
#### Scenario: Non-interactive fallback keeps current behavior
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
- **WHEN** executing `openspec change validate` without a change name
- **THEN** do not prompt interactively
- **AND** print the existing hint including available change IDs
- **AND** set `process.exitCode = 1`
@@ -0,0 +1,23 @@
# CLI Spec Command Spec
## ADDED Requirements
### Requirement: Interactive spec validation
The spec validate command SHALL support interactive selection when no spec-id is provided.
#### Scenario: Interactive spec selection for validation
- **WHEN** executing `openspec spec validate` without arguments
- **THEN** display an interactive list of available specs
- **AND** allow the user to select a spec to validate
- **AND** validate the selected spec
- **AND** maintain all existing validation options (--strict, --json)
#### Scenario: Non-interactive fallback keeps current behavior
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
- **WHEN** executing `openspec spec validate` without a spec-id
- **THEN** do not prompt interactively
- **AND** print the existing error message for missing spec-id
- **AND** set non-zero exit code
@@ -0,0 +1,141 @@
# CLI Validate Command Spec
## ADDED Requirements
### Requirement: Top-level validate command
The CLI SHALL provide a top-level `validate` command for validating changes and specs with flexible selection options.
#### Scenario: Interactive validation selection
- **WHEN** executing `openspec validate` without arguments
- **THEN** prompt user to select what to validate (all, changes, specs, or specific item)
- **AND** perform validation based on selection
- **AND** display results with appropriate formatting
#### Scenario: Non-interactive environments do not prompt
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
- **WHEN** executing `openspec validate` without arguments
- **THEN** do not prompt interactively
- **AND** print a helpful hint listing available commands/flags and exit with code 1
#### Scenario: Direct item validation
- **WHEN** executing `openspec validate <item-name>`
- **THEN** automatically detect if item is a change or spec
- **AND** validate the specified item
- **AND** display validation results
### Requirement: Bulk and filtered validation
The validate command SHALL support flags for bulk validation (--all) and filtered validation by type (--changes, --specs).
#### Scenario: Validate everything
- **WHEN** executing `openspec validate --all`
- **THEN** validate all changes in openspec/changes/ (excluding archive)
- **AND** validate all specs in openspec/specs/
- **AND** display a summary showing passed/failed items
- **AND** exit with code 1 if any validation fails
#### Scenario: Scope of bulk validation
- **WHEN** validating with `--all` or `--changes`
- **THEN** include all change proposals under `openspec/changes/`
- **AND** exclude the `openspec/changes/archive/` directory
- **WHEN** validating with `--specs`
- **THEN** include all specs that have a `spec.md` under `openspec/specs/<id>/spec.md`
#### Scenario: Validate all changes
- **WHEN** executing `openspec validate --changes`
- **THEN** validate all changes in openspec/changes/ (excluding archive)
- **AND** display results for each change
- **AND** show summary statistics
#### Scenario: Validate all specs
- **WHEN** executing `openspec validate --specs`
- **THEN** validate all specs in openspec/specs/
- **AND** display results for each spec
- **AND** show summary statistics
### Requirement: Validation options and progress indication
The validate command SHALL support standard validation options (--strict, --json) and display progress during bulk operations.
#### Scenario: Strict validation
- **WHEN** executing `openspec validate --all --strict`
- **THEN** apply strict validation to all items
- **AND** treat warnings as errors
- **AND** fail if any item has warnings or errors
#### Scenario: JSON output
- **WHEN** executing `openspec validate --all --json`
- **THEN** output validation results as JSON
- **AND** include detailed issues for each item
- **AND** include summary statistics
#### Scenario: JSON output schema for bulk validation
- **WHEN** executing `openspec validate --all --json` (or `--changes` / `--specs`)
- **THEN** output a JSON object with the following shape:
- `items`: Array of objects with fields `{ id: string, type: "change"|"spec", valid: boolean, issues: Issue[], durationMs: number }`
- `summary`: Object `{ totals: { items: number, passed: number, failed: number }, byType: { change?: { items: number, passed: number, failed: number }, spec?: { items: number, passed: number, failed: number } } }`
- `version`: String identifier for the schema (e.g., `"1.0"`)
- **AND** exit with code 1 if any `items[].valid === false`
Where `Issue` follows the existing per-item validation report shape `{ level: "ERROR"|"WARNING"|"INFO", path: string, message: string }`.
#### Scenario: Show validation progress
- **WHEN** validating multiple items (--all, --changes, or --specs)
- **THEN** show progress indicator or status updates
- **AND** indicate which item is currently being validated
- **AND** display running count of passed/failed items
#### Scenario: Concurrency limits for performance
- **WHEN** validating multiple items
- **THEN** run validations with a bounded concurrency (e.g., 4–8 in parallel)
- **AND** ensure progress indicators remain responsive
### Requirement: Item type detection and ambiguity handling
#### Scenario: Direct item validation with automatic type detection
- **WHEN** executing `openspec validate <item-name>`
- **THEN** if `<item-name>` uniquely matches a change or a spec, validate that item
#### Scenario: Ambiguity between change and spec names
- **GIVEN** `<item-name>` exists both as a change and as a spec
- **WHEN** executing `openspec validate <item-name>`
- **THEN** print an ambiguity error explaining both matches
- **AND** suggest passing `--type change` or `--type spec`, or using `openspec change validate` / `openspec spec validate`
- **AND** exit with code 1 without performing validation
#### Scenario: Unknown item name
- **WHEN** the `<item-name>` matches neither a change nor a spec
- **THEN** print a not-found error
- **AND** show nearest-match suggestions when available
- **AND** exit with code 1
#### Scenario: Explicit type override
- **WHEN** executing `openspec validate --type change <item>`
- **THEN** treat `<item>` as a change ID and validate it (skipping auto-detection)
- **WHEN** executing `openspec validate --type spec <item>`
- **THEN** treat `<item>` as a spec ID and validate it (skipping auto-detection)
### Requirement: Interactivity controls
- The CLI SHALL respect `--no-interactive` to disable prompts.
- The CLI SHALL respect `OPEN_SPEC_INTERACTIVE=0` to disable prompts globally.
- Interactive prompts SHALL only be shown when stdin is a TTY and interactivity is not disabled.
@@ -0,0 +1,81 @@
# Implementation Tasks
## 1. Change Command: Interactive Validation Selection
- [x] 1.1 Add `--no-interactive` flag to `change validate` in `src/cli/index.ts`
- [x] 1.2 Implement interactivity gate respecting TTY and `OPEN_SPEC_INTERACTIVE=0` in `src/commands/change.ts`
- [x] 1.3 When no `[change-name]` is provided and interactivity is allowed, prompt with a list of active changes (exclude `archive/`) and validate the selected one
- [x] 1.4 Preserve current non-interactive fallback: print available change IDs and hint, set `process.exitCode = 1`
- [x] 1.5 Tests: add coverage for interactive and non-interactive flows
- Added `test/commands/change.interactive-validate.test.ts`
## 2. Spec Command: Interactive Validation Selection
- [x] 2.1 Make `spec validate` accept optional `[spec-id]` in `src/commands/spec.ts` registration
- [x] 2.2 Add `--no-interactive` flag to `spec validate`
- [x] 2.3 Implement interactivity gate respecting TTY and `OPEN_SPEC_INTERACTIVE=0`
- [x] 2.4 When no `[spec-id]` provided and interactivity allowed, prompt to select from `openspec/specs/*/spec.md` and validate the selected spec
- [x] 2.5 Preserve current non-interactive fallback when no spec-id and no interactivity: print existing error and exit code non-zero
- [x] 2.6 Tests: add coverage for interactive and non-interactive flows
- Added `test/commands/spec.interactive-validate.test.ts`
## 3. New Top-level `validate` Command
- [x] 3.1 Add `validate` command in `src/cli/index.ts`
- Options: `--all`, `--changes`, `--specs`, `--type <change|spec>`, `--strict`, `--json`, `--no-interactive`
- Usage: `openspec validate [item-name]`
- [x] 3.2 Create `src/commands/validate.ts` implementing:
- [x] 3.2.1 Interactive selector when no args (choices: All, Changes, Specs, Specific item)
- [x] 3.2.2 Non-interactive fallback with helpful hint and exit code 1
- [x] 3.2.3 Direct item validation with automatic type detection
- [x] 3.2.4 Ambiguity error when name exists as both change and spec; suggest `--type` or subcommands
- [x] 3.2.5 Unknown item handling with nearest-match suggestions
- [x] 3.2.6 Bulk validation for `--all`, `--changes`, `--specs` (exclude `openspec/changes/archive/`)
- [x] 3.2.7 Respect `--strict` and `--json` options; JSON shape per spec
- [x] 3.2.8 Exit with code 1 if any validation fails
- [x] 3.2.9 Bounded concurrency (default 4–8) for bulk validation
- [x] 3.2.10 Progress indication during bulk runs (current item, running counts)
## 4. Utilities and Shared Helpers
- [x] 4.1 Add `src/utils/interactive.ts` with `isInteractive(stdin: NodeJS.ReadStream, noInteractiveFlag?: boolean): boolean`
- Considers: `process.stdin.isTTY`, `--no-interactive`, `OPEN_SPEC_INTERACTIVE=0`
- [x] 4.2 Add `src/utils/item-discovery.ts` with:
- `getActiveChangeIds(root = process.cwd()): Promise<string[]>` (exclude `archive/`)
- `getSpecIds(root = process.cwd()): Promise<string[]>` (folders with `spec.md`)
- [ ] 4.3 Optional: `src/utils/concurrency.ts` helper for bounded parallelism
- [x] 4.4 Reuse `src/core/validation/validator.ts` for item validation
## 5. JSON Output (Bulk Validation)
- [x] 5.1 Implement JSON schema:
- `items: Array<{ id: string, type: "change"|"spec", valid: boolean, issues: Issue[], durationMs: number }>`
- `summary: { totals: { items: number, passed: number, failed: number }, byType: { change?: { items: number, passed: number, failed: number }, spec?: { items: number, passed: number, failed: number } } }`
- `version: "1.0"`
- [x] 5.2 Ensure process exit code is 1 if any `items[].valid === false`
- [x] 5.3 Tests for JSON shape (keys, types, counts) and exit code behavior
- Added `test/commands/validate.test.ts`
## 6. Progress and UX
- [x] 6.1 Use `ora` or minimal console progress to show current item and running counts
- [x] 6.2 Keep output stable in `--json` mode (no extra logs to stdout; use stderr for progress if needed)
- [x] 6.3 Ensure responsiveness with concurrency limits
## 7. Tests
- [x] 7.1 Add top-level validate tests: `test/commands/validate.test.ts`
- Includes non-interactive hint, --all JSON, --specs with concurrency, ambiguity error
- [ ] 7.2 Add unit tests for `isInteractive` and item discovery helpers
- [x] 7.3 Extend existing change/spec command tests to cover interactive `validate`
- Added `test/commands/change.interactive-validate.test.ts`, `test/commands/spec.interactive-validate.test.ts`
## 8. CLI Help and Docs
- [x] 8.1 Update command descriptions/options in `src/cli/index.ts`
- [x] 8.2 Verify help output includes `validate` command and flags
- [x] 8.3 Ensure existing specs under `openspec/changes/bulk-validation-interactive-selection/specs/*` remain satisfied
## 9. Non-functional
- [x] 9.1 Code style and types: explicit types for exported APIs; avoid `any`
- [x] 9.2 No linter errors; stable formatting; avoid unrelated refactors
- [x] 9.3 Maintain existing behavior for unaffected commands
## 10. Acceptance Criteria Mapping
- [x] AC-1: `openspec change validate` interactive selection when no arg (TTY only; respects `--no-interactive`/env) — matches cli-change spec
- [x] AC-2: `openspec spec validate` interactive selection when no arg (TTY only; respects `--no-interactive`/env) — matches cli-spec spec
- [x] AC-3: New `openspec validate` supports interactive selection, bulk/filtered validation, JSON schema, progress, concurrency, exit codes — matches cli-validate spec
+26 -1
View File
@@ -9,6 +9,7 @@ import { ListCommand } from '../core/list.js';
import { ArchiveCommand } from '../core/archive.js';
import { registerSpecCommand } from '../commands/spec.js';
import { ChangeCommand } from '../commands/change.js';
import { ValidateCommand } from '../commands/validate.js';
const program = new Command();
@@ -146,7 +147,8 @@ changeCmd
.description('Validate a change proposal')
.option('--strict', 'Enable strict validation mode')
.option('--json', 'Output validation report as JSON')
.action(async (changeName?: string, options?: { strict?: boolean; json?: boolean }) => {
.option('--no-interactive', 'Disable interactive prompts')
.action(async (changeName?: string, options?: { strict?: boolean; json?: boolean; noInteractive?: boolean }) => {
try {
const changeCommand = new ChangeCommand();
await changeCommand.validate(changeName, options);
@@ -175,4 +177,27 @@ program
registerSpecCommand(program);
// Top-level validate command
program
.command('validate [item-name]')
.description('Validate changes and specs')
.option('--all', 'Validate all changes and specs')
.option('--changes', 'Validate all changes')
.option('--specs', 'Validate all specs')
.option('--type <type>', 'Specify item type when ambiguous: change|spec')
.option('--strict', 'Enable strict validation mode')
.option('--json', 'Output validation results as JSON')
.option('--concurrency <n>', 'Max concurrent validations (defaults to env OPENSPEC_CONCURRENCY or 6)')
.option('--no-interactive', 'Disable interactive prompts')
.action(async (itemName?: string, options?: { all?: boolean; changes?: boolean; specs?: boolean; type?: string; strict?: boolean; json?: boolean; noInteractive?: boolean; concurrency?: string }) => {
try {
const validateCommand = new ValidateCommand();
await validateCommand.execute(itemName, options);
} catch (error) {
console.log();
ora().fail(`Error: ${(error as Error).message}`);
process.exit(1);
}
});
program.parse();
+20 -8
View File
@@ -1,9 +1,12 @@
import { promises as fs } from 'fs';
import path from 'path';
import { select } from '@inquirer/prompts';
import { JsonConverter } from '../core/converters/json-converter.js';
import { Validator } from '../core/validation/validator.js';
import { ChangeParser } from '../core/parsers/change-parser.js';
import { Change } from '../core/schemas/index.js';
import { isInteractive } from '../utils/interactive.js';
import { getActiveChangeIds } from '../utils/item-discovery.js';
// Constants for better maintainability
const ARCHIVE_DIR = 'archive';
@@ -170,19 +173,28 @@ export class ChangeCommand {
}
}
async validate(changeName?: string, options?: { strict?: boolean; json?: boolean }): Promise<void> {
async validate(changeName?: string, options?: { strict?: boolean; json?: boolean; noInteractive?: boolean }): Promise<void> {
const changesPath = path.join(process.cwd(), 'openspec', 'changes');
if (!changeName) {
const changes = await this.getActiveChanges(changesPath);
if (changes.length === 0) {
console.error('No change specified. No active changes found.');
const canPrompt = isInteractive(options?.noInteractive);
const changes = await getActiveChangeIds();
if (canPrompt && changes.length > 0) {
const selected = await select({
message: 'Select a change to validate',
choices: changes.map(id => ({ name: id, value: id })),
});
changeName = selected;
} else {
console.error(`No change specified. Available IDs: ${changes.join(', ')}`);
if (changes.length === 0) {
console.error('No change specified. No active changes found.');
} else {
console.error(`No change specified. Available IDs: ${changes.join(', ')}`);
}
console.error('Hint: use "openspec change list" to view available changes.');
process.exitCode = 1;
return;
}
console.error('Hint: use "openspec change list" to view available changes.');
process.exitCode = 1;
return;
}
const proposalPath = path.join(changesPath, changeName, 'proposal.md');
+19 -2
View File
@@ -4,6 +4,9 @@ import { join } from 'path';
import { MarkdownParser } from '../core/parsers/markdown-parser.js';
import { Validator } from '../core/validation/validator.js';
import type { Spec } from '../core/schemas/index.js';
import { select } from '@inquirer/prompts';
import { isInteractive } from '../utils/interactive.js';
import { getSpecIds } from '../utils/item-discovery.js';
const SPECS_DIR = 'openspec/specs';
@@ -166,12 +169,26 @@ export function registerSpecCommand(rootProgram: typeof program) {
});
specCommand
.command('validate <spec-id>')
.command('validate [spec-id]')
.description('Validate a specification structure')
.option('--strict', 'Enable strict validation mode')
.option('--json', 'Output validation report as JSON')
.action(async (specId: string, options: { strict?: boolean; json?: boolean }) => {
.option('--no-interactive', 'Disable interactive prompts')
.action(async (specId: string | undefined, options: { strict?: boolean; json?: boolean; noInteractive?: boolean }) => {
try {
if (!specId) {
const canPrompt = isInteractive(options?.noInteractive);
const specIds = await getSpecIds();
if (canPrompt && specIds.length > 0) {
specId = await select({
message: 'Select a spec to validate',
choices: specIds.map(id => ({ name: id, value: id })),
});
} else {
throw new Error('Missing required argument <spec-id>');
}
}
const specPath = join(SPECS_DIR, specId, 'spec.md');
if (!existsSync(specPath)) {
+312
View File
@@ -0,0 +1,312 @@
import { select } from '@inquirer/prompts';
import ora from 'ora';
import path from 'path';
import { Validator } from '../core/validation/validator.js';
import { isInteractive } from '../utils/interactive.js';
import { getActiveChangeIds, getSpecIds } from '../utils/item-discovery.js';
type ItemType = 'change' | 'spec';
interface ExecuteOptions {
all?: boolean;
changes?: boolean;
specs?: boolean;
type?: string;
strict?: boolean;
json?: boolean;
noInteractive?: boolean;
concurrency?: string;
}
interface BulkItemResult {
id: string;
type: ItemType;
valid: boolean;
issues: { level: 'ERROR' | 'WARNING' | 'INFO'; path: string; message: string }[];
durationMs: number;
}
export class ValidateCommand {
async execute(itemName: string | undefined, options: ExecuteOptions = {}): Promise<void> {
const interactive = isInteractive(options.noInteractive);
// Handle bulk flags first
if (options.all || options.changes || options.specs) {
await this.runBulkValidation({
changes: !!options.all || !!options.changes,
specs: !!options.all || !!options.specs,
}, { strict: !!options.strict, json: !!options.json, concurrency: options.concurrency });
return;
}
// No item and no flags
if (!itemName) {
if (interactive) {
await this.runInteractiveSelector({ strict: !!options.strict, json: !!options.json, concurrency: options.concurrency });
return;
}
this.printNonInteractiveHint();
process.exitCode = 1;
return;
}
// Direct item validation with type detection or override
const typeOverride = this.normalizeType(options.type);
await this.validateDirectItem(itemName, { typeOverride, strict: !!options.strict, json: !!options.json });
}
private normalizeType(value?: string): ItemType | undefined {
if (!value) return undefined;
const v = value.toLowerCase();
if (v === 'change' || v === 'spec') return v;
return undefined;
}
private async runInteractiveSelector(opts: { strict: boolean; json: boolean; concurrency?: string }): Promise<void> {
const choice = await select({
message: 'What would you like to validate?',
choices: [
{ name: 'All (changes + specs)', value: 'all' },
{ name: 'All changes', value: 'changes' },
{ name: 'All specs', value: 'specs' },
{ name: 'Pick a specific change or spec', value: 'one' },
],
});
if (choice === 'all') return this.runBulkValidation({ changes: true, specs: true }, opts);
if (choice === 'changes') return this.runBulkValidation({ changes: true, specs: false }, opts);
if (choice === 'specs') return this.runBulkValidation({ changes: false, specs: true }, opts);
// one
const [changes, specs] = await Promise.all([getActiveChangeIds(), getSpecIds()]);
const items: { name: string; value: { type: ItemType; id: string } }[] = [];
items.push(...changes.map(id => ({ name: `change/${id}`, value: { type: 'change' as const, id } })));
items.push(...specs.map(id => ({ name: `spec/${id}`, value: { type: 'spec' as const, id } })));
if (items.length === 0) {
console.error('No items found to validate.');
process.exitCode = 1;
return;
}
const picked = await select<{ type: ItemType; id: string }>({ message: 'Pick an item', choices: items });
await this.validateByType(picked.type, picked.id, opts);
}
private printNonInteractiveHint(): void {
console.error('Nothing to validate. Try one of:');
console.error(' openspec validate --all');
console.error(' openspec validate --changes');
console.error(' openspec validate --specs');
console.error(' openspec validate <item-name>');
console.error('Or run in an interactive terminal.');
}
private async validateDirectItem(itemName: string, opts: { typeOverride?: ItemType; strict: boolean; json: boolean }): Promise<void> {
const [changes, specs] = await Promise.all([getActiveChangeIds(), getSpecIds()]);
const isChange = changes.includes(itemName);
const isSpec = specs.includes(itemName);
const type = opts.typeOverride ?? (isChange ? 'change' : isSpec ? 'spec' : undefined);
if (!type) {
console.error(`Unknown item '${itemName}'`);
const suggestions = nearestMatches(itemName, [...changes, ...specs]);
if (suggestions.length) console.error(`Did you mean: ${suggestions.join(', ')}?`);
process.exitCode = 1;
return;
}
if (!opts.typeOverride && isChange && isSpec) {
console.error(`Ambiguous item '${itemName}' matches both a change and a spec.`);
console.error('Pass --type change|spec, or use: openspec change validate / openspec spec validate');
process.exitCode = 1;
return;
}
await this.validateByType(type, itemName, opts);
}
private async validateByType(type: ItemType, id: string, opts: { strict: boolean; json: boolean }): Promise<void> {
const validator = new Validator(opts.strict);
if (type === 'change') {
const file = path.join(process.cwd(), 'openspec', 'changes', id, 'proposal.md');
const start = Date.now();
const report = await validator.validateChange(file);
const durationMs = Date.now() - start;
this.printReport('change', id, report, durationMs, opts.json);
process.exitCode = report.valid ? 0 : 1;
return;
}
const file = path.join(process.cwd(), 'openspec', 'specs', id, 'spec.md');
const start = Date.now();
const report = await validator.validateSpec(file);
const durationMs = Date.now() - start;
this.printReport('spec', id, report, durationMs, opts.json);
process.exitCode = report.valid ? 0 : 1;
}
private printReport(type: ItemType, id: string, report: { valid: boolean; issues: any[] }, durationMs: number, json: boolean): void {
if (json) {
const out = { items: [{ id, type, valid: report.valid, issues: report.issues, durationMs }], summary: { totals: { items: 1, passed: report.valid ? 1 : 0, failed: report.valid ? 0 : 1 }, byType: { [type]: { items: 1, passed: report.valid ? 1 : 0, failed: report.valid ? 0 : 1 } } }, version: '1.0' };
console.log(JSON.stringify(out, null, 2));
return;
}
if (report.valid) {
console.log(`${type === 'change' ? 'Change' : 'Specification'} '${id}' is valid`);
} else {
console.error(`${type === 'change' ? 'Change' : 'Specification'} '${id}' has issues`);
for (const issue of report.issues) {
const label = issue.level === 'ERROR' ? 'ERROR' : issue.level;
const prefix = issue.level === 'ERROR' ? '✗' : issue.level === 'WARNING' ? '⚠' : 'ℹ';
console.error(`${prefix} [${label}] ${issue.path}: ${issue.message}`);
}
}
}
private async runBulkValidation(scope: { changes: boolean; specs: boolean }, opts: { strict: boolean; json: boolean; concurrency?: string }): Promise<void> {
const spinner = !opts.json ? ora('Validating...').start() : undefined;
const [changeIds, specIds] = await Promise.all([
scope.changes ? getActiveChangeIds() : Promise.resolve<string[]>([]),
scope.specs ? getSpecIds() : Promise.resolve<string[]>([]),
]);
const DEFAULT_CONCURRENCY = 6;
const maxSuggestions = 5; // used by nearestMatches
const concurrency = normalizeConcurrency(opts.concurrency) ?? normalizeConcurrency(process.env.OPENSPEC_CONCURRENCY) ?? DEFAULT_CONCURRENCY;
const validator = new Validator(opts.strict);
const queue: Array<() => Promise<BulkItemResult>> = [];
for (const id of changeIds) {
queue.push(async () => {
const start = Date.now();
const file = path.join(process.cwd(), 'openspec', 'changes', id, 'proposal.md');
const report = await validator.validateChange(file);
const durationMs = Date.now() - start;
return { id, type: 'change' as const, valid: report.valid, issues: report.issues, durationMs };
});
}
for (const id of specIds) {
queue.push(async () => {
const start = Date.now();
const file = path.join(process.cwd(), 'openspec', 'specs', id, 'spec.md');
const report = await validator.validateSpec(file);
const durationMs = Date.now() - start;
return { id, type: 'spec' as const, valid: report.valid, issues: report.issues, durationMs };
});
}
const results: BulkItemResult[] = [];
let index = 0;
let running = 0;
let passed = 0;
let failed = 0;
await new Promise<void>((resolve) => {
const next = () => {
while (running < concurrency && index < queue.length) {
const currentIndex = index++;
const task = queue[currentIndex];
running++;
if (spinner) spinner.text = `Validating (${currentIndex + 1}/${queue.length})...`;
task()
.then(res => {
results.push(res);
if (res.valid) passed++; else failed++;
})
.catch((error: any) => {
const message = error?.message || 'Unknown error';
const res: BulkItemResult = { id: getPlannedId(currentIndex, changeIds, specIds) ?? 'unknown', type: getPlannedType(currentIndex, changeIds, specIds) ?? 'change', valid: false, issues: [{ level: 'ERROR', path: 'file', message }], durationMs: 0 };
results.push(res);
failed++;
})
.finally(() => {
running--;
if (index >= queue.length && running === 0) resolve();
else next();
});
}
};
next();
});
spinner?.stop();
results.sort((a, b) => a.id.localeCompare(b.id));
const summary = {
totals: { items: results.length, passed, failed },
byType: {
...(scope.changes ? { change: summarizeType(results, 'change') } : {}),
...(scope.specs ? { spec: summarizeType(results, 'spec') } : {}),
},
} as const;
if (opts.json) {
const out = { items: results, summary, version: '1.0' };
console.log(JSON.stringify(out, null, 2));
} else {
for (const res of results) {
if (res.valid) console.log(`✓ ${res.type}/${res.id}`);
else console.error(`✗ ${res.type}/${res.id}`);
}
console.log(`Totals: ${summary.totals.passed} passed, ${summary.totals.failed} failed (${summary.totals.items} items)`);
}
process.exitCode = failed > 0 ? 1 : 0;
}
}
function summarizeType(results: BulkItemResult[], type: ItemType) {
const filtered = results.filter(r => r.type === type);
const items = filtered.length;
const passed = filtered.filter(r => r.valid).length;
const failed = items - passed;
return { items, passed, failed };
}
function nearestMatches(input: string, candidates: string[], max: number = 5): string[] {
const scored = candidates.map(c => ({ c, d: levenshtein(input, c) }));
scored.sort((a, b) => a.d - b.d);
return scored.slice(0, max).map(s => s.c);
}
function levenshtein(a: string, b: string): number {
const m = a.length;
const n = b.length;
const dp: number[][] = Array.from({ length: m + 1 }, () => Array(n + 1).fill(0));
for (let i = 0; i <= m; i++) dp[i][0] = i;
for (let j = 0; j <= n; j++) dp[0][j] = j;
for (let i = 1; i <= m; i++) {
for (let j = 1; j <= n; j++) {
const cost = a[i - 1] === b[j - 1] ? 0 : 1;
dp[i][j] = Math.min(
dp[i - 1][j] + 1,
dp[i][j - 1] + 1,
dp[i - 1][j - 1] + cost
);
}
}
return dp[m][n];
}
function normalizeConcurrency(value?: string): number | undefined {
if (!value) return undefined;
const n = parseInt(value, 10);
if (Number.isNaN(n) || n <= 0) return undefined;
return n;
}
function getPlannedId(index: number, changeIds: string[], specIds: string[]): string | undefined {
const totalChanges = changeIds.length;
if (index < totalChanges) return changeIds[index];
const specIndex = index - totalChanges;
return specIds[specIndex];
}
function getPlannedType(index: number, changeIds: string[], specIds: string[]): ItemType | undefined {
const totalChanges = changeIds.length;
if (index < totalChanges) return 'change';
const specIndex = index - totalChanges;
if (specIndex >= 0 && specIndex < specIds.length) return 'spec';
return undefined;
}
+4 -1
View File
@@ -94,7 +94,8 @@ export class ChangeParser extends MarkdownParser {
spec: specName,
operation: 'ADDED' as DeltaOperation,
description: `Add requirement: ${req.text}`,
// Use plural form to satisfy validators that expect an array
// Provide both single and plural forms for compatibility
requirement: req,
requirements: [req],
});
});
@@ -109,6 +110,7 @@ export class ChangeParser extends MarkdownParser {
spec: specName,
operation: 'MODIFIED' as DeltaOperation,
description: `Modify requirement: ${req.text}`,
requirement: req,
requirements: [req],
});
});
@@ -123,6 +125,7 @@ export class ChangeParser extends MarkdownParser {
spec: specName,
operation: 'REMOVED' as DeltaOperation,
description: `Remove requirement: ${req.text}`,
requirement: req,
requirements: [req],
});
});
+7
View File
@@ -0,0 +1,7 @@
export function isInteractive(noInteractiveFlag?: boolean): boolean {
if (noInteractiveFlag) return false;
if (process.env.OPEN_SPEC_INTERACTIVE === '0') return false;
return !!process.stdin.isTTY;
}
+38
View File
@@ -0,0 +1,38 @@
import { promises as fs } from 'fs';
import path from 'path';
export async function getActiveChangeIds(root: string = process.cwd()): Promise<string[]> {
const changesPath = path.join(root, 'openspec', 'changes');
try {
const entries = await fs.readdir(changesPath, { withFileTypes: true });
return entries
.filter(entry => entry.isDirectory() && !entry.name.startsWith('.') && entry.name !== 'archive')
.map(entry => entry.name)
.sort();
} catch {
return [];
}
}
export async function getSpecIds(root: string = process.cwd()): Promise<string[]> {
const specsPath = path.join(root, 'openspec', 'specs');
const result: string[] = [];
try {
const entries = await fs.readdir(specsPath, { withFileTypes: true });
for (const entry of entries) {
if (!entry.isDirectory() || entry.name.startsWith('.')) continue;
const specFile = path.join(specsPath, entry.name, 'spec.md');
try {
await fs.access(specFile);
result.push(entry.name);
} catch {
// ignore
}
}
} catch {
// ignore
}
return result.sort();
}
@@ -0,0 +1,51 @@
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { execSync } from 'child_process';
// Note: We cannot truly simulate TTY prompts in this test runner easily.
// Instead, we verify non-interactive fallback behavior and basic invocation.
describe('change validate (interactive behavior)', () => {
const projectRoot = process.cwd();
const testDir = path.join(projectRoot, 'test-change-validate-tmp');
const changesDir = path.join(testDir, 'openspec', 'changes');
const bin = path.join(projectRoot, 'bin', 'openspec.js');
beforeAll(() => {
execSync('pnpm -s build', { stdio: 'pipe' });
});
beforeEach(async () => {
await fs.mkdir(changesDir, { recursive: true });
const content = `# Change: Demo\n\n## Why\nBecause reasons that are sufficiently long.\n\n## What Changes\n- **spec-x:** Add something`;
await fs.mkdir(path.join(changesDir, 'demo'), { recursive: true });
await fs.writeFile(path.join(changesDir, 'demo', 'proposal.md'), content, 'utf-8');
});
afterEach(async () => {
await fs.rm(testDir, { recursive: true, force: true });
});
it('prints list hint and exits non-zero when no arg and non-interactive', () => {
const originalCwd = process.cwd();
const originalEnv = { ...process.env };
try {
process.chdir(testDir);
process.env.OPEN_SPEC_INTERACTIVE = '0';
let err: any;
try {
execSync(`node ${bin} change validate`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.status).not.toBe(0);
expect(err.stderr.toString()).toContain('Available IDs:');
expect(err.stderr.toString()).toContain('openspec change list');
} finally {
process.chdir(originalCwd);
process.env = originalEnv;
}
});
});
@@ -0,0 +1,47 @@
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { execSync } from 'child_process';
describe('spec validate (interactive behavior)', () => {
const projectRoot = process.cwd();
const testDir = path.join(projectRoot, 'test-spec-validate-tmp');
const specsDir = path.join(testDir, 'openspec', 'specs');
const bin = path.join(projectRoot, 'bin', 'openspec.js');
beforeAll(() => {
execSync('pnpm -s build', { stdio: 'pipe' });
});
beforeEach(async () => {
await fs.mkdir(specsDir, { recursive: true });
const content = `## Purpose\nValid spec for interactive test.\n\n## Requirements\n\n### Requirement: X\nText`;
await fs.mkdir(path.join(specsDir, 's1'), { recursive: true });
await fs.writeFile(path.join(specsDir, 's1', 'spec.md'), content, 'utf-8');
});
afterEach(async () => {
await fs.rm(testDir, { recursive: true, force: true });
});
it('errors when no arg and non-interactive', () => {
const originalCwd = process.cwd();
const originalEnv = { ...process.env };
try {
process.chdir(testDir);
process.env.OPEN_SPEC_INTERACTIVE = '0';
let err: any;
try {
execSync(`node ${bin} spec validate`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.status).not.toBe(0);
expect(err.stderr.toString()).toContain('Missing required argument <spec-id>');
} finally {
process.chdir(originalCwd);
process.env = originalEnv;
}
});
});
+125
View File
@@ -0,0 +1,125 @@
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import { execSync } from 'child_process';
describe('top-level validate command', () => {
const projectRoot = process.cwd();
const testDir = path.join(projectRoot, 'test-validate-command-tmp');
const changesDir = path.join(testDir, 'openspec', 'changes');
const specsDir = path.join(testDir, 'openspec', 'specs');
const bin = path.join(projectRoot, 'bin', 'openspec.js');
beforeAll(() => {
execSync('pnpm -s build', { stdio: 'pipe' });
});
beforeEach(async () => {
await fs.mkdir(changesDir, { recursive: true });
await fs.mkdir(specsDir, { recursive: true });
// Create a valid spec
const specContent = `## Purpose
Valid spec for testing.
## Requirements
### Requirement: Foo
Text
#### Scenario: Bar
Given A\nWhen B\nThen C`;
await fs.mkdir(path.join(specsDir, 'alpha'), { recursive: true });
await fs.writeFile(path.join(specsDir, 'alpha', 'spec.md'), specContent, 'utf-8');
// Create a simple change with bullets (parser supports this)
const changeContent = `# Test Change\n\n## Why\nBecause reasons that are sufficiently long for validation.\n\n## What Changes\n- **alpha:** Add something`;
await fs.mkdir(path.join(changesDir, 'c1'), { recursive: true });
await fs.writeFile(path.join(changesDir, 'c1', 'proposal.md'), changeContent, 'utf-8');
// Duplicate name for ambiguity test
await fs.mkdir(path.join(changesDir, 'dup'), { recursive: true });
await fs.writeFile(path.join(changesDir, 'dup', 'proposal.md'), changeContent, 'utf-8');
await fs.mkdir(path.join(specsDir, 'dup'), { recursive: true });
await fs.writeFile(path.join(specsDir, 'dup', 'spec.md'), specContent, 'utf-8');
});
afterEach(async () => {
await fs.rm(testDir, { recursive: true, force: true });
});
it('prints a helpful hint when no args in non-interactive mode', () => {
const originalCwd = process.cwd();
const originalEnv = { ...process.env };
try {
process.chdir(testDir);
process.env.OPEN_SPEC_INTERACTIVE = '0';
let err: any;
try {
execSync(`node ${bin} validate`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.status).not.toBe(0);
expect(err.stderr.toString()).toContain('Nothing to validate. Try one of:');
} finally {
process.chdir(originalCwd);
process.env = originalEnv;
}
});
it('validates all with --all and outputs JSON summary', () => {
const originalCwd = process.cwd();
try {
process.chdir(testDir);
let outStr = '';
try {
outStr = execSync(`node ${bin} validate --all --json`, { encoding: 'utf-8' });
} catch (e: any) {
// If exit code is non-zero (e.g., on failures), still parse stdout JSON
outStr = e.stdout?.toString?.() ?? '';
}
const json = JSON.parse(outStr);
expect(Array.isArray(json.items)).toBe(true);
expect(json.summary?.totals?.items).toBeDefined();
expect(json.version).toBe('1.0');
} finally {
process.chdir(originalCwd);
}
});
it('validates only specs with --specs and respects --concurrency', () => {
const originalCwd = process.cwd();
try {
process.chdir(testDir);
let outStr = '';
try {
outStr = execSync(`node ${bin} validate --specs --json --concurrency 1`, { encoding: 'utf-8' });
} catch (e: any) {
outStr = e.stdout?.toString?.() ?? '';
}
const json = JSON.parse(outStr);
// All items should be specs
expect(json.items.every((i: any) => i.type === 'spec')).toBe(true);
} finally {
process.chdir(originalCwd);
}
});
it('errors on ambiguous item names and suggests type override', () => {
const originalCwd = process.cwd();
try {
process.chdir(testDir);
let err: any;
try {
execSync(`node ${bin} validate dup`, { encoding: 'utf-8' });
} catch (e) { err = e; }
expect(err).toBeDefined();
expect(err.stderr.toString()).toContain('Ambiguous item');
expect(err.status).not.toBe(0);
} finally {
process.chdir(originalCwd);
}
});
});