mirror of
https://github.com/Fission-AI/OpenSpec.git
synced 2026-10-03 22:13:19 +08:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8ac50289f0 | ||
|
|
1f295cec52 | ||
|
|
ad8e213cf9 | ||
|
|
a6c1a90165 | ||
|
|
21b5a3e680 | ||
|
|
1bda5be96c | ||
|
|
0faf44807e | ||
|
|
f9c1d07edb | ||
|
|
2bd1a4417c |
@@ -0,0 +1,343 @@
|
||||
# Comprehensive Retrospective: Creating an OpenSpec Change Proposal
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document consolidates learnings from creating the `bulk-validation-interactive-selection` change proposal for OpenSpec. The process revealed critical gaps in documentation, unhelpful error messages, and areas where the system could be more user-friendly. While OpenSpec's core functionality works correctly, the user experience for creating changes needs significant improvement.
|
||||
|
||||
## Table of Contents
|
||||
1. [Errors Encountered](#errors-encountered)
|
||||
2. [System Issues vs User Errors](#system-issues-vs-user-errors)
|
||||
3. [Documentation Gaps Analysis](#documentation-gaps-analysis)
|
||||
4. [Key Learnings](#key-learnings)
|
||||
5. [Recommendations](#recommendations)
|
||||
6. [Conclusion](#conclusion)
|
||||
|
||||
---
|
||||
|
||||
## Errors Encountered
|
||||
|
||||
### Error 1: Misunderstanding Delta Structure
|
||||
**Error Message:** `✗ [ERROR] deltas: Change must have at least one delta`
|
||||
|
||||
**What Happened:**
|
||||
Initially attempted to define deltas directly in the `proposal.md` file using markdown sections like:
|
||||
```markdown
|
||||
### Delta: Add validate-all command
|
||||
**Type**: Feature addition
|
||||
**Effort**: Small (< 100 lines)
|
||||
```
|
||||
|
||||
**Root Cause:**
|
||||
Fundamental misunderstanding of how OpenSpec processes deltas. Deltas are derived from spec files in the change's `specs/` directory, not from the proposal itself.
|
||||
|
||||
**Discovery Process:**
|
||||
- Examined `ChangeParser` class in `/src/core/parsers/change-parser.ts`
|
||||
- Found that `parseDeltaSpecs()` method looks for spec files in `specs/` subdirectory
|
||||
- Learned that deltas are extracted by comparing spec files against existing specs
|
||||
|
||||
### Error 2: Missing Operation Prefix in Section Headers
|
||||
**Error Message:** `✗ [ERROR] deltas: Change must have at least one delta`
|
||||
|
||||
**What Happened:**
|
||||
Created new spec files with standard `## Requirements` headers instead of operation-prefixed headers.
|
||||
|
||||
**Root Cause:**
|
||||
Failed to understand that ALL spec files in a change need operation prefixes (`ADDED`, `MODIFIED`, etc.) in their section headers, regardless of whether they're new specs or modifications.
|
||||
|
||||
**Discovery Process:**
|
||||
- Created new specs with `## Requirements` → No deltas detected
|
||||
- Changed to `## ADDED Requirements` → Deltas detected successfully!
|
||||
- Realized creating new specs works perfectly fine once properly formatted
|
||||
|
||||
**Important Clarification:**
|
||||
OpenSpec fully supports creating new specs. They appear as ADDED operations in the deltas. My initial analysis incorrectly suggested this was a limitation, but it was actually just a formatting issue.
|
||||
|
||||
### Error 3: Improper Scenario Formatting
|
||||
**Error Message:**
|
||||
```
|
||||
✗ [ERROR] deltas.0.requirements.0.scenarios: Requirement must have at least one scenario
|
||||
✗ [ERROR] deltas.1.requirements.0.scenarios: Requirement must have at least one scenario
|
||||
```
|
||||
|
||||
**What Happened:**
|
||||
Formatted scenarios as bullet lists under a bold "Scenarios:" label:
|
||||
```markdown
|
||||
**Scenarios:**
|
||||
- **WHEN** executing command
|
||||
- **THEN** expected behavior
|
||||
```
|
||||
|
||||
**Root Cause:**
|
||||
OpenSpec's parser expects scenarios to be defined as level 4 headers (`####`) with specific formatting:
|
||||
```markdown
|
||||
#### Scenario: Descriptive scenario name
|
||||
|
||||
- **WHEN** executing command
|
||||
- **THEN** expected behavior
|
||||
```
|
||||
|
||||
**Discovery Process:**
|
||||
- Checked parsed JSON output: `npx openspec change show bulk-validation-interactive-selection --json`
|
||||
- Saw `"scenarios": []` empty array despite having scenario content
|
||||
- Examined working spec files and found the `#### Scenario:` header pattern
|
||||
|
||||
### Error 4: File Path Confusion
|
||||
**Initial Confusion:**
|
||||
Wasn't clear whether to create specs that would become part of the main `openspec/specs/` or just define them in the change.
|
||||
|
||||
**Resolution:**
|
||||
Learned that changes can:
|
||||
1. Create new specs (they start in `changes/{change-name}/specs/` and move to `openspec/specs/` when archived)
|
||||
2. Modify existing specs (by creating a spec file with the same name as one in `openspec/specs/`)
|
||||
3. The validation system detects both patterns and creates appropriate deltas
|
||||
|
||||
---
|
||||
|
||||
## System Issues vs User Errors
|
||||
|
||||
### System Issues / Bugs
|
||||
|
||||
#### 1. Unhelpful Error Messages ⚠️
|
||||
**Issue:** `✗ [ERROR] deltas: Change must have at least one delta`
|
||||
|
||||
**Why This Is a System Problem:**
|
||||
- Error message provides no guidance on HOW to create deltas
|
||||
- Doesn't mention that deltas come from `specs/` subdirectory
|
||||
- Doesn't explain the required section headers
|
||||
- A better error would be: "No deltas found. Ensure your change has a specs/ directory with .md files containing sections like '## ADDED Requirements'"
|
||||
|
||||
#### 2. Silent Scenario Parsing Failures ⚠️
|
||||
**Issue:** When scenarios were formatted incorrectly, they were silently ignored
|
||||
|
||||
**Why This Is a System Problem:**
|
||||
- Parser silently returns empty scenarios array instead of warning
|
||||
- No validation error explaining the format issue
|
||||
- User gets "Requirement must have at least one scenario" without knowing their scenarios exist but aren't parsed
|
||||
|
||||
**Evidence:**
|
||||
```json
|
||||
{
|
||||
"text": "The CLI SHALL provide a top-level `show` command with interactive selection.",
|
||||
"scenarios": [] // Silent failure - scenarios existed but weren't parsed
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. No Validation for Proposal Structure During Creation
|
||||
**Issue:** System allows creating invalid proposals without early feedback
|
||||
|
||||
**Why This Is a System Problem:**
|
||||
- No scaffolding or template commands
|
||||
- No incremental validation as you build
|
||||
- Must fully create the change before discovering structural issues
|
||||
|
||||
### User Errors (My Mistakes)
|
||||
|
||||
#### 1. Trying to Define Deltas in Proposal.md
|
||||
- Incorrectly assumed deltas could be inline in the proposal
|
||||
- System correctly expects deltas in separate spec files
|
||||
|
||||
#### 2. Using Wrong Section Headers
|
||||
- Used `## Requirements` instead of `## ADDED Requirements`
|
||||
- Convention is documented in existing changes, but I didn't examine carefully
|
||||
|
||||
#### 3. Wrong Scenario Format
|
||||
- Used bullet lists instead of `#### Scenario:` headers
|
||||
- Made assumptions instead of checking existing patterns
|
||||
|
||||
### Gray Areas
|
||||
|
||||
1. **Documentation gaps** - While examples exist, there's no comprehensive "How to Create a Change" guide
|
||||
2. **Lack of tooling** - No scaffolding commands to create properly structured changes
|
||||
|
||||
---
|
||||
|
||||
## Documentation Gaps Analysis
|
||||
|
||||
### Critical Gaps in openspec/README.md
|
||||
|
||||
#### 1. Scenario Format - COMPLETELY MISSING ⚠️
|
||||
**What README Shows:** No scenario examples at all
|
||||
|
||||
**What's Actually Required:**
|
||||
```markdown
|
||||
#### Scenario: Descriptive name
|
||||
- **WHEN** condition
|
||||
- **THEN** expected outcome
|
||||
- **AND** additional outcomes
|
||||
```
|
||||
|
||||
**Impact:** This was the biggest struggle. The README mentions requirements but never shows how to write scenarios. Without this, requirements fail validation.
|
||||
|
||||
#### 2. Complete Spec File Example - MISSING
|
||||
**What README Shows:** Only fragments
|
||||
|
||||
**What's Actually Needed:** A complete working example showing:
|
||||
- Full spec file structure
|
||||
- Proper requirement format
|
||||
- Scenario formatting
|
||||
- All required elements
|
||||
|
||||
#### 3. Validation Commands - NOT MENTIONED
|
||||
**Missing from README:**
|
||||
- `npx openspec change validate <change-name>`
|
||||
- `npx openspec change show <change-name> --json`
|
||||
- The `--strict` flag for catching warnings
|
||||
|
||||
#### 4. Delta Detection Explanation - INCOMPLETE
|
||||
**What's Missing:**
|
||||
- WHERE the system looks for specs (specs/ subdirectory)
|
||||
- THAT deltas are automatically extracted
|
||||
- HOW to debug when deltas aren't detected
|
||||
- WHAT error messages mean
|
||||
|
||||
### Misleading Documentation
|
||||
|
||||
#### "Store only the changes" - MISLEADING
|
||||
**Line 145:** `# - Store only the changes (not complete future state)`
|
||||
|
||||
**Problem:** This suggests storing diffs or partial content. In reality, you need:
|
||||
- Complete requirements in their final form
|
||||
- Full scenario definitions
|
||||
- The entire requirement text
|
||||
|
||||
### Documentation That Was Helpful
|
||||
1. Delta section headers (`## ADDED Requirements`) - clearly documented
|
||||
2. Directory structure - excellent visualization
|
||||
3. When to create proposals - well defined
|
||||
|
||||
---
|
||||
|
||||
## Key Learnings
|
||||
|
||||
### 1. OpenSpec's Delta Detection Algorithm
|
||||
The system follows this process:
|
||||
1. Scans `openspec/changes/{change-name}/specs/` directory
|
||||
2. For each spec file found, parses for delta sections (`ADDED`, `MODIFIED`, `REMOVED`, `RENAMED`)
|
||||
3. Creates delta objects with operation type, affected spec, and requirements
|
||||
4. Validates that at least one delta exists for the change to be valid
|
||||
|
||||
**Important:** Creating entirely new specs is fully supported! New specs use `## ADDED Requirements` and appear as ADDED operations in the deltas.
|
||||
|
||||
### 2. Spec File Structure Requirements
|
||||
Valid spec files must follow this structure:
|
||||
```markdown
|
||||
# Spec Title
|
||||
|
||||
## [ADDED|MODIFIED|REMOVED|RENAMED] Requirements
|
||||
|
||||
### Requirement: Clear requirement statement
|
||||
|
||||
The requirement description using SHALL/SHOULD/MAY.
|
||||
|
||||
#### Scenario: Scenario name
|
||||
|
||||
- **WHEN** condition
|
||||
- **THEN** expected outcome
|
||||
- **AND** additional outcomes
|
||||
```
|
||||
|
||||
### 3. Change Proposal Structure
|
||||
A valid change must have:
|
||||
- `## Why` section - explaining the motivation
|
||||
- `## What Changes` section - summarizing the changes
|
||||
- `specs/` directory with properly formatted spec files containing deltas
|
||||
- Each delta must have at least one requirement with at least one scenario
|
||||
|
||||
### 4. Validation Commands Are Essential
|
||||
```bash
|
||||
# Basic validation
|
||||
npx openspec change validate {change-name}
|
||||
|
||||
# Strict validation (recommended)
|
||||
npx openspec change validate {change-name} --strict
|
||||
|
||||
# Debug delta detection
|
||||
npx openspec change show {change-name} --json | jq '.deltas'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### High Priority (System Bugs to Fix)
|
||||
|
||||
1. **Improve Error Messages**
|
||||
- Add actionable guidance to error messages
|
||||
- Example: "No deltas found. Check: 1) specs/ directory exists, 2) Files use ## ADDED Requirements headers, 3) Each requirement has #### Scenario: sections"
|
||||
|
||||
2. **Add Warnings for Malformed Content**
|
||||
- Warn when scenarios exist but aren't properly formatted
|
||||
- Show which line/file has the issue
|
||||
|
||||
3. **Add Delta Detection Debugging**
|
||||
- Command like `openspec change debug-deltas {change-name}`
|
||||
- Show which files were scanned, what was found, what was rejected
|
||||
|
||||
### Medium Priority (Documentation Improvements)
|
||||
|
||||
1. **Add Complete Working Example to README**
|
||||
- Full change proposal with all files
|
||||
- Properly formatted specs with scenarios
|
||||
- Show the validation output
|
||||
|
||||
2. **Add Troubleshooting Section**
|
||||
- Common errors and their solutions
|
||||
- How to debug delta detection
|
||||
- Scenario formatting requirements
|
||||
|
||||
3. **Add Validation Best Practices**
|
||||
- When to use `--strict`
|
||||
- How to use JSON output for debugging
|
||||
- Common validation patterns
|
||||
|
||||
### Low Priority (Developer Experience)
|
||||
|
||||
1. **Add Scaffolding Command**
|
||||
```bash
|
||||
openspec change scaffold {change-name}
|
||||
```
|
||||
- Creates proper directory structure
|
||||
- Includes template files with correct formatting
|
||||
- Adds example scenarios
|
||||
|
||||
2. **Add Interactive Creation Wizard**
|
||||
- Guide users through change creation
|
||||
- Validate as they go
|
||||
- Suggest fixes for common issues
|
||||
|
||||
3. **Add Auto-fix Capability**
|
||||
- `--fix` flag to correct common formatting issues
|
||||
- Convert bullet list scenarios to proper headers
|
||||
- Add missing operation prefixes
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The OpenSpec system works correctly for its intended design, but the user experience for creating changes needs significant improvement. The core issues stem from:
|
||||
|
||||
### System Issues
|
||||
- **Unhelpful error messages** that don't guide users to solutions
|
||||
- **Silent parsing failures** that provide no feedback about malformed content
|
||||
- **Lack of debugging tools** to understand what went wrong
|
||||
|
||||
### Documentation Issues
|
||||
- **Critical formatting requirements missing** (especially scenario format)
|
||||
- **No complete working examples** showing all required elements
|
||||
- **Validation commands not documented** despite being essential
|
||||
|
||||
### User Issues
|
||||
- **Incorrect assumptions** about how the system works
|
||||
- **Not examining existing patterns** carefully enough
|
||||
- **Trying to shortcut** instead of following established conventions
|
||||
|
||||
### The Path Forward
|
||||
|
||||
With better error messages, complete documentation, and basic tooling support, most of the errors encountered could be prevented. The system's delta-centric approach is powerful and ensures changes are atomic and trackable, but it needs to be more discoverable and user-friendly.
|
||||
|
||||
The most impactful improvements would be:
|
||||
1. Adding scenario format documentation to the README
|
||||
2. Improving error messages with actionable guidance
|
||||
3. Creating a scaffolding command for new changes
|
||||
|
||||
These changes would transform OpenSpec from a system that works correctly but is hard to use, into one that actively helps developers succeed.
|
||||
@@ -0,0 +1,20 @@
|
||||
## Why
|
||||
|
||||
Users frequently need to view changes and specs but must know in advance whether they're looking at a change or spec. The current subcommand structure (`change show`, `spec show`) creates friction when:
|
||||
- Users want to quickly view an item without remembering its type
|
||||
- Exploring the codebase requires switching between different show commands
|
||||
- Show commands without arguments return errors instead of helpful guidance
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add new top-level `show` command for displaying changes or specs with intelligent selection
|
||||
- Support direct item display: `openspec show <item>` with automatic type detection
|
||||
- Interactive selection when no arguments provided
|
||||
- Enhance existing `change show` and `spec show` to support interactive selection (backwards compatibility)
|
||||
- Maintain all existing format options (--json, --deltas-only, --requirements, etc.)
|
||||
|
||||
## Impact
|
||||
|
||||
- New specs to create: cli-show
|
||||
- Specs to enhance: cli-change, cli-spec (for backwards compatibility)
|
||||
- Affected code: src/cli/index.ts, src/commands/show.ts (new), src/commands/spec.ts, src/commands/change.ts
|
||||
@@ -0,0 +1,23 @@
|
||||
# CLI Change Command Spec
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Interactive show selection
|
||||
|
||||
The change show command SHALL support interactive selection when no change name is provided.
|
||||
|
||||
#### Scenario: Interactive change selection for show
|
||||
|
||||
- **WHEN** executing `openspec change show` without arguments
|
||||
- **THEN** display an interactive list of available changes
|
||||
- **AND** allow the user to select a change to show
|
||||
- **AND** display the selected change content
|
||||
- **AND** maintain all existing show options (--json, --deltas-only)
|
||||
|
||||
#### Scenario: Non-interactive fallback keeps current behavior
|
||||
|
||||
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
|
||||
- **WHEN** executing `openspec change show` without a change name
|
||||
- **THEN** do not prompt interactively
|
||||
- **AND** print the existing hint including available change IDs
|
||||
- **AND** set `process.exitCode = 1`
|
||||
@@ -0,0 +1,83 @@
|
||||
# CLI Show Command Spec
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Top-level show command
|
||||
|
||||
The CLI SHALL provide a top-level `show` command for displaying changes and specs with intelligent selection.
|
||||
|
||||
#### Scenario: Interactive show selection
|
||||
|
||||
- **WHEN** executing `openspec show` without arguments
|
||||
- **THEN** prompt user to select type (change or spec)
|
||||
- **AND** display list of available items for selected type
|
||||
- **AND** show the selected item's content
|
||||
|
||||
#### Scenario: Non-interactive environments do not prompt
|
||||
|
||||
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
|
||||
- **WHEN** executing `openspec show` without arguments
|
||||
- **THEN** do not prompt
|
||||
- **AND** print a helpful hint with examples for `openspec show <item>` or `openspec change/spec show`
|
||||
- **AND** exit with code 1
|
||||
|
||||
#### Scenario: Direct item display
|
||||
|
||||
- **WHEN** executing `openspec show <item-name>`
|
||||
- **THEN** automatically detect if item is a change or spec
|
||||
- **AND** display the item's content
|
||||
- **AND** use appropriate formatting based on item type
|
||||
|
||||
#### Scenario: Type detection and ambiguity handling
|
||||
|
||||
- **WHEN** executing `openspec show <item-name>`
|
||||
- **THEN** if `<item-name>` uniquely matches a change or a spec, show that item
|
||||
- **AND** if it matches both, print an ambiguity error and suggest `--type change|spec` or using `openspec change show`/`openspec spec show`
|
||||
- **AND** if it matches neither, print not-found with nearest-match suggestions
|
||||
|
||||
#### Scenario: Explicit type override
|
||||
|
||||
- **WHEN** executing `openspec show --type change <item>`
|
||||
- **THEN** treat `<item>` as a change ID and show it (skipping auto-detection)
|
||||
|
||||
- **WHEN** executing `openspec show --type spec <item>`
|
||||
- **THEN** treat `<item>` as a spec ID and show it (skipping auto-detection)
|
||||
|
||||
### Requirement: Output format options
|
||||
|
||||
The show command SHALL support various output formats consistent with existing commands.
|
||||
|
||||
#### Scenario: JSON output
|
||||
|
||||
- **WHEN** executing `openspec show <item> --json`
|
||||
- **THEN** output the item in JSON format
|
||||
- **AND** include parsed metadata and structure
|
||||
- **AND** maintain format consistency with existing change/spec show commands
|
||||
|
||||
#### Scenario: Flag scoping and delegation
|
||||
|
||||
- **WHEN** showing a change or a spec via the top-level command
|
||||
- **THEN** accept common flags such as `--json`
|
||||
- **AND** pass through type-specific flags to the corresponding implementation
|
||||
- Change-only flags: `--deltas-only` (alias `--requirements-only` deprecated)
|
||||
- Spec-only flags: `--requirements`, `--no-scenarios`, `-r/--requirement`
|
||||
- **AND** ignore irrelevant flags for the detected type with a warning
|
||||
|
||||
### Requirement: Interactivity controls
|
||||
|
||||
- The CLI SHALL respect `--no-interactive` to disable prompts.
|
||||
- The CLI SHALL respect `OPEN_SPEC_INTERACTIVE=0` to disable prompts globally.
|
||||
- Interactive prompts SHALL only be shown when stdin is a TTY and interactivity is not disabled.
|
||||
|
||||
#### Scenario: Change-specific options
|
||||
|
||||
- **WHEN** showing a change with `openspec show <change-name> --deltas-only`
|
||||
- **THEN** display only the deltas in JSON format
|
||||
- **AND** maintain compatibility with existing change show options
|
||||
|
||||
#### Scenario: Spec-specific options
|
||||
|
||||
- **WHEN** showing a spec with `openspec show <spec-id> --requirements`
|
||||
- **THEN** display only requirements in JSON format
|
||||
- **AND** support other spec options (--no-scenarios, -r)
|
||||
- **AND** maintain compatibility with existing spec show options
|
||||
@@ -0,0 +1,23 @@
|
||||
# CLI Spec Command Spec
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Interactive spec show
|
||||
|
||||
The spec show command SHALL support interactive selection when no spec-id is provided.
|
||||
|
||||
#### Scenario: Interactive spec selection for show
|
||||
|
||||
- **WHEN** executing `openspec spec show` without arguments
|
||||
- **THEN** display an interactive list of available specs
|
||||
- **AND** allow the user to select a spec to show
|
||||
- **AND** display the selected spec content
|
||||
- **AND** maintain all existing show options (--json, --requirements, --no-scenarios, -r)
|
||||
|
||||
#### Scenario: Non-interactive fallback keeps current behavior
|
||||
|
||||
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
|
||||
- **WHEN** executing `openspec spec show` without a spec-id
|
||||
- **THEN** do not prompt interactively
|
||||
- **AND** print the existing error message for missing spec-id
|
||||
- **AND** set non-zero exit code
|
||||
@@ -0,0 +1,20 @@
|
||||
## Why
|
||||
|
||||
Currently, users must validate changes and specs individually by specifying each ID. This creates friction when:
|
||||
- Teams want to validate all changes/specs before a release
|
||||
- Developers need to ensure consistency across multiple related changes
|
||||
- Users run validation commands without arguments and receive errors instead of helpful guidance
|
||||
- The subcommand structure requires users to know in advance whether they're validating a change or spec
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add new top-level `validate` command with intuitive flags (--all, --changes, --specs)
|
||||
- Enhance existing `change validate` and `spec validate` to support interactive selection (backwards compatibility)
|
||||
- Interactive selection by default when no arguments provided
|
||||
- Support direct item validation: `openspec validate <item>` with automatic type detection
|
||||
|
||||
## Impact
|
||||
|
||||
- New specs to create: cli-validate
|
||||
- Specs to enhance: cli-change, cli-spec (for backwards compatibility)
|
||||
- Affected code: src/cli/index.ts, src/commands/validate.ts (new), src/commands/spec.ts, src/commands/change.ts
|
||||
@@ -0,0 +1,22 @@
|
||||
# CLI Change Command Spec
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Interactive validation selection
|
||||
|
||||
The change validate command SHALL support interactive selection when no change name is provided.
|
||||
|
||||
#### Scenario: Interactive change selection for validation
|
||||
|
||||
- **WHEN** executing `openspec change validate` without arguments
|
||||
- **THEN** display an interactive list of available changes
|
||||
- **AND** allow the user to select a change to validate
|
||||
- **AND** validate the selected change
|
||||
|
||||
#### Scenario: Non-interactive fallback keeps current behavior
|
||||
|
||||
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
|
||||
- **WHEN** executing `openspec change validate` without a change name
|
||||
- **THEN** do not prompt interactively
|
||||
- **AND** print the existing hint including available change IDs
|
||||
- **AND** set `process.exitCode = 1`
|
||||
@@ -0,0 +1,23 @@
|
||||
# CLI Spec Command Spec
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Interactive spec validation
|
||||
|
||||
The spec validate command SHALL support interactive selection when no spec-id is provided.
|
||||
|
||||
#### Scenario: Interactive spec selection for validation
|
||||
|
||||
- **WHEN** executing `openspec spec validate` without arguments
|
||||
- **THEN** display an interactive list of available specs
|
||||
- **AND** allow the user to select a spec to validate
|
||||
- **AND** validate the selected spec
|
||||
- **AND** maintain all existing validation options (--strict, --json)
|
||||
|
||||
#### Scenario: Non-interactive fallback keeps current behavior
|
||||
|
||||
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
|
||||
- **WHEN** executing `openspec spec validate` without a spec-id
|
||||
- **THEN** do not prompt interactively
|
||||
- **AND** print the existing error message for missing spec-id
|
||||
- **AND** set non-zero exit code
|
||||
@@ -0,0 +1,141 @@
|
||||
# CLI Validate Command Spec
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Top-level validate command
|
||||
|
||||
The CLI SHALL provide a top-level `validate` command for validating changes and specs with flexible selection options.
|
||||
|
||||
#### Scenario: Interactive validation selection
|
||||
|
||||
- **WHEN** executing `openspec validate` without arguments
|
||||
- **THEN** prompt user to select what to validate (all, changes, specs, or specific item)
|
||||
- **AND** perform validation based on selection
|
||||
- **AND** display results with appropriate formatting
|
||||
|
||||
#### Scenario: Non-interactive environments do not prompt
|
||||
|
||||
- **GIVEN** stdin is not a TTY or `--no-interactive` is provided or environment variable `OPEN_SPEC_INTERACTIVE=0`
|
||||
- **WHEN** executing `openspec validate` without arguments
|
||||
- **THEN** do not prompt interactively
|
||||
- **AND** print a helpful hint listing available commands/flags and exit with code 1
|
||||
|
||||
#### Scenario: Direct item validation
|
||||
|
||||
- **WHEN** executing `openspec validate <item-name>`
|
||||
- **THEN** automatically detect if item is a change or spec
|
||||
- **AND** validate the specified item
|
||||
- **AND** display validation results
|
||||
|
||||
### Requirement: Bulk and filtered validation
|
||||
|
||||
The validate command SHALL support flags for bulk validation (--all) and filtered validation by type (--changes, --specs).
|
||||
|
||||
#### Scenario: Validate everything
|
||||
|
||||
- **WHEN** executing `openspec validate --all`
|
||||
- **THEN** validate all changes in openspec/changes/ (excluding archive)
|
||||
- **AND** validate all specs in openspec/specs/
|
||||
- **AND** display a summary showing passed/failed items
|
||||
- **AND** exit with code 1 if any validation fails
|
||||
|
||||
#### Scenario: Scope of bulk validation
|
||||
|
||||
- **WHEN** validating with `--all` or `--changes`
|
||||
- **THEN** include all change proposals under `openspec/changes/`
|
||||
- **AND** exclude the `openspec/changes/archive/` directory
|
||||
|
||||
- **WHEN** validating with `--specs`
|
||||
- **THEN** include all specs that have a `spec.md` under `openspec/specs/<id>/spec.md`
|
||||
|
||||
#### Scenario: Validate all changes
|
||||
|
||||
- **WHEN** executing `openspec validate --changes`
|
||||
- **THEN** validate all changes in openspec/changes/ (excluding archive)
|
||||
- **AND** display results for each change
|
||||
- **AND** show summary statistics
|
||||
|
||||
#### Scenario: Validate all specs
|
||||
|
||||
- **WHEN** executing `openspec validate --specs`
|
||||
- **THEN** validate all specs in openspec/specs/
|
||||
- **AND** display results for each spec
|
||||
- **AND** show summary statistics
|
||||
|
||||
### Requirement: Validation options and progress indication
|
||||
|
||||
The validate command SHALL support standard validation options (--strict, --json) and display progress during bulk operations.
|
||||
|
||||
#### Scenario: Strict validation
|
||||
|
||||
- **WHEN** executing `openspec validate --all --strict`
|
||||
- **THEN** apply strict validation to all items
|
||||
- **AND** treat warnings as errors
|
||||
- **AND** fail if any item has warnings or errors
|
||||
|
||||
#### Scenario: JSON output
|
||||
|
||||
- **WHEN** executing `openspec validate --all --json`
|
||||
- **THEN** output validation results as JSON
|
||||
- **AND** include detailed issues for each item
|
||||
- **AND** include summary statistics
|
||||
|
||||
#### Scenario: JSON output schema for bulk validation
|
||||
|
||||
- **WHEN** executing `openspec validate --all --json` (or `--changes` / `--specs`)
|
||||
- **THEN** output a JSON object with the following shape:
|
||||
- `items`: Array of objects with fields `{ id: string, type: "change"|"spec", valid: boolean, issues: Issue[], durationMs: number }`
|
||||
- `summary`: Object `{ totals: { items: number, passed: number, failed: number }, byType: { change?: { items: number, passed: number, failed: number }, spec?: { items: number, passed: number, failed: number } } }`
|
||||
- `version`: String identifier for the schema (e.g., `"1.0"`)
|
||||
- **AND** exit with code 1 if any `items[].valid === false`
|
||||
|
||||
Where `Issue` follows the existing per-item validation report shape `{ level: "ERROR"|"WARNING"|"INFO", path: string, message: string }`.
|
||||
|
||||
#### Scenario: Show validation progress
|
||||
|
||||
- **WHEN** validating multiple items (--all, --changes, or --specs)
|
||||
- **THEN** show progress indicator or status updates
|
||||
- **AND** indicate which item is currently being validated
|
||||
- **AND** display running count of passed/failed items
|
||||
|
||||
#### Scenario: Concurrency limits for performance
|
||||
|
||||
- **WHEN** validating multiple items
|
||||
- **THEN** run validations with a bounded concurrency (e.g., 4–8 in parallel)
|
||||
- **AND** ensure progress indicators remain responsive
|
||||
|
||||
### Requirement: Item type detection and ambiguity handling
|
||||
|
||||
#### Scenario: Direct item validation with automatic type detection
|
||||
|
||||
- **WHEN** executing `openspec validate <item-name>`
|
||||
- **THEN** if `<item-name>` uniquely matches a change or a spec, validate that item
|
||||
|
||||
#### Scenario: Ambiguity between change and spec names
|
||||
|
||||
- **GIVEN** `<item-name>` exists both as a change and as a spec
|
||||
- **WHEN** executing `openspec validate <item-name>`
|
||||
- **THEN** print an ambiguity error explaining both matches
|
||||
- **AND** suggest passing `--type change` or `--type spec`, or using `openspec change validate` / `openspec spec validate`
|
||||
- **AND** exit with code 1 without performing validation
|
||||
|
||||
#### Scenario: Unknown item name
|
||||
|
||||
- **WHEN** the `<item-name>` matches neither a change nor a spec
|
||||
- **THEN** print a not-found error
|
||||
- **AND** show nearest-match suggestions when available
|
||||
- **AND** exit with code 1
|
||||
|
||||
#### Scenario: Explicit type override
|
||||
|
||||
- **WHEN** executing `openspec validate --type change <item>`
|
||||
- **THEN** treat `<item>` as a change ID and validate it (skipping auto-detection)
|
||||
|
||||
- **WHEN** executing `openspec validate --type spec <item>`
|
||||
- **THEN** treat `<item>` as a spec ID and validate it (skipping auto-detection)
|
||||
|
||||
### Requirement: Interactivity controls
|
||||
|
||||
- The CLI SHALL respect `--no-interactive` to disable prompts.
|
||||
- The CLI SHALL respect `OPEN_SPEC_INTERACTIVE=0` to disable prompts globally.
|
||||
- Interactive prompts SHALL only be shown when stdin is a TTY and interactivity is not disabled.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Implementation Tasks
|
||||
|
||||
## 1. Change Command: Interactive Validation Selection
|
||||
- [x] 1.1 Add `--no-interactive` flag to `change validate` in `src/cli/index.ts`
|
||||
- [x] 1.2 Implement interactivity gate respecting TTY and `OPEN_SPEC_INTERACTIVE=0` in `src/commands/change.ts`
|
||||
- [x] 1.3 When no `[change-name]` is provided and interactivity is allowed, prompt with a list of active changes (exclude `archive/`) and validate the selected one
|
||||
- [x] 1.4 Preserve current non-interactive fallback: print available change IDs and hint, set `process.exitCode = 1`
|
||||
- [x] 1.5 Tests: add coverage for interactive and non-interactive flows
|
||||
- Added `test/commands/change.interactive-validate.test.ts`
|
||||
|
||||
## 2. Spec Command: Interactive Validation Selection
|
||||
- [x] 2.1 Make `spec validate` accept optional `[spec-id]` in `src/commands/spec.ts` registration
|
||||
- [x] 2.2 Add `--no-interactive` flag to `spec validate`
|
||||
- [x] 2.3 Implement interactivity gate respecting TTY and `OPEN_SPEC_INTERACTIVE=0`
|
||||
- [x] 2.4 When no `[spec-id]` provided and interactivity allowed, prompt to select from `openspec/specs/*/spec.md` and validate the selected spec
|
||||
- [x] 2.5 Preserve current non-interactive fallback when no spec-id and no interactivity: print existing error and exit code non-zero
|
||||
- [x] 2.6 Tests: add coverage for interactive and non-interactive flows
|
||||
- Added `test/commands/spec.interactive-validate.test.ts`
|
||||
|
||||
## 3. New Top-level `validate` Command
|
||||
- [x] 3.1 Add `validate` command in `src/cli/index.ts`
|
||||
- Options: `--all`, `--changes`, `--specs`, `--type <change|spec>`, `--strict`, `--json`, `--no-interactive`
|
||||
- Usage: `openspec validate [item-name]`
|
||||
- [x] 3.2 Create `src/commands/validate.ts` implementing:
|
||||
- [x] 3.2.1 Interactive selector when no args (choices: All, Changes, Specs, Specific item)
|
||||
- [x] 3.2.2 Non-interactive fallback with helpful hint and exit code 1
|
||||
- [x] 3.2.3 Direct item validation with automatic type detection
|
||||
- [x] 3.2.4 Ambiguity error when name exists as both change and spec; suggest `--type` or subcommands
|
||||
- [x] 3.2.5 Unknown item handling with nearest-match suggestions
|
||||
- [x] 3.2.6 Bulk validation for `--all`, `--changes`, `--specs` (exclude `openspec/changes/archive/`)
|
||||
- [x] 3.2.7 Respect `--strict` and `--json` options; JSON shape per spec
|
||||
- [x] 3.2.8 Exit with code 1 if any validation fails
|
||||
- [x] 3.2.9 Bounded concurrency (default 4–8) for bulk validation
|
||||
- [x] 3.2.10 Progress indication during bulk runs (current item, running counts)
|
||||
|
||||
## 4. Utilities and Shared Helpers
|
||||
- [x] 4.1 Add `src/utils/interactive.ts` with `isInteractive(stdin: NodeJS.ReadStream, noInteractiveFlag?: boolean): boolean`
|
||||
- Considers: `process.stdin.isTTY`, `--no-interactive`, `OPEN_SPEC_INTERACTIVE=0`
|
||||
- [x] 4.2 Add `src/utils/item-discovery.ts` with:
|
||||
- `getActiveChangeIds(root = process.cwd()): Promise<string[]>` (exclude `archive/`)
|
||||
- `getSpecIds(root = process.cwd()): Promise<string[]>` (folders with `spec.md`)
|
||||
- [ ] 4.3 Optional: `src/utils/concurrency.ts` helper for bounded parallelism
|
||||
- [x] 4.4 Reuse `src/core/validation/validator.ts` for item validation
|
||||
|
||||
## 5. JSON Output (Bulk Validation)
|
||||
- [x] 5.1 Implement JSON schema:
|
||||
- `items: Array<{ id: string, type: "change"|"spec", valid: boolean, issues: Issue[], durationMs: number }>`
|
||||
- `summary: { totals: { items: number, passed: number, failed: number }, byType: { change?: { items: number, passed: number, failed: number }, spec?: { items: number, passed: number, failed: number } } }`
|
||||
- `version: "1.0"`
|
||||
- [x] 5.2 Ensure process exit code is 1 if any `items[].valid === false`
|
||||
- [x] 5.3 Tests for JSON shape (keys, types, counts) and exit code behavior
|
||||
- Added `test/commands/validate.test.ts`
|
||||
|
||||
## 6. Progress and UX
|
||||
- [x] 6.1 Use `ora` or minimal console progress to show current item and running counts
|
||||
- [x] 6.2 Keep output stable in `--json` mode (no extra logs to stdout; use stderr for progress if needed)
|
||||
- [x] 6.3 Ensure responsiveness with concurrency limits
|
||||
|
||||
## 7. Tests
|
||||
- [x] 7.1 Add top-level validate tests: `test/commands/validate.test.ts`
|
||||
- Includes non-interactive hint, --all JSON, --specs with concurrency, ambiguity error
|
||||
- [ ] 7.2 Add unit tests for `isInteractive` and item discovery helpers
|
||||
- [x] 7.3 Extend existing change/spec command tests to cover interactive `validate`
|
||||
- Added `test/commands/change.interactive-validate.test.ts`, `test/commands/spec.interactive-validate.test.ts`
|
||||
|
||||
## 8. CLI Help and Docs
|
||||
- [x] 8.1 Update command descriptions/options in `src/cli/index.ts`
|
||||
- [x] 8.2 Verify help output includes `validate` command and flags
|
||||
- [x] 8.3 Ensure existing specs under `openspec/changes/bulk-validation-interactive-selection/specs/*` remain satisfied
|
||||
|
||||
## 9. Non-functional
|
||||
- [x] 9.1 Code style and types: explicit types for exported APIs; avoid `any`
|
||||
- [x] 9.2 No linter errors; stable formatting; avoid unrelated refactors
|
||||
- [x] 9.3 Maintain existing behavior for unaffected commands
|
||||
|
||||
## 10. Acceptance Criteria Mapping
|
||||
- [x] AC-1: `openspec change validate` interactive selection when no arg (TTY only; respects `--no-interactive`/env) — matches cli-change spec
|
||||
- [x] AC-2: `openspec spec validate` interactive selection when no arg (TTY only; respects `--no-interactive`/env) — matches cli-spec spec
|
||||
- [x] AC-3: New `openspec validate` supports interactive selection, bulk/filtered validation, JSON schema, progress, concurrency, exit codes — matches cli-validate spec
|
||||
|
||||
|
||||
+26
-1
@@ -9,6 +9,7 @@ import { ListCommand } from '../core/list.js';
|
||||
import { ArchiveCommand } from '../core/archive.js';
|
||||
import { registerSpecCommand } from '../commands/spec.js';
|
||||
import { ChangeCommand } from '../commands/change.js';
|
||||
import { ValidateCommand } from '../commands/validate.js';
|
||||
|
||||
const program = new Command();
|
||||
|
||||
@@ -146,7 +147,8 @@ changeCmd
|
||||
.description('Validate a change proposal')
|
||||
.option('--strict', 'Enable strict validation mode')
|
||||
.option('--json', 'Output validation report as JSON')
|
||||
.action(async (changeName?: string, options?: { strict?: boolean; json?: boolean }) => {
|
||||
.option('--no-interactive', 'Disable interactive prompts')
|
||||
.action(async (changeName?: string, options?: { strict?: boolean; json?: boolean; noInteractive?: boolean }) => {
|
||||
try {
|
||||
const changeCommand = new ChangeCommand();
|
||||
await changeCommand.validate(changeName, options);
|
||||
@@ -175,4 +177,27 @@ program
|
||||
|
||||
registerSpecCommand(program);
|
||||
|
||||
// Top-level validate command
|
||||
program
|
||||
.command('validate [item-name]')
|
||||
.description('Validate changes and specs')
|
||||
.option('--all', 'Validate all changes and specs')
|
||||
.option('--changes', 'Validate all changes')
|
||||
.option('--specs', 'Validate all specs')
|
||||
.option('--type <type>', 'Specify item type when ambiguous: change|spec')
|
||||
.option('--strict', 'Enable strict validation mode')
|
||||
.option('--json', 'Output validation results as JSON')
|
||||
.option('--concurrency <n>', 'Max concurrent validations (defaults to env OPENSPEC_CONCURRENCY or 6)')
|
||||
.option('--no-interactive', 'Disable interactive prompts')
|
||||
.action(async (itemName?: string, options?: { all?: boolean; changes?: boolean; specs?: boolean; type?: string; strict?: boolean; json?: boolean; noInteractive?: boolean; concurrency?: string }) => {
|
||||
try {
|
||||
const validateCommand = new ValidateCommand();
|
||||
await validateCommand.execute(itemName, options);
|
||||
} catch (error) {
|
||||
console.log();
|
||||
ora().fail(`Error: ${(error as Error).message}`);
|
||||
process.exit(1);
|
||||
}
|
||||
});
|
||||
|
||||
program.parse();
|
||||
+20
-8
@@ -1,9 +1,12 @@
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { select } from '@inquirer/prompts';
|
||||
import { JsonConverter } from '../core/converters/json-converter.js';
|
||||
import { Validator } from '../core/validation/validator.js';
|
||||
import { ChangeParser } from '../core/parsers/change-parser.js';
|
||||
import { Change } from '../core/schemas/index.js';
|
||||
import { isInteractive } from '../utils/interactive.js';
|
||||
import { getActiveChangeIds } from '../utils/item-discovery.js';
|
||||
|
||||
// Constants for better maintainability
|
||||
const ARCHIVE_DIR = 'archive';
|
||||
@@ -170,19 +173,28 @@ export class ChangeCommand {
|
||||
}
|
||||
}
|
||||
|
||||
async validate(changeName?: string, options?: { strict?: boolean; json?: boolean }): Promise<void> {
|
||||
async validate(changeName?: string, options?: { strict?: boolean; json?: boolean; noInteractive?: boolean }): Promise<void> {
|
||||
const changesPath = path.join(process.cwd(), 'openspec', 'changes');
|
||||
|
||||
if (!changeName) {
|
||||
const changes = await this.getActiveChanges(changesPath);
|
||||
if (changes.length === 0) {
|
||||
console.error('No change specified. No active changes found.');
|
||||
const canPrompt = isInteractive(options?.noInteractive);
|
||||
const changes = await getActiveChangeIds();
|
||||
if (canPrompt && changes.length > 0) {
|
||||
const selected = await select({
|
||||
message: 'Select a change to validate',
|
||||
choices: changes.map(id => ({ name: id, value: id })),
|
||||
});
|
||||
changeName = selected;
|
||||
} else {
|
||||
console.error(`No change specified. Available IDs: ${changes.join(', ')}`);
|
||||
if (changes.length === 0) {
|
||||
console.error('No change specified. No active changes found.');
|
||||
} else {
|
||||
console.error(`No change specified. Available IDs: ${changes.join(', ')}`);
|
||||
}
|
||||
console.error('Hint: use "openspec change list" to view available changes.');
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
console.error('Hint: use "openspec change list" to view available changes.');
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
const proposalPath = path.join(changesPath, changeName, 'proposal.md');
|
||||
|
||||
+19
-2
@@ -4,6 +4,9 @@ import { join } from 'path';
|
||||
import { MarkdownParser } from '../core/parsers/markdown-parser.js';
|
||||
import { Validator } from '../core/validation/validator.js';
|
||||
import type { Spec } from '../core/schemas/index.js';
|
||||
import { select } from '@inquirer/prompts';
|
||||
import { isInteractive } from '../utils/interactive.js';
|
||||
import { getSpecIds } from '../utils/item-discovery.js';
|
||||
|
||||
const SPECS_DIR = 'openspec/specs';
|
||||
|
||||
@@ -166,12 +169,26 @@ export function registerSpecCommand(rootProgram: typeof program) {
|
||||
});
|
||||
|
||||
specCommand
|
||||
.command('validate <spec-id>')
|
||||
.command('validate [spec-id]')
|
||||
.description('Validate a specification structure')
|
||||
.option('--strict', 'Enable strict validation mode')
|
||||
.option('--json', 'Output validation report as JSON')
|
||||
.action(async (specId: string, options: { strict?: boolean; json?: boolean }) => {
|
||||
.option('--no-interactive', 'Disable interactive prompts')
|
||||
.action(async (specId: string | undefined, options: { strict?: boolean; json?: boolean; noInteractive?: boolean }) => {
|
||||
try {
|
||||
if (!specId) {
|
||||
const canPrompt = isInteractive(options?.noInteractive);
|
||||
const specIds = await getSpecIds();
|
||||
if (canPrompt && specIds.length > 0) {
|
||||
specId = await select({
|
||||
message: 'Select a spec to validate',
|
||||
choices: specIds.map(id => ({ name: id, value: id })),
|
||||
});
|
||||
} else {
|
||||
throw new Error('Missing required argument <spec-id>');
|
||||
}
|
||||
}
|
||||
|
||||
const specPath = join(SPECS_DIR, specId, 'spec.md');
|
||||
|
||||
if (!existsSync(specPath)) {
|
||||
|
||||
@@ -0,0 +1,312 @@
|
||||
import { select } from '@inquirer/prompts';
|
||||
import ora from 'ora';
|
||||
import path from 'path';
|
||||
import { Validator } from '../core/validation/validator.js';
|
||||
import { isInteractive } from '../utils/interactive.js';
|
||||
import { getActiveChangeIds, getSpecIds } from '../utils/item-discovery.js';
|
||||
|
||||
type ItemType = 'change' | 'spec';
|
||||
|
||||
interface ExecuteOptions {
|
||||
all?: boolean;
|
||||
changes?: boolean;
|
||||
specs?: boolean;
|
||||
type?: string;
|
||||
strict?: boolean;
|
||||
json?: boolean;
|
||||
noInteractive?: boolean;
|
||||
concurrency?: string;
|
||||
}
|
||||
|
||||
interface BulkItemResult {
|
||||
id: string;
|
||||
type: ItemType;
|
||||
valid: boolean;
|
||||
issues: { level: 'ERROR' | 'WARNING' | 'INFO'; path: string; message: string }[];
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
export class ValidateCommand {
|
||||
async execute(itemName: string | undefined, options: ExecuteOptions = {}): Promise<void> {
|
||||
const interactive = isInteractive(options.noInteractive);
|
||||
|
||||
// Handle bulk flags first
|
||||
if (options.all || options.changes || options.specs) {
|
||||
await this.runBulkValidation({
|
||||
changes: !!options.all || !!options.changes,
|
||||
specs: !!options.all || !!options.specs,
|
||||
}, { strict: !!options.strict, json: !!options.json, concurrency: options.concurrency });
|
||||
return;
|
||||
}
|
||||
|
||||
// No item and no flags
|
||||
if (!itemName) {
|
||||
if (interactive) {
|
||||
await this.runInteractiveSelector({ strict: !!options.strict, json: !!options.json, concurrency: options.concurrency });
|
||||
return;
|
||||
}
|
||||
this.printNonInteractiveHint();
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
// Direct item validation with type detection or override
|
||||
const typeOverride = this.normalizeType(options.type);
|
||||
await this.validateDirectItem(itemName, { typeOverride, strict: !!options.strict, json: !!options.json });
|
||||
}
|
||||
|
||||
private normalizeType(value?: string): ItemType | undefined {
|
||||
if (!value) return undefined;
|
||||
const v = value.toLowerCase();
|
||||
if (v === 'change' || v === 'spec') return v;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
private async runInteractiveSelector(opts: { strict: boolean; json: boolean; concurrency?: string }): Promise<void> {
|
||||
const choice = await select({
|
||||
message: 'What would you like to validate?',
|
||||
choices: [
|
||||
{ name: 'All (changes + specs)', value: 'all' },
|
||||
{ name: 'All changes', value: 'changes' },
|
||||
{ name: 'All specs', value: 'specs' },
|
||||
{ name: 'Pick a specific change or spec', value: 'one' },
|
||||
],
|
||||
});
|
||||
|
||||
if (choice === 'all') return this.runBulkValidation({ changes: true, specs: true }, opts);
|
||||
if (choice === 'changes') return this.runBulkValidation({ changes: true, specs: false }, opts);
|
||||
if (choice === 'specs') return this.runBulkValidation({ changes: false, specs: true }, opts);
|
||||
|
||||
// one
|
||||
const [changes, specs] = await Promise.all([getActiveChangeIds(), getSpecIds()]);
|
||||
const items: { name: string; value: { type: ItemType; id: string } }[] = [];
|
||||
items.push(...changes.map(id => ({ name: `change/${id}`, value: { type: 'change' as const, id } })));
|
||||
items.push(...specs.map(id => ({ name: `spec/${id}`, value: { type: 'spec' as const, id } })));
|
||||
if (items.length === 0) {
|
||||
console.error('No items found to validate.');
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const picked = await select<{ type: ItemType; id: string }>({ message: 'Pick an item', choices: items });
|
||||
await this.validateByType(picked.type, picked.id, opts);
|
||||
}
|
||||
|
||||
private printNonInteractiveHint(): void {
|
||||
console.error('Nothing to validate. Try one of:');
|
||||
console.error(' openspec validate --all');
|
||||
console.error(' openspec validate --changes');
|
||||
console.error(' openspec validate --specs');
|
||||
console.error(' openspec validate <item-name>');
|
||||
console.error('Or run in an interactive terminal.');
|
||||
}
|
||||
|
||||
private async validateDirectItem(itemName: string, opts: { typeOverride?: ItemType; strict: boolean; json: boolean }): Promise<void> {
|
||||
const [changes, specs] = await Promise.all([getActiveChangeIds(), getSpecIds()]);
|
||||
const isChange = changes.includes(itemName);
|
||||
const isSpec = specs.includes(itemName);
|
||||
|
||||
const type = opts.typeOverride ?? (isChange ? 'change' : isSpec ? 'spec' : undefined);
|
||||
|
||||
if (!type) {
|
||||
console.error(`Unknown item '${itemName}'`);
|
||||
const suggestions = nearestMatches(itemName, [...changes, ...specs]);
|
||||
if (suggestions.length) console.error(`Did you mean: ${suggestions.join(', ')}?`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
if (!opts.typeOverride && isChange && isSpec) {
|
||||
console.error(`Ambiguous item '${itemName}' matches both a change and a spec.`);
|
||||
console.error('Pass --type change|spec, or use: openspec change validate / openspec spec validate');
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
await this.validateByType(type, itemName, opts);
|
||||
}
|
||||
|
||||
private async validateByType(type: ItemType, id: string, opts: { strict: boolean; json: boolean }): Promise<void> {
|
||||
const validator = new Validator(opts.strict);
|
||||
if (type === 'change') {
|
||||
const file = path.join(process.cwd(), 'openspec', 'changes', id, 'proposal.md');
|
||||
const start = Date.now();
|
||||
const report = await validator.validateChange(file);
|
||||
const durationMs = Date.now() - start;
|
||||
this.printReport('change', id, report, durationMs, opts.json);
|
||||
process.exitCode = report.valid ? 0 : 1;
|
||||
return;
|
||||
}
|
||||
const file = path.join(process.cwd(), 'openspec', 'specs', id, 'spec.md');
|
||||
const start = Date.now();
|
||||
const report = await validator.validateSpec(file);
|
||||
const durationMs = Date.now() - start;
|
||||
this.printReport('spec', id, report, durationMs, opts.json);
|
||||
process.exitCode = report.valid ? 0 : 1;
|
||||
}
|
||||
|
||||
private printReport(type: ItemType, id: string, report: { valid: boolean; issues: any[] }, durationMs: number, json: boolean): void {
|
||||
if (json) {
|
||||
const out = { items: [{ id, type, valid: report.valid, issues: report.issues, durationMs }], summary: { totals: { items: 1, passed: report.valid ? 1 : 0, failed: report.valid ? 0 : 1 }, byType: { [type]: { items: 1, passed: report.valid ? 1 : 0, failed: report.valid ? 0 : 1 } } }, version: '1.0' };
|
||||
console.log(JSON.stringify(out, null, 2));
|
||||
return;
|
||||
}
|
||||
if (report.valid) {
|
||||
console.log(`${type === 'change' ? 'Change' : 'Specification'} '${id}' is valid`);
|
||||
} else {
|
||||
console.error(`${type === 'change' ? 'Change' : 'Specification'} '${id}' has issues`);
|
||||
for (const issue of report.issues) {
|
||||
const label = issue.level === 'ERROR' ? 'ERROR' : issue.level;
|
||||
const prefix = issue.level === 'ERROR' ? '✗' : issue.level === 'WARNING' ? '⚠' : 'ℹ';
|
||||
console.error(`${prefix} [${label}] ${issue.path}: ${issue.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async runBulkValidation(scope: { changes: boolean; specs: boolean }, opts: { strict: boolean; json: boolean; concurrency?: string }): Promise<void> {
|
||||
const spinner = !opts.json ? ora('Validating...').start() : undefined;
|
||||
const [changeIds, specIds] = await Promise.all([
|
||||
scope.changes ? getActiveChangeIds() : Promise.resolve<string[]>([]),
|
||||
scope.specs ? getSpecIds() : Promise.resolve<string[]>([]),
|
||||
]);
|
||||
|
||||
const DEFAULT_CONCURRENCY = 6;
|
||||
const maxSuggestions = 5; // used by nearestMatches
|
||||
const concurrency = normalizeConcurrency(opts.concurrency) ?? normalizeConcurrency(process.env.OPENSPEC_CONCURRENCY) ?? DEFAULT_CONCURRENCY;
|
||||
const validator = new Validator(opts.strict);
|
||||
const queue: Array<() => Promise<BulkItemResult>> = [];
|
||||
|
||||
for (const id of changeIds) {
|
||||
queue.push(async () => {
|
||||
const start = Date.now();
|
||||
const file = path.join(process.cwd(), 'openspec', 'changes', id, 'proposal.md');
|
||||
const report = await validator.validateChange(file);
|
||||
const durationMs = Date.now() - start;
|
||||
return { id, type: 'change' as const, valid: report.valid, issues: report.issues, durationMs };
|
||||
});
|
||||
}
|
||||
for (const id of specIds) {
|
||||
queue.push(async () => {
|
||||
const start = Date.now();
|
||||
const file = path.join(process.cwd(), 'openspec', 'specs', id, 'spec.md');
|
||||
const report = await validator.validateSpec(file);
|
||||
const durationMs = Date.now() - start;
|
||||
return { id, type: 'spec' as const, valid: report.valid, issues: report.issues, durationMs };
|
||||
});
|
||||
}
|
||||
|
||||
const results: BulkItemResult[] = [];
|
||||
let index = 0;
|
||||
let running = 0;
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
|
||||
await new Promise<void>((resolve) => {
|
||||
const next = () => {
|
||||
while (running < concurrency && index < queue.length) {
|
||||
const currentIndex = index++;
|
||||
const task = queue[currentIndex];
|
||||
running++;
|
||||
if (spinner) spinner.text = `Validating (${currentIndex + 1}/${queue.length})...`;
|
||||
task()
|
||||
.then(res => {
|
||||
results.push(res);
|
||||
if (res.valid) passed++; else failed++;
|
||||
})
|
||||
.catch((error: any) => {
|
||||
const message = error?.message || 'Unknown error';
|
||||
const res: BulkItemResult = { id: getPlannedId(currentIndex, changeIds, specIds) ?? 'unknown', type: getPlannedType(currentIndex, changeIds, specIds) ?? 'change', valid: false, issues: [{ level: 'ERROR', path: 'file', message }], durationMs: 0 };
|
||||
results.push(res);
|
||||
failed++;
|
||||
})
|
||||
.finally(() => {
|
||||
running--;
|
||||
if (index >= queue.length && running === 0) resolve();
|
||||
else next();
|
||||
});
|
||||
}
|
||||
};
|
||||
next();
|
||||
});
|
||||
|
||||
spinner?.stop();
|
||||
|
||||
results.sort((a, b) => a.id.localeCompare(b.id));
|
||||
const summary = {
|
||||
totals: { items: results.length, passed, failed },
|
||||
byType: {
|
||||
...(scope.changes ? { change: summarizeType(results, 'change') } : {}),
|
||||
...(scope.specs ? { spec: summarizeType(results, 'spec') } : {}),
|
||||
},
|
||||
} as const;
|
||||
|
||||
if (opts.json) {
|
||||
const out = { items: results, summary, version: '1.0' };
|
||||
console.log(JSON.stringify(out, null, 2));
|
||||
} else {
|
||||
for (const res of results) {
|
||||
if (res.valid) console.log(`✓ ${res.type}/${res.id}`);
|
||||
else console.error(`✗ ${res.type}/${res.id}`);
|
||||
}
|
||||
console.log(`Totals: ${summary.totals.passed} passed, ${summary.totals.failed} failed (${summary.totals.items} items)`);
|
||||
}
|
||||
|
||||
process.exitCode = failed > 0 ? 1 : 0;
|
||||
}
|
||||
}
|
||||
|
||||
function summarizeType(results: BulkItemResult[], type: ItemType) {
|
||||
const filtered = results.filter(r => r.type === type);
|
||||
const items = filtered.length;
|
||||
const passed = filtered.filter(r => r.valid).length;
|
||||
const failed = items - passed;
|
||||
return { items, passed, failed };
|
||||
}
|
||||
|
||||
function nearestMatches(input: string, candidates: string[], max: number = 5): string[] {
|
||||
const scored = candidates.map(c => ({ c, d: levenshtein(input, c) }));
|
||||
scored.sort((a, b) => a.d - b.d);
|
||||
return scored.slice(0, max).map(s => s.c);
|
||||
}
|
||||
|
||||
function levenshtein(a: string, b: string): number {
|
||||
const m = a.length;
|
||||
const n = b.length;
|
||||
const dp: number[][] = Array.from({ length: m + 1 }, () => Array(n + 1).fill(0));
|
||||
for (let i = 0; i <= m; i++) dp[i][0] = i;
|
||||
for (let j = 0; j <= n; j++) dp[0][j] = j;
|
||||
for (let i = 1; i <= m; i++) {
|
||||
for (let j = 1; j <= n; j++) {
|
||||
const cost = a[i - 1] === b[j - 1] ? 0 : 1;
|
||||
dp[i][j] = Math.min(
|
||||
dp[i - 1][j] + 1,
|
||||
dp[i][j - 1] + 1,
|
||||
dp[i - 1][j - 1] + cost
|
||||
);
|
||||
}
|
||||
}
|
||||
return dp[m][n];
|
||||
}
|
||||
|
||||
function normalizeConcurrency(value?: string): number | undefined {
|
||||
if (!value) return undefined;
|
||||
const n = parseInt(value, 10);
|
||||
if (Number.isNaN(n) || n <= 0) return undefined;
|
||||
return n;
|
||||
}
|
||||
|
||||
function getPlannedId(index: number, changeIds: string[], specIds: string[]): string | undefined {
|
||||
const totalChanges = changeIds.length;
|
||||
if (index < totalChanges) return changeIds[index];
|
||||
const specIndex = index - totalChanges;
|
||||
return specIds[specIndex];
|
||||
}
|
||||
|
||||
function getPlannedType(index: number, changeIds: string[], specIds: string[]): ItemType | undefined {
|
||||
const totalChanges = changeIds.length;
|
||||
if (index < totalChanges) return 'change';
|
||||
const specIndex = index - totalChanges;
|
||||
if (specIndex >= 0 && specIndex < specIds.length) return 'spec';
|
||||
return undefined;
|
||||
}
|
||||
|
||||
|
||||
@@ -94,7 +94,8 @@ export class ChangeParser extends MarkdownParser {
|
||||
spec: specName,
|
||||
operation: 'ADDED' as DeltaOperation,
|
||||
description: `Add requirement: ${req.text}`,
|
||||
// Use plural form to satisfy validators that expect an array
|
||||
// Provide both single and plural forms for compatibility
|
||||
requirement: req,
|
||||
requirements: [req],
|
||||
});
|
||||
});
|
||||
@@ -109,6 +110,7 @@ export class ChangeParser extends MarkdownParser {
|
||||
spec: specName,
|
||||
operation: 'MODIFIED' as DeltaOperation,
|
||||
description: `Modify requirement: ${req.text}`,
|
||||
requirement: req,
|
||||
requirements: [req],
|
||||
});
|
||||
});
|
||||
@@ -123,6 +125,7 @@ export class ChangeParser extends MarkdownParser {
|
||||
spec: specName,
|
||||
operation: 'REMOVED' as DeltaOperation,
|
||||
description: `Remove requirement: ${req.text}`,
|
||||
requirement: req,
|
||||
requirements: [req],
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
export function isInteractive(noInteractiveFlag?: boolean): boolean {
|
||||
if (noInteractiveFlag) return false;
|
||||
if (process.env.OPEN_SPEC_INTERACTIVE === '0') return false;
|
||||
return !!process.stdin.isTTY;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
export async function getActiveChangeIds(root: string = process.cwd()): Promise<string[]> {
|
||||
const changesPath = path.join(root, 'openspec', 'changes');
|
||||
try {
|
||||
const entries = await fs.readdir(changesPath, { withFileTypes: true });
|
||||
return entries
|
||||
.filter(entry => entry.isDirectory() && !entry.name.startsWith('.') && entry.name !== 'archive')
|
||||
.map(entry => entry.name)
|
||||
.sort();
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export async function getSpecIds(root: string = process.cwd()): Promise<string[]> {
|
||||
const specsPath = path.join(root, 'openspec', 'specs');
|
||||
const result: string[] = [];
|
||||
try {
|
||||
const entries = await fs.readdir(specsPath, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory() || entry.name.startsWith('.')) continue;
|
||||
const specFile = path.join(specsPath, entry.name, 'spec.md');
|
||||
try {
|
||||
await fs.access(specFile);
|
||||
result.push(entry.name);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
return result.sort();
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
// Note: We cannot truly simulate TTY prompts in this test runner easily.
|
||||
// Instead, we verify non-interactive fallback behavior and basic invocation.
|
||||
|
||||
describe('change validate (interactive behavior)', () => {
|
||||
const projectRoot = process.cwd();
|
||||
const testDir = path.join(projectRoot, 'test-change-validate-tmp');
|
||||
const changesDir = path.join(testDir, 'openspec', 'changes');
|
||||
const bin = path.join(projectRoot, 'bin', 'openspec.js');
|
||||
|
||||
beforeAll(() => {
|
||||
execSync('pnpm -s build', { stdio: 'pipe' });
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await fs.mkdir(changesDir, { recursive: true });
|
||||
const content = `# Change: Demo\n\n## Why\nBecause reasons that are sufficiently long.\n\n## What Changes\n- **spec-x:** Add something`;
|
||||
await fs.mkdir(path.join(changesDir, 'demo'), { recursive: true });
|
||||
await fs.writeFile(path.join(changesDir, 'demo', 'proposal.md'), content, 'utf-8');
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fs.rm(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('prints list hint and exits non-zero when no arg and non-interactive', () => {
|
||||
const originalCwd = process.cwd();
|
||||
const originalEnv = { ...process.env };
|
||||
try {
|
||||
process.chdir(testDir);
|
||||
process.env.OPEN_SPEC_INTERACTIVE = '0';
|
||||
let err: any;
|
||||
try {
|
||||
execSync(`node ${bin} change validate`, { encoding: 'utf-8' });
|
||||
} catch (e) { err = e; }
|
||||
expect(err).toBeDefined();
|
||||
expect(err.status).not.toBe(0);
|
||||
expect(err.stderr.toString()).toContain('Available IDs:');
|
||||
expect(err.stderr.toString()).toContain('openspec change list');
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
process.env = originalEnv;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
describe('spec validate (interactive behavior)', () => {
|
||||
const projectRoot = process.cwd();
|
||||
const testDir = path.join(projectRoot, 'test-spec-validate-tmp');
|
||||
const specsDir = path.join(testDir, 'openspec', 'specs');
|
||||
const bin = path.join(projectRoot, 'bin', 'openspec.js');
|
||||
|
||||
beforeAll(() => {
|
||||
execSync('pnpm -s build', { stdio: 'pipe' });
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await fs.mkdir(specsDir, { recursive: true });
|
||||
const content = `## Purpose\nValid spec for interactive test.\n\n## Requirements\n\n### Requirement: X\nText`;
|
||||
await fs.mkdir(path.join(specsDir, 's1'), { recursive: true });
|
||||
await fs.writeFile(path.join(specsDir, 's1', 'spec.md'), content, 'utf-8');
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fs.rm(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('errors when no arg and non-interactive', () => {
|
||||
const originalCwd = process.cwd();
|
||||
const originalEnv = { ...process.env };
|
||||
try {
|
||||
process.chdir(testDir);
|
||||
process.env.OPEN_SPEC_INTERACTIVE = '0';
|
||||
let err: any;
|
||||
try {
|
||||
execSync(`node ${bin} spec validate`, { encoding: 'utf-8' });
|
||||
} catch (e) { err = e; }
|
||||
expect(err).toBeDefined();
|
||||
expect(err.status).not.toBe(0);
|
||||
expect(err.stderr.toString()).toContain('Missing required argument <spec-id>');
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
process.env = originalEnv;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
describe('top-level validate command', () => {
|
||||
const projectRoot = process.cwd();
|
||||
const testDir = path.join(projectRoot, 'test-validate-command-tmp');
|
||||
const changesDir = path.join(testDir, 'openspec', 'changes');
|
||||
const specsDir = path.join(testDir, 'openspec', 'specs');
|
||||
const bin = path.join(projectRoot, 'bin', 'openspec.js');
|
||||
|
||||
beforeAll(() => {
|
||||
execSync('pnpm -s build', { stdio: 'pipe' });
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await fs.mkdir(changesDir, { recursive: true });
|
||||
await fs.mkdir(specsDir, { recursive: true });
|
||||
|
||||
// Create a valid spec
|
||||
const specContent = `## Purpose
|
||||
Valid spec for testing.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Foo
|
||||
Text
|
||||
|
||||
#### Scenario: Bar
|
||||
Given A\nWhen B\nThen C`;
|
||||
await fs.mkdir(path.join(specsDir, 'alpha'), { recursive: true });
|
||||
await fs.writeFile(path.join(specsDir, 'alpha', 'spec.md'), specContent, 'utf-8');
|
||||
|
||||
// Create a simple change with bullets (parser supports this)
|
||||
const changeContent = `# Test Change\n\n## Why\nBecause reasons that are sufficiently long for validation.\n\n## What Changes\n- **alpha:** Add something`;
|
||||
await fs.mkdir(path.join(changesDir, 'c1'), { recursive: true });
|
||||
await fs.writeFile(path.join(changesDir, 'c1', 'proposal.md'), changeContent, 'utf-8');
|
||||
|
||||
// Duplicate name for ambiguity test
|
||||
await fs.mkdir(path.join(changesDir, 'dup'), { recursive: true });
|
||||
await fs.writeFile(path.join(changesDir, 'dup', 'proposal.md'), changeContent, 'utf-8');
|
||||
await fs.mkdir(path.join(specsDir, 'dup'), { recursive: true });
|
||||
await fs.writeFile(path.join(specsDir, 'dup', 'spec.md'), specContent, 'utf-8');
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fs.rm(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('prints a helpful hint when no args in non-interactive mode', () => {
|
||||
const originalCwd = process.cwd();
|
||||
const originalEnv = { ...process.env };
|
||||
try {
|
||||
process.chdir(testDir);
|
||||
process.env.OPEN_SPEC_INTERACTIVE = '0';
|
||||
let err: any;
|
||||
try {
|
||||
execSync(`node ${bin} validate`, { encoding: 'utf-8' });
|
||||
} catch (e) { err = e; }
|
||||
expect(err).toBeDefined();
|
||||
expect(err.status).not.toBe(0);
|
||||
expect(err.stderr.toString()).toContain('Nothing to validate. Try one of:');
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
process.env = originalEnv;
|
||||
}
|
||||
});
|
||||
|
||||
it('validates all with --all and outputs JSON summary', () => {
|
||||
const originalCwd = process.cwd();
|
||||
try {
|
||||
process.chdir(testDir);
|
||||
let outStr = '';
|
||||
try {
|
||||
outStr = execSync(`node ${bin} validate --all --json`, { encoding: 'utf-8' });
|
||||
} catch (e: any) {
|
||||
// If exit code is non-zero (e.g., on failures), still parse stdout JSON
|
||||
outStr = e.stdout?.toString?.() ?? '';
|
||||
}
|
||||
const json = JSON.parse(outStr);
|
||||
expect(Array.isArray(json.items)).toBe(true);
|
||||
expect(json.summary?.totals?.items).toBeDefined();
|
||||
expect(json.version).toBe('1.0');
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
}
|
||||
});
|
||||
|
||||
it('validates only specs with --specs and respects --concurrency', () => {
|
||||
const originalCwd = process.cwd();
|
||||
try {
|
||||
process.chdir(testDir);
|
||||
let outStr = '';
|
||||
try {
|
||||
outStr = execSync(`node ${bin} validate --specs --json --concurrency 1`, { encoding: 'utf-8' });
|
||||
} catch (e: any) {
|
||||
outStr = e.stdout?.toString?.() ?? '';
|
||||
}
|
||||
const json = JSON.parse(outStr);
|
||||
// All items should be specs
|
||||
expect(json.items.every((i: any) => i.type === 'spec')).toBe(true);
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
}
|
||||
});
|
||||
|
||||
it('errors on ambiguous item names and suggests type override', () => {
|
||||
const originalCwd = process.cwd();
|
||||
try {
|
||||
process.chdir(testDir);
|
||||
let err: any;
|
||||
try {
|
||||
execSync(`node ${bin} validate dup`, { encoding: 'utf-8' });
|
||||
} catch (e) { err = e; }
|
||||
expect(err).toBeDefined();
|
||||
expect(err.stderr.toString()).toContain('Ambiguous item');
|
||||
expect(err.status).not.toBe(0);
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user