Files
headroom/tests/test_code_compressor_bash.py
宇宙的山峰 ee902074a0 fix(compression): preserve Bash control flow (#3425)
## Description

Closes #3423

Pasting valid Bash into a code-aware compression path could classify it
as `UNKNOWN` and send it to lossy Kompress. Bash control-flow words such
as `then` and `fi` could then be dropped, changing the script's meaning
or syntax.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New tests added for the regression

## Changes Made

- Add `CodeLanguage.BASH` plus the `SHELL` enum alias and common
`shell`/`sh`/`zsh` fence aliases.
- Detect Bash from shebangs and shell control-flow/expansion syntax
using the existing pre-filter.
- Register the Bash tree-sitter grammar and validate recognized Bash,
but preserve it losslessly for now. The generic AST reassembler is not
safe for Bash's anonymous `then`/`fi`, `do`/`done`, and `esac`
delimiters, so valid shell is no longer sent to Kompress and malformed
shell is reported as invalid rather than rewritten.
- Add regression coverage using the control-flow example from the issue,
including the exact `then`/`fi` delimiters and nested expansions.

## Testing

- [x] Focused unit tests pass
- [x] Linting passes for touched files
- [x] Python compilation passes
- [x] New regression tests added

### Test Output

```text
PYTHONPATH= VIRTUAL_ENV= ./.venv/Scripts/python.exe -m pytest tests/test_code_compressor_bash.py tests/test_code_compressor_language_alias.py tests/test_code_compressor_syntax_breaker.py tests/test_transforms/test_code_compressor.py -q
131 passed in 15.46s

uvx --from 'ruff==0.16.3' ruff check headroom/transforms/code_compressor.py tests/test_code_compressor_bash.py
All checks passed!

uvx --from 'ruff==0.16.3' ruff format --check headroom/transforms/code_compressor.py tests/test_code_compressor_bash.py
2 files already formatted

PYTHONPATH= VIRTUAL_ENV= ./.venv/Scripts/python.exe -m py_compile headroom/transforms/code_compressor.py tests/test_code_compressor_bash.py
```

## Real Behavior Proof

- Environment: Windows 10, Python 3.12.14, `tree-sitter==0.26.0`,
`tree-sitter-language-pack==0.13.0`, clean checkout of
`headroomlabs-ai/headroom` `main`.
- Exact command / steps: ran `CodeAwareCompressor` with
`min_tokens_for_compression=1` and `enable_ccr=False` against the
issue's Bash sample; also ran the new Bash regression tests.
- Observed result: the sample is detected as `CodeLanguage.BASH`, parses
successfully, returns `syntax_valid=True`, and the compressed result is
byte-for-byte identical to the input (`compression_ratio=1.0`),
preserving both `then`/`fi` pairs and all shell expansions.
- Not tested: `shellcheck`, a full repository test run, and a production
proxy process with Kompress model weights. No Kompress model is needed
for the regression path because recognized Bash is deliberately handled
before fallback.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, for recognized Bash/shell input:
it now avoids lossy fallback and passes the source through unchanged.
- Kill switch / disable path: None needed; this is a safety-preserving
classification path.
- Unsafe override required: No.
- Qualification impact: Bash control-flow tokens are no longer silently
removed by Kompress fallback.
- Rollback path: revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have added tests that prove my fix is effective
- [x] New and existing focused tests pass locally
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

This deliberately chooses losslessness over speculative Bash
compression. A future Bash-specific compressor can be added behind the
validated `CodeLanguage.BASH` route without reopening the corruption
bug.
2026-09-05 10:07:26 +05:30

70 lines
2.2 KiB
Python

"""Regression tests for lossless Bash handling in CodeAwareCompressor."""
from headroom.transforms.code_compressor import (
CodeAwareCompressor,
CodeCompressorConfig,
CodeLanguage,
coerce_language,
detect_language,
)
BASH_SNIPPET = """#!/usr/bin/env bash
set -euo pipefail
if [ -x "$(command -v /opt/bin/forgejo)" ]
then
\texport GITEA_WORK_DIR=/git/forgejo
\texport FORGEJO_CUSTOM=/etc/forgejo
\talias forge="sudo -Eu git GITEA_CUSTOM=${FORGEJO_CUSTOM} GITEA_WORK_DIR=${GITEA_WORK_DIR} /opt/bin/forgejo"
fi
if [[ "$( git config --global alias.pushall 2>/dev/null )" != '!git remote | xargs -L1 git push --all' ]]
then
\tgit config --global --unset-all alias.pushall
git config --global --add alias.pushall '!git remote | xargs -L1 git push --all'
fi
"""
def test_bash_aliases_are_canonicalized():
assert coerce_language("bash") is CodeLanguage.BASH
assert coerce_language("shell") is CodeLanguage.BASH
assert coerce_language("sh") is CodeLanguage.BASH
assert coerce_language("zsh") is CodeLanguage.BASH
def test_bash_is_detected_from_control_flow():
detected, confidence = detect_language(BASH_SNIPPET.removeprefix("#!/usr/bin/env bash\n"))
assert detected is CodeLanguage.BASH
assert confidence > 0
def test_bash_is_detected_from_shebang():
detected, confidence = detect_language("#!/bin/sh\necho hello\n")
assert detected is CodeLanguage.BASH
assert confidence > 0
def test_bash_compression_is_lossless_and_does_not_use_kompress():
compressor = CodeAwareCompressor(
CodeCompressorConfig(min_tokens_for_compression=1, enable_ccr=False)
)
result = compressor.compress(BASH_SNIPPET)
assert result.language is CodeLanguage.BASH
assert result.syntax_valid is True
assert result.compressed == BASH_SNIPPET
assert result.compressed_tokens == result.original_tokens
assert result.compression_ratio == 1.0
def test_shell_hint_is_lossless():
compressor = CodeAwareCompressor(
CodeCompressorConfig(min_tokens_for_compression=1, enable_ccr=False)
)
result = compressor.compress(BASH_SNIPPET, language="shell")
assert result.language is CodeLanguage.BASH
assert result.compressed == BASH_SNIPPET
assert result.syntax_valid is True