Files
headroom/tests/test_cache/test_openai.py
2480100915 fix(cache): keep indented lines intact in OpenAI whitespace normalization (#4343)
## Description

`OpenAICacheOptimizer._normalize_whitespace` measured a line's
indentation, collapsed the line with `" ".join(stripped.split())` (which
already drops the indentation), and then sliced
`content_part[leading:]`. So every indented line in a system prompt lost
its first `leading` characters: `" def f():"` became `" f():"`, `"
return x"` became `" "`, and tab-indented lines lost characters and had
their tabs turned into spaces. Unindented lines were fine, which is why
the existing test didn't catch it.

The fix keeps the line's original leading whitespace and appends the
collapsed content: `line[:leading] + content_part`. Inner runs of spaces
are still collapsed, trailing whitespace is still trimmed, and tabs in
the indentation are now kept as tabs instead of being rewritten as
spaces.

Fixes #4336

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cache/openai.py`: in `_normalize_whitespace`, emit
`line[:leading] + content_part` instead of `" " * leading +
content_part[leading:]`.
- `tests/test_cache/test_openai.py`:
`test_whitespace_normalization_keeps_indented_lines_intact` checks that
a space-indented code block round-trips unchanged, that a tab-indented
block keeps its tabs while inner spaces are collapsed, and that the same
system prompt comes out of `optimize()` unchanged. It fails on `main`
and passes with the fix.
- Follow-up from review: trim only surrounding newlines at the end of
`_normalize_whitespace`, so a prompt that starts with an indented line
keeps its indentation
(`test_whitespace_normalization_keeps_first_line_indentation`).

## Testing

- [x] Unit tests pass (`pytest`) - `tests/test_cache/`: 298 passed, 2
skipped
- [x] Linting passes (`ruff check .`) - `ruff check` and `ruff format
--check` on the two changed files
- [x] Type checking passes (`mypy headroom`) - `mypy
headroom/cache/openai.py`, no issues
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

New test on `main` (fix stashed):

```text
$ pytest tests/test_cache/test_openai.py -q -k indented
E   AssertionError: assert 'Example:\n    f():' == 'Example:\n  ...     return x'
E       Example:
E     -     def f():
E     ?    ----     -
E     +     f():
E     -         return x
======================= 1 failed, 13 deselected in 0.26s =======================
```

With the fix:

```text
$ pytest tests/test_cache/test_openai.py -q -k indented
======================= 1 passed, 13 deselected in 0.24s =======================

$ pytest tests/test_cache/ -q
======================= 298 passed, 2 skipped in 15.08s ========================
```

## Real Behavior Proof

- Environment: macOS 26.2, Python 3.14.7, repo deps via `uv sync --extra
dev`, optimizer resolved the same way `HeadroomClient` does it
(`CacheOptimizerRegistry.get("openai")`), default `CacheConfig`.
- Exact command / steps: a small script that gets the `"openai"`
optimizer from the registry and runs `optimize()` on a system prompt
with a space-indented code block, a nested bullet and a tab-indented
block:

  ```python
  from headroom.cache import OptimizationContext
  from headroom.cache.registry import CacheOptimizerRegistry
  opt = CacheOptimizerRegistry.get("openai")
prompt = "You are a code reviewer. Follow this style:\n def
handler(event):\n return process(event)\n - nested bullet\n\tif
ok:\n\t\treturn x"
res = opt.optimize([{"role": "system", "content": prompt}, {"role":
"user", "content": "hi"}], OptimizationContext(provider="openai",
model="gpt-4o"))
  print(repr(res.messages[0]["content"]))
  ```

- Observed result: indentation and text are preserved after the fix.
Before and after output:

  Before (`main`):
  ```text
'You are a code reviewer. Follow this style:\n handler(event):\n
rocess(event)\n nested bullet\n f ok:\n turn x'
  ```
  After:
  ```text
'You are a code reviewer. Follow this style:\n def handler(event):\n
return process(event)\n - nested bullet\n\tif ok:\n\t\treturn x'
  ```
Indentation and text are preserved; the double space in `nested bullet`
and `return x` is still collapsed.

- Not tested: a live request through `HeadroomClient` to the OpenAI API
(no key used); the optimizer output above is exactly what that path
sends upstream. The Anthropic and Google optimizers were not touched.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, only in that indented
system-prompt lines are no longer truncated. Tab indentation is now kept
as tabs instead of being converted to spaces.
- Kill switch / disable path: `CacheConfig(normalize_whitespace=False)`
(unchanged)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`, it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Comments and docs: the existing comments already describe the intended
behavior ("Collapse multiple spaces in content (not indentation)"), so
no new comment or doc change was needed. The cache prefix hash for
prompts with indented lines will change once, since the normalized text
is now correct.

---------

Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:38:06 -07:00

229 lines
8.1 KiB
Python

"""Tests for OpenAICacheOptimizer."""
import pytest
from headroom.cache import CacheConfig, OpenAICacheOptimizer, OptimizationContext
from headroom.cache.base import CacheStrategy
class TestOpenAICacheOptimizer:
"""Test OpenAICacheOptimizer functionality."""
@pytest.fixture
def optimizer(self):
"""Create optimizer instance."""
return OpenAICacheOptimizer()
@pytest.fixture
def context(self):
"""Create optimization context."""
return OptimizationContext(
provider="openai",
model="gpt-4",
)
def test_optimizer_properties(self, optimizer):
"""Test optimizer properties."""
assert optimizer.name == "openai-prefix-stabilizer"
assert optimizer.provider == "openai"
assert optimizer.strategy == CacheStrategy.PREFIX_STABILIZATION
def test_optimize_simple_messages(self, optimizer, context):
"""Test optimizing simple messages."""
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
assert result.messages is not None
assert len(result.messages) == 2
assert result.metrics.stable_prefix_hash != ""
def test_date_extraction(self, optimizer, context):
"""Test that dates are extracted from system prompt."""
messages = [
{
"role": "system",
"content": "Today is January 7, 2026. You are a helpful assistant.",
},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
# Check that date was extracted and moved
system_content = result.messages[0]["content"]
# The date should be moved to a dynamic section at the end
assert "You are a helpful assistant" in system_content
def test_whitespace_normalization(self, optimizer, context):
"""Test whitespace normalization."""
messages = [
{
"role": "system",
"content": "You are a helpful assistant.",
},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
# Whitespace should be normalized
system_content = result.messages[0]["content"]
assert " " not in system_content # Multiple spaces collapsed
def test_whitespace_normalization_keeps_indented_lines_intact(self, optimizer, context):
"""Indented lines keep their indentation and their first characters (#4336)."""
code = "Example:\n def f():\n return x"
assert optimizer._normalize_whitespace(code) == code
tabbed = "Example:\n\tif ok:\n\t\treturn x"
assert optimizer._normalize_whitespace(tabbed) == "Example:\n\tif ok:\n\t\treturn x"
messages = [
{"role": "system", "content": code},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
assert result.messages[0]["content"] == code
def test_whitespace_normalization_keeps_first_line_indentation(self, optimizer, context):
"""A prompt that starts with an indented line keeps that indentation."""
code = " if ready:\n run()"
assert optimizer._normalize_whitespace(code) == code
# Surrounding blank lines and trailing whitespace are still trimmed.
padded = "\n \n if ready:\n run() \n\n"
assert optimizer._normalize_whitespace(padded) == code
messages = [
{"role": "system", "content": code},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
assert result.messages[0]["content"] == code
def test_optimize_disabled(self, context):
"""Test optimization when disabled."""
config = CacheConfig(enabled=False)
optimizer = OpenAICacheOptimizer(config)
messages = [
{"role": "system", "content": "Test"},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
assert result.transforms_applied == []
def test_prefix_stability_tracking(self, optimizer, context):
"""Test that prefix stability is tracked."""
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"},
]
# First call
optimizer.optimize(messages, context)
# Second call with same messages
result2 = optimizer.optimize(messages, context)
# Second call should detect stable prefix
assert result2.metrics.estimated_cache_hit is True
assert result2.metrics.prefix_changed_from_previous is False
def test_prefix_change_detection(self, optimizer, context):
"""Test detection of prefix changes."""
messages1 = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"},
]
messages2 = [
{"role": "system", "content": "You are a different assistant."},
{"role": "user", "content": "Hello!"},
]
optimizer.optimize(messages1, context)
result2 = optimizer.optimize(messages2, context)
# Second call should detect prefix change
assert result2.metrics.prefix_changed_from_previous is True
def test_token_threshold_warning(self, optimizer, context):
"""Test warning when below token threshold."""
messages = [
{"role": "system", "content": "Short."},
{"role": "user", "content": "Hi"},
]
result = optimizer.optimize(messages, context)
# Should have warning about being below threshold
assert any("1024" in w for w in result.warnings)
def test_estimate_savings_below_threshold(self, optimizer, context):
"""Test savings estimation below threshold."""
messages = [
{"role": "system", "content": "Short system prompt."},
{"role": "user", "content": "Hello!"},
]
savings = optimizer.estimate_savings(messages, context)
assert savings == 0.0 # Below threshold
def test_estimate_savings_above_threshold(self, optimizer, context):
"""Test savings estimation above threshold."""
messages = [
{"role": "system", "content": "You are helpful. " * 500},
{"role": "user", "content": "Hello!"},
]
# First call to establish baseline
optimizer.optimize(messages, context)
# Second call should show savings
savings = optimizer.estimate_savings(messages, context)
assert savings > 0.0
def test_uuid_pattern_detection(self, optimizer, context):
"""Test detection of UUIDs in content."""
messages = [
{
"role": "system",
"content": "Request ID: 12345678-1234-1234-1234-123456789012. Be helpful.",
},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
# UUID should be detected as dynamic content
assert result.metrics.stable_prefix_hash != ""
def test_content_block_format(self, optimizer, context):
"""Test handling of content block format."""
messages = [
{
"role": "system",
"content": [{"type": "text", "text": "You are helpful."}],
},
{"role": "user", "content": "Hello!"},
]
result = optimizer.optimize(messages, context)
assert result.messages is not None
def test_metrics_recording(self, optimizer, context):
"""Test that metrics are recorded."""
messages = [
{"role": "system", "content": "You are helpful."},
{"role": "user", "content": "Hello!"},
]
optimizer.optimize(messages, context)
metrics = optimizer.get_metrics()
assert metrics.stable_prefix_hash != ""