Commit Graph
3 Commits
Author SHA1 Message Date
2480100915 fix(cache): keep indented lines intact in OpenAI whitespace normalization (#4343)
## Description

`OpenAICacheOptimizer._normalize_whitespace` measured a line's
indentation, collapsed the line with `" ".join(stripped.split())` (which
already drops the indentation), and then sliced
`content_part[leading:]`. So every indented line in a system prompt lost
its first `leading` characters: `" def f():"` became `" f():"`, `"
return x"` became `" "`, and tab-indented lines lost characters and had
their tabs turned into spaces. Unindented lines were fine, which is why
the existing test didn't catch it.

The fix keeps the line's original leading whitespace and appends the
collapsed content: `line[:leading] + content_part`. Inner runs of spaces
are still collapsed, trailing whitespace is still trimmed, and tabs in
the indentation are now kept as tabs instead of being rewritten as
spaces.

Fixes #4336

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cache/openai.py`: in `_normalize_whitespace`, emit
`line[:leading] + content_part` instead of `" " * leading +
content_part[leading:]`.
- `tests/test_cache/test_openai.py`:
`test_whitespace_normalization_keeps_indented_lines_intact` checks that
a space-indented code block round-trips unchanged, that a tab-indented
block keeps its tabs while inner spaces are collapsed, and that the same
system prompt comes out of `optimize()` unchanged. It fails on `main`
and passes with the fix.
- Follow-up from review: trim only surrounding newlines at the end of
`_normalize_whitespace`, so a prompt that starts with an indented line
keeps its indentation
(`test_whitespace_normalization_keeps_first_line_indentation`).

## Testing

- [x] Unit tests pass (`pytest`) - `tests/test_cache/`: 298 passed, 2
skipped
- [x] Linting passes (`ruff check .`) - `ruff check` and `ruff format
--check` on the two changed files
- [x] Type checking passes (`mypy headroom`) - `mypy
headroom/cache/openai.py`, no issues
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

New test on `main` (fix stashed):

```text
$ pytest tests/test_cache/test_openai.py -q -k indented
E   AssertionError: assert 'Example:\n    f():' == 'Example:\n  ...     return x'
E       Example:
E     -     def f():
E     ?    ----     -
E     +     f():
E     -         return x
======================= 1 failed, 13 deselected in 0.26s =======================
```

With the fix:

```text
$ pytest tests/test_cache/test_openai.py -q -k indented
======================= 1 passed, 13 deselected in 0.24s =======================

$ pytest tests/test_cache/ -q
======================= 298 passed, 2 skipped in 15.08s ========================
```

## Real Behavior Proof

- Environment: macOS 26.2, Python 3.14.7, repo deps via `uv sync --extra
dev`, optimizer resolved the same way `HeadroomClient` does it
(`CacheOptimizerRegistry.get("openai")`), default `CacheConfig`.
- Exact command / steps: a small script that gets the `"openai"`
optimizer from the registry and runs `optimize()` on a system prompt
with a space-indented code block, a nested bullet and a tab-indented
block:

  ```python
  from headroom.cache import OptimizationContext
  from headroom.cache.registry import CacheOptimizerRegistry
  opt = CacheOptimizerRegistry.get("openai")
prompt = "You are a code reviewer. Follow this style:\n def
handler(event):\n return process(event)\n - nested bullet\n\tif
ok:\n\t\treturn x"
res = opt.optimize([{"role": "system", "content": prompt}, {"role":
"user", "content": "hi"}], OptimizationContext(provider="openai",
model="gpt-4o"))
  print(repr(res.messages[0]["content"]))
  ```

- Observed result: indentation and text are preserved after the fix.
Before and after output:

  Before (`main`):
  ```text
'You are a code reviewer. Follow this style:\n handler(event):\n
rocess(event)\n nested bullet\n f ok:\n turn x'
  ```
  After:
  ```text
'You are a code reviewer. Follow this style:\n def handler(event):\n
return process(event)\n - nested bullet\n\tif ok:\n\t\treturn x'
  ```
Indentation and text are preserved; the double space in `nested bullet`
and `return x` is still collapsed.

- Not tested: a live request through `HeadroomClient` to the OpenAI API
(no key used); the optimizer output above is exactly what that path
sends upstream. The Anthropic and Google optimizers were not touched.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, only in that indented
system-prompt lines are no longer truncated. Tab indentation is now kept
as tabs instead of being converted to spaces.
- Kill switch / disable path: `CacheConfig(normalize_whitespace=False)`
(unchanged)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`, it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Comments and docs: the existing comments already describe the intended
behavior ("Collapse multiple spaces in content (not indentation)"), so
no new comment or doc change was needed. The cache prefix hash for
prompts with indented lines will change once, since the normalized text
is now correct.

---------

Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:38:06 -07:00
chopratejas e4a41faa33 Fix all ruff lint and format errors for CI
- Fix E402: Move module-level imports to top of file
- Fix F401: Add noqa for availability check imports
- Fix F402: Rename loop variables shadowing imports
- Fix E722: Replace bare except with except Exception
- Fix B904: Add exception chaining (from e)
- Fix F811: Remove duplicate imports
- Fix B027: Add noqa for empty close() method
- Fix E741: Rename ambiguous variable l -> label
- Fix I001: Import sorting issues
- Apply ruff format to all 106 files

All 902 tests pass.
2026-01-10 15:33:44 -08:00
chopratejas 7a05808e0f Add cache optimization module with scalable dynamic content detection
Implements a comprehensive cache optimization layer for LLM providers:

- Provider-specific optimizers (Anthropic, OpenAI, Google) with distinct
  caching strategies: explicit breakpoints, prefix stabilization, and
  CachedContent API respectively

- Scalable dynamic content detector using three strategies:
  1. Structural detection: "Label: value" patterns (language-agnostic)
  2. Entropy-based detection: high-entropy strings (IDs, tokens, hashes)
  3. Universal patterns: ISO 8601, UUIDs, JWTs, hex hashes

- NO hardcoded locale-specific patterns (no month names, etc.)

- Semantic caching layer with LRU eviction and TTL support

- Plugin registry for provider selection and custom optimizers

- 131 tests, real-world benchmarks showing 20-55% compression at <0.3ms
2026-01-07 14:07:49 -08:00