Commit Graph
1 Commits
Author SHA1 Message Date
580cc83a9a fix(transforms): log the compression failures that silently become "no savings" (#4092)
## Description

Five places in the compression path caught every exception and returned
the input unchanged, with no log line. A compression bug therefore
looked exactly like content that did not compress. Each now logs a
WARNING that names the exception type and counts, and a DEBUG line that
names the exception types and code locations (`describe_exception`). No
level logs the exception text or a traceback, because compressor and
store errors can quote tool output or credentials. Return values do not
change.

Refs #4136

Depends on #4138 (`headroom/log_safety.py`); its files show in this diff
until it merges.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `compression_batches.compress_batch_with_router`: warn when
`router.compress` raises (the batch still passes through).
- `cross_turn_dedup.dedup_blocks`: warn on failure. The `"error": True`
stat it returned was never read.
- `lossless_compaction.compact_lossless`: warn when a fold raises, with
the kind and length. It still never raises.
- `kompress_compressor.store_kompress_in_ccr`: warn on a CCR store
failure. The output then has no retrieval marker.
- `smart_crusher` CCR mirror: a store write failure is now WARNING, not
DEBUG.
- Each WARNING carries `type(exc).__name__` plus counts or the hash, and
never the exception text. The detail is a separate DEBUG line built with
`describe_exception(e)`, which gives types and code locations and never
the message or traceback (unless the operator sets
`HEADROOM_DEBUG_DUMP=full`). It only runs when DEBUG is enabled.
- New `tests/test_transforms_failure_logging.py`: one parametrized test
over all five paths. It checks pass-through, one WARNING and one DEBUG
detail, and formats every captured record with `logging.Formatter` to
assert a content canary and a credential canary (raised in the exception
and its cause) are absent. All five fail on main, and all five fail on
the earlier `exc_info=True` version of this PR. A second parametrized
test checks `describe_exception` is not called when DEBUG is off. A
third runs the real `compact_lossless` fallback with DEBUG on and an
`OSError` whose errno is out of the C int range (`2**31`, and
`10**10000` built at runtime), with the integer digit limit on and off.
It checks the original content comes back and the DEBUG line stays
bounded.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ pytest tests/test_transforms_failure_logging.py   # with main's product code
5 failed
$ pytest tests/test_transforms_failure_logging.py tests/test_lossless_mode.py tests/test_cross_turn_dedup.py tests/test_compression_batches.py tests/test_kompress_marker_gate.py tests/test_garbled_compression_fixes.py tests/test_ccr.py tests/test_lossless_excluded_compaction.py tests/test_lossless_first_dispatch.py tests/test_bash_search_lossless_fold.py tests/test_lossless_timestamp_guard.py tests/test_remote_kompress_dropin.py tests/test_ccr_marker_policy.py tests/test_responses_cross_turn_dedup.py tests/test_proxy_handlers_batch.py
258 passed in 24.04s
$ ruff check / ruff format (0.16.8): All checks passed;  mypy (5 modules): Success
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo `.venv`, this branch's worktree
with the prebuilt `headroom._core`. Temp `HOME`,
`HEADROOM_WORKSPACE_DIR` and `CODEX_HOME`; nothing touched the real
home.
- Exact command / steps: (1) shared smoke harness: a real proxy from
this checkout against a local fake upstream, with one Anthropic and one
OpenAI request carrying a compressible tool result (`smoke.sh
<worktree>`); (2) forced failure: `compact_lossless` with
`collapse_runs` patched to raise an error whose text contains a fake
token, logging at INFO.
- Observed result: The proxy serves both requests (200) with savings.
The forced failure returns the content unchanged and logs one WARNING
naming RuntimeError, without the token text. Output below.
- Not tested: A live proxy request that hits one of the five failure
paths: they need an injected bug, which the direct check and the unit
tests supply.

```text
branch: cleanup/transforms-log-silent-failures @ f2557723
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines ($SMOKE/ws/logs/proxy-64662.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: $SMOKE)
```

```text
$ HOME=$T/home HEADROOM_WORKSPACE_DIR=$T/ws CODEX_HOME=$T/codex python -c "<compact_lossless with collapse_runs patched to raise RuntimeError('token sk-live-123 in tool output')>"
headroom.transforms.lossless_compaction - WARNING - Lossless compaction (log) failed (RuntimeError) on 205 chars; leaving content as-is
unchanged: True
```

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Not applicable; no rollout-managed behaviour
is touched.
- Stable/default behavior changed: Only log output. Five failure paths
now log at WARNING; one of them was DEBUG.
- Kill switch / disable path: Not applicable; the change only adds log
lines.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this single commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

Documentation changes are not in this PR. They are recorded in the
cleanup docs ledger for the single follow-up docs PR.

The image tile-optimizer item from the audit was dropped. A
non-decodable image makes PIL raise `UnidentifiedImageError` there, so a
warning would fire on ordinary bad input. There is no once-guard. Each
warning fires only when a compressor raises or the CCR store fails, the
same idiom as the existing CCR write-through warnings.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-16 14:30:13 -07:00