Files
headroom/tests/test_codex_ws_compression_scheduler.py
Tejas Chopra 31d378b0e5 refactor(proxy): remove test-only and dead helpers from the handlers (#4117)
## Description

This removes proxy handler helpers that production code never calls, and
one test that always skipped. Behaviour does not change.

Refs #4136

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/openai.py`: removed
`_compact_openai_tool_schema_value`. It had no callers anywhere.
- `headroom/proxy/handlers/streaming.py`:
- Removed `StreamingMixin._parse_sse_usage`, the 92-line single-chunk
SSE usage parser. Production only calls `_parse_sse_usage_from_buffer`.
- Removed `_sse_contains_error_event`, a 5-line wrapper over
`_sse_event_outcome` that only `tests/test_route_advice.py` called.
- `headroom/proxy/handlers/gemini.py`: removed
`GeminiHandlerMixin._count_texts_offloaded`, which only tests called.
`tests/test_tokenizer_count_offload.py:77` already asserts that no
handler uses it.
- Test changes:
- `tests/test_proxy_cache_ttl_metrics.py`: the TTL-bucket test now
drives `_parse_sse_usage_from_buffer`.
- `tests/test_streaming_sse_malformed_events.py`: dropped the
chunk-parser variants. The buffered-parser variants of the same cases
stay.
- `tests/test_route_advice.py`: carries the error-event helper locally,
so its `_sse_event_outcome` coverage is unchanged.
- `tests/test_tokenizer_count_offload.py`: calls
`token_counting.count_texts_offloaded(proxy, ...)` directly, still on a
real proxy and its worker thread.
- `tests/test_codex_ws_compression_scheduler.py`: removed
`test_codex_ws_emits_perf_log_with_cache_keys`, which called
`pytest.skip` unconditionally ("Pending: full WS lifecycle harness"),
along with the fixtures only it used.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ uvx ruff@0.16.8 check headroom/proxy/handlers tests/test_codex_ws_compression_scheduler.py tests/test_tokenizer_count_offload.py tests/test_route_advice.py tests/test_streaming_sse_malformed_events.py tests/test_proxy_cache_ttl_metrics.py
All checks passed!
$ pytest -q tests/test_streaming_sse_malformed_events.py tests/test_proxy_cache_ttl_metrics.py tests/test_route_advice.py tests/test_tokenizer_count_offload.py tests/test_codex_ws_compression_scheduler.py tests/test_openai_responses_context_compaction.py tests/test_openai_responses_additional_tools.py tests/test_gemini_compression_offload.py tests/test_proxy_gemini_integration.py tests/test_proxy_gemini_native_integration.py
470 passed, 27 skipped in 24.13s
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo venv, branch
`cleanup/proxy-handlers-dead-helpers` @ c15e679e, temp
`HOME`/`HEADROOM_WORKSPACE_DIR`/`CODEX_HOME`, `HEADROOM_OFFLINE=1`
- Exact command / steps: (1) a `grep -rnE
'_compact_openai_tool_schema_value|_parse_sse_usage\b|_sse_contains_error_event|_count_texts_offloaded'`
over headroom, tests, crates, plugins, sdk, examples, benchmarks,
scripts, e2e, docs, wiki, .github, pyproject.toml and every consumer
repo clone; (2) the shared smoke harness: a real `headroom proxy` from
this checkout against a local fake upstream, with one Anthropic and one
OpenAI turn; (3) a direct check that the removed names are gone from the
assembled `HeadroomProxy` and that the live SSE usage parser still works
- Observed result: (1) before the change, only the definitions and tests
matched; after it, nothing in `headroom/` references the removed names.
(2) and (3) are shown below: both turns return 200 with savings, the
names are absent, and the live parser returns the usage.
- Not tested: a real hosted upstream

```text
$ bash smoke.sh <worktree>
branch: cleanup/proxy-handlers-dead-helpers @ c15e679e
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines:
   (only the pre-existing offline-mode notice and Kompress offline-prefetch warnings)

$ HOME=$T/h HEADROOM_WORKSPACE_DIR=$T/ws CODEX_HOME=$T/c python -c "<check HeadroomProxy attrs; run _parse_sse_usage_from_buffer>"
_parse_sse_usage on HeadroomProxy: False
_sse_contains_error_event on HeadroomProxy: False
_count_texts_offloaded on HeadroomProxy: False
_compact_openai_tool_schema_value in openai: False
live SSE usage parser: {'input_tokens': 9, 'output_tokens': 2, 'cache_read_input_tokens': 0}
```

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: stable
- Stable/default behavior changed: no
- Kill switch / disable path: not applicable (dead code removal)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

- `headroom.proxy.token_counting.count_texts_offloaded` is now called
only by tests. It is outside this cleanup slice, so it is left in place
and flagged for the token-counting owner.
- No `headroom.proxy.handlers` symbol appears in the consumer import
index.
- mypy and comments: N/A, since this PR only deletes code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-10-10 09:00:49 -07:00

194 lines
8.5 KiB
Python
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""P2 — Codex compression scheduler regression coverage.
The pre-fix code throttled all concurrent Codex WS compression units
through a process-global ``threading.BoundedSemaphore(10)`` and created
a fresh ``ThreadPoolExecutor`` per frame. Under realistic concurrent
load (≥10 sessions) the semaphore saturated, ``elapsed_ms`` was measured
INCLUDING the wait time, and frames hit the parent 30s timeout.
The fix:
* Deletes the module-global ``_CODEX_WS_UNIT_ROUTER_SEMAPHORE``.
* Deletes the per-call inner ``ThreadPoolExecutor``.
* Processes routed units serially inside the frame-level worker thread
(``self._compression_executor`` already provides frame-level parallelism
via the proxy-wide bounded executor).
* Adds a ``PERF`` log emission from ``handle_openai_responses_ws`` so
Codex traffic is no longer invisible to ``headroom perf``.
These tests verify that future contributors cannot silently re-introduce
either bottleneck.
"""
from __future__ import annotations
import concurrent.futures
import sys
import time
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
OPENAI_HANDLER = REPO_ROOT / "headroom" / "proxy" / "handlers" / "openai.py"
# ── Source-level regression guards ──────────────────────────────────────
def test_module_global_unit_semaphore_is_removed() -> None:
"""The 10-slot global semaphore that caused 30s frame timeouts must stay gone.
Read the source file directly — imported module state is not authoritative
because Python caches bytecode independently. The regression we are
guarding against is "someone reintroduces a module-level semaphore on
the Codex WS dispatch path" — that is detectable in source.
"""
source = OPENAI_HANDLER.read_text()
assert "_CODEX_WS_UNIT_ROUTER_SEMAPHORE" not in source, (
"Module-global semaphore on Codex WS path reintroduced. The P2 fix "
"deleted it because it saturated at 10 concurrent units and caused "
"the production cascade documented in issue #327's sibling slowness "
"report. Use `self._compression_executor` (the proxy-wide bounded "
"pool) for any new concurrency needs."
)
assert "_CODEX_WS_UNIT_ROUTER_MAX_WORKERS" not in source, (
"Module-global slot count for the (deleted) Codex unit semaphore reintroduced."
)
assert "_codex_ws_unit_worker_count" not in source, (
"The per-call inner-pool worker-count helper was deleted because the "
"inner pool was deleted. Reintroducing it suggests the inner pool "
"is back too — re-read docs/superpowers/specs/P2-codex-scheduler-fix.md."
)
assert "HEADROOM_CODEX_WS_UNIT_WORKERS" not in source, (
"The HEADROOM_CODEX_WS_UNIT_WORKERS env knob was removed. It only "
"existed to tune around the semaphore bottleneck, which is gone."
)
def test_no_per_call_threadpool_inside_compress_routed_units() -> None:
"""The inner ``ThreadPoolExecutor`` created per frame must stay gone.
Pre-fix, every call to ``_compress_openai_responses_payload`` created
and tore down a ``ThreadPoolExecutor(max_workers=worker_count)`` to run
routed units, layered on top of ``self._compression_executor``. That
pool-on-pool pattern added latency variance, fought for OS threads,
and made the global semaphore the binding constraint.
The exact phrase ``concurrent.futures.ThreadPoolExecutor`` should not
appear anywhere in openai.py — the dispatch uses the proxy's shared
bounded executor instead.
"""
source = OPENAI_HANDLER.read_text()
assert "concurrent.futures.ThreadPoolExecutor" not in source, (
"Per-call ThreadPoolExecutor reintroduced in handlers/openai.py. "
"Submit work to `self._compression_executor` (instrumented and "
"lifecycle-managed) instead of creating a new pool per frame."
)
# ── Concurrency stress (Tier 2) ─────────────────────────────────────────
#
# The smoking gun: with the old code, 30 concurrent calls to
# ``_compress_openai_responses_payload`` produced p99 per-call latency of
# ~2.4s on a 12-CPU machine because of the 10-slot global semaphore. After
# the fix, units run serially within the frame-level worker, but the
# frame-level compression executor lets 30 frames run in parallel without contention.
#
# Pass criteria mirror docs/superpowers/specs/P2-codex-scheduler-fix.md
# "Success criteria":
# - p99 per-frame < 250ms (vs baseline 2433ms)
# - p99/p50 < 3× (vs baseline 24×)
# - errors == 0
@pytest.mark.slow
def test_concurrent_compression_has_no_semaphore_tail() -> None:
"""Probe the 10-slot semaphore boundary with uniform-size workload.
Design notes — addresses a CI-vs-dev hardware skew that bit the
first iteration of this test:
* **12 concurrent sessions** (> the deleted 10-slot semaphore size).
Enough to saturate the gate if it ever reappears; small enough
that a 2-vCPU CI runner doesn't drown in OS-level scheduler
noise.
* **All frames the same size (4 KB)** so size-induced compute
variance cancels out. Pre-refactor the bug produced bimodal
latency (waiters vs holders) regardless of frame size; this
test must measure THAT, not size variance.
* **5 frames per session** = 60 total. Enough samples to make
the p99 statistic meaningful. Bounded runtime even on slow CI.
* **Threshold ratio < 4×.** On uniform-size workload the only
sources of p99/p50 spread are (a) the deleted semaphore tail
(≈27×) or (b) OS-level scheduler noise (≈2–3×). 4× sits
comfortably between the two — catches the bug, tolerates
hardware. (First iteration tried 5× with mixed sizes, which
let size-variance push CI ratios to 7.4×.) The ratio is only
enforced once p99 clears a scheduler-noise floor — on very fast
runners p50 rounds to 0ms and the ratio becomes pure jitter.
Marked ``slow`` so a normal ``pytest`` run can skip it via
``-m 'not slow'``. CI matrix runs all marks.
"""
sys.path.insert(0, str(REPO_ROOT))
from scripts.replay_codex_ws_load import ( # noqa: E402
Frame,
Scenario,
boot_proxy,
replay_session,
warmup,
)
proxy = boot_proxy()
warmup_ms = warmup(proxy)
assert warmup_ms < 30_000, (
f"Warmup took {warmup_ms:.0f}ms — Kompress model failed to load? "
"Subsequent timing assertions are meaningless without a warm router."
)
# 12 sessions × 5 frames = 60 total. Uniform 4 KB plain-text
# payload — each frame's compute time should be identical modulo
# scheduler noise.
UNIFORM_FRAME = Frame(bytes_estimate=4096, text_shape="plain_text_like")
scenarios = [
Scenario(
request_id=f"stress-{i:02d}",
frames=[UNIFORM_FRAME] * 5,
)
for i in range(12)
]
results: list = []
started = time.perf_counter()
with concurrent.futures.ThreadPoolExecutor(max_workers=12) as pool:
futures = [pool.submit(replay_session, proxy, s, "gpt-4o-mini") for s in scenarios]
for fut in concurrent.futures.as_completed(futures):
results.extend(fut.result())
wall_s = time.perf_counter() - started
elapsed = sorted(r.elapsed_ms for r in results)
p50 = elapsed[len(elapsed) // 2]
p99 = elapsed[int(len(elapsed) * 0.99)]
errors = [r for r in results if r.error]
# Always print the distribution so CI logs show numbers for
# diagnosing failures and tracking drift across runs.
print(
f"\n[stress] frames={len(results)} wall={wall_s:.2f}s "
f"p50={p50:.0f}ms p99={p99:.0f}ms ratio={p99 / max(p50, 1):.2f}× errors={len(errors)}"
)
assert not errors, f"Got {len(errors)} errors; first: {errors[0].error}"
SEMAPHORE_P99_CEILING_MS = 1_000.0
assert p99 < SEMAPHORE_P99_CEILING_MS, (
f"p99 is {p99:.0f}ms; expected < {SEMAPHORE_P99_CEILING_MS:.0f}ms on "
"uniform-size workload. The pre-fix semaphore baseline was ~2433ms."
)
# Do not add a p99/p50 wall-clock ratio here. A hosted runner can park one
# worker independently of this code path, making an otherwise healthy
# 2ms/76ms distribution look like a 35x contention tail. The property is
# covered structurally above (the semaphore and nested executor must stay
# absent), while this absolute ceiling still rejects the measured 2433ms
# pre-fix behavior without pretending scheduler jitter is product state.