mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description This removes proxy handler helpers that production code never calls, and one test that always skipped. Behaviour does not change. Refs #4136 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: removed `_compact_openai_tool_schema_value`. It had no callers anywhere. - `headroom/proxy/handlers/streaming.py`: - Removed `StreamingMixin._parse_sse_usage`, the 92-line single-chunk SSE usage parser. Production only calls `_parse_sse_usage_from_buffer`. - Removed `_sse_contains_error_event`, a 5-line wrapper over `_sse_event_outcome` that only `tests/test_route_advice.py` called. - `headroom/proxy/handlers/gemini.py`: removed `GeminiHandlerMixin._count_texts_offloaded`, which only tests called. `tests/test_tokenizer_count_offload.py:77` already asserts that no handler uses it. - Test changes: - `tests/test_proxy_cache_ttl_metrics.py`: the TTL-bucket test now drives `_parse_sse_usage_from_buffer`. - `tests/test_streaming_sse_malformed_events.py`: dropped the chunk-parser variants. The buffered-parser variants of the same cases stay. - `tests/test_route_advice.py`: carries the error-event helper locally, so its `_sse_event_outcome` coverage is unchanged. - `tests/test_tokenizer_count_offload.py`: calls `token_counting.count_texts_offloaded(proxy, ...)` directly, still on a real proxy and its worker thread. - `tests/test_codex_ws_compression_scheduler.py`: removed `test_codex_ws_emits_perf_log_with_cache_keys`, which called `pytest.skip` unconditionally ("Pending: full WS lifecycle harness"), along with the fixtures only it used. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.16.8 check headroom/proxy/handlers tests/test_codex_ws_compression_scheduler.py tests/test_tokenizer_count_offload.py tests/test_route_advice.py tests/test_streaming_sse_malformed_events.py tests/test_proxy_cache_ttl_metrics.py All checks passed! $ pytest -q tests/test_streaming_sse_malformed_events.py tests/test_proxy_cache_ttl_metrics.py tests/test_route_advice.py tests/test_tokenizer_count_offload.py tests/test_codex_ws_compression_scheduler.py tests/test_openai_responses_context_compaction.py tests/test_openai_responses_additional_tools.py tests/test_gemini_compression_offload.py tests/test_proxy_gemini_integration.py tests/test_proxy_gemini_native_integration.py 470 passed, 27 skipped in 24.13s ``` ## Real Behavior Proof - Environment: macOS, Python 3.12, repo venv, branch `cleanup/proxy-handlers-dead-helpers` @c15e679e, temp `HOME`/`HEADROOM_WORKSPACE_DIR`/`CODEX_HOME`, `HEADROOM_OFFLINE=1` - Exact command / steps: (1) a `grep -rnE '_compact_openai_tool_schema_value|_parse_sse_usage\b|_sse_contains_error_event|_count_texts_offloaded'` over headroom, tests, crates, plugins, sdk, examples, benchmarks, scripts, e2e, docs, wiki, .github, pyproject.toml and every consumer repo clone; (2) the shared smoke harness: a real `headroom proxy` from this checkout against a local fake upstream, with one Anthropic and one OpenAI turn; (3) a direct check that the removed names are gone from the assembled `HeadroomProxy` and that the live SSE usage parser still works - Observed result: (1) before the change, only the definitions and tests matched; after it, nothing in `headroom/` references the removed names. (2) and (3) are shown below: both turns return 200 with savings, the names are absent, and the live parser returns the usage. - Not tested: a real hosted upstream ```text $ bash smoke.sh <worktree> branch: cleanup/proxy-handlers-dead-helpers @c15e679ehealth: 200 POST /v1/messages -> HTTP200 smoke-ok POST /v1/chat/completions -> HTTP200 smoke-ok stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82 proxy.log WARNING/ERROR lines: (only the pre-existing offline-mode notice and Kompress offline-prefetch warnings) $ HOME=$T/h HEADROOM_WORKSPACE_DIR=$T/ws CODEX_HOME=$T/c python -c "<check HeadroomProxy attrs; run _parse_sse_usage_from_buffer>" _parse_sse_usage on HeadroomProxy: False _sse_contains_error_event on HeadroomProxy: False _count_texts_offloaded on HeadroomProxy: False _compact_openai_tool_schema_value in openai: False live SSE usage parser: {'input_tokens': 9, 'output_tokens': 2, 'cache_read_input_tokens': 0} ``` ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: stable - Stable/default behavior changed: no - Kill switch / disable path: not applicable (dead code removal) - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes Part of maintainer-requested cleanup #4136 (dead code, duplication, debuggability), split into small PRs for review. - `headroom.proxy.token_counting.count_texts_offloaded` is now called only by tests. It is outside this cleanup slice, so it is left in place and flagged for the token-counting owner. - No `headroom.proxy.handlers` symbol appears in the consumer import index. - mypy and comments: N/A, since this PR only deletes code. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
194 lines
8.5 KiB
Python
194 lines
8.5 KiB
Python
"""P2 — Codex compression scheduler regression coverage.
|
||
|
||
The pre-fix code throttled all concurrent Codex WS compression units
|
||
through a process-global ``threading.BoundedSemaphore(10)`` and created
|
||
a fresh ``ThreadPoolExecutor`` per frame. Under realistic concurrent
|
||
load (≥10 sessions) the semaphore saturated, ``elapsed_ms`` was measured
|
||
INCLUDING the wait time, and frames hit the parent 30s timeout.
|
||
|
||
The fix:
|
||
|
||
* Deletes the module-global ``_CODEX_WS_UNIT_ROUTER_SEMAPHORE``.
|
||
* Deletes the per-call inner ``ThreadPoolExecutor``.
|
||
* Processes routed units serially inside the frame-level worker thread
|
||
(``self._compression_executor`` already provides frame-level parallelism
|
||
via the proxy-wide bounded executor).
|
||
* Adds a ``PERF`` log emission from ``handle_openai_responses_ws`` so
|
||
Codex traffic is no longer invisible to ``headroom perf``.
|
||
|
||
These tests verify that future contributors cannot silently re-introduce
|
||
either bottleneck.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import concurrent.futures
|
||
import sys
|
||
import time
|
||
from pathlib import Path
|
||
|
||
import pytest
|
||
|
||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||
OPENAI_HANDLER = REPO_ROOT / "headroom" / "proxy" / "handlers" / "openai.py"
|
||
|
||
|
||
# ── Source-level regression guards ──────────────────────────────────────
|
||
|
||
|
||
def test_module_global_unit_semaphore_is_removed() -> None:
|
||
"""The 10-slot global semaphore that caused 30s frame timeouts must stay gone.
|
||
|
||
Read the source file directly — imported module state is not authoritative
|
||
because Python caches bytecode independently. The regression we are
|
||
guarding against is "someone reintroduces a module-level semaphore on
|
||
the Codex WS dispatch path" — that is detectable in source.
|
||
"""
|
||
source = OPENAI_HANDLER.read_text()
|
||
assert "_CODEX_WS_UNIT_ROUTER_SEMAPHORE" not in source, (
|
||
"Module-global semaphore on Codex WS path reintroduced. The P2 fix "
|
||
"deleted it because it saturated at 10 concurrent units and caused "
|
||
"the production cascade documented in issue #327's sibling slowness "
|
||
"report. Use `self._compression_executor` (the proxy-wide bounded "
|
||
"pool) for any new concurrency needs."
|
||
)
|
||
assert "_CODEX_WS_UNIT_ROUTER_MAX_WORKERS" not in source, (
|
||
"Module-global slot count for the (deleted) Codex unit semaphore reintroduced."
|
||
)
|
||
assert "_codex_ws_unit_worker_count" not in source, (
|
||
"The per-call inner-pool worker-count helper was deleted because the "
|
||
"inner pool was deleted. Reintroducing it suggests the inner pool "
|
||
"is back too — re-read docs/superpowers/specs/P2-codex-scheduler-fix.md."
|
||
)
|
||
assert "HEADROOM_CODEX_WS_UNIT_WORKERS" not in source, (
|
||
"The HEADROOM_CODEX_WS_UNIT_WORKERS env knob was removed. It only "
|
||
"existed to tune around the semaphore bottleneck, which is gone."
|
||
)
|
||
|
||
|
||
def test_no_per_call_threadpool_inside_compress_routed_units() -> None:
|
||
"""The inner ``ThreadPoolExecutor`` created per frame must stay gone.
|
||
|
||
Pre-fix, every call to ``_compress_openai_responses_payload`` created
|
||
and tore down a ``ThreadPoolExecutor(max_workers=worker_count)`` to run
|
||
routed units, layered on top of ``self._compression_executor``. That
|
||
pool-on-pool pattern added latency variance, fought for OS threads,
|
||
and made the global semaphore the binding constraint.
|
||
|
||
The exact phrase ``concurrent.futures.ThreadPoolExecutor`` should not
|
||
appear anywhere in openai.py — the dispatch uses the proxy's shared
|
||
bounded executor instead.
|
||
"""
|
||
source = OPENAI_HANDLER.read_text()
|
||
assert "concurrent.futures.ThreadPoolExecutor" not in source, (
|
||
"Per-call ThreadPoolExecutor reintroduced in handlers/openai.py. "
|
||
"Submit work to `self._compression_executor` (instrumented and "
|
||
"lifecycle-managed) instead of creating a new pool per frame."
|
||
)
|
||
|
||
|
||
# ── Concurrency stress (Tier 2) ─────────────────────────────────────────
|
||
#
|
||
# The smoking gun: with the old code, 30 concurrent calls to
|
||
# ``_compress_openai_responses_payload`` produced p99 per-call latency of
|
||
# ~2.4s on a 12-CPU machine because of the 10-slot global semaphore. After
|
||
# the fix, units run serially within the frame-level worker, but the
|
||
# frame-level compression executor lets 30 frames run in parallel without contention.
|
||
#
|
||
# Pass criteria mirror docs/superpowers/specs/P2-codex-scheduler-fix.md
|
||
# "Success criteria":
|
||
# - p99 per-frame < 250ms (vs baseline 2433ms)
|
||
# - p99/p50 < 3× (vs baseline 24×)
|
||
# - errors == 0
|
||
|
||
|
||
@pytest.mark.slow
|
||
def test_concurrent_compression_has_no_semaphore_tail() -> None:
|
||
"""Probe the 10-slot semaphore boundary with uniform-size workload.
|
||
|
||
Design notes — addresses a CI-vs-dev hardware skew that bit the
|
||
first iteration of this test:
|
||
|
||
* **12 concurrent sessions** (> the deleted 10-slot semaphore size).
|
||
Enough to saturate the gate if it ever reappears; small enough
|
||
that a 2-vCPU CI runner doesn't drown in OS-level scheduler
|
||
noise.
|
||
* **All frames the same size (4 KB)** so size-induced compute
|
||
variance cancels out. Pre-refactor the bug produced bimodal
|
||
latency (waiters vs holders) regardless of frame size; this
|
||
test must measure THAT, not size variance.
|
||
* **5 frames per session** = 60 total. Enough samples to make
|
||
the p99 statistic meaningful. Bounded runtime even on slow CI.
|
||
* **Threshold ratio < 4×.** On uniform-size workload the only
|
||
sources of p99/p50 spread are (a) the deleted semaphore tail
|
||
(≈27×) or (b) OS-level scheduler noise (≈2–3×). 4× sits
|
||
comfortably between the two — catches the bug, tolerates
|
||
hardware. (First iteration tried 5× with mixed sizes, which
|
||
let size-variance push CI ratios to 7.4×.) The ratio is only
|
||
enforced once p99 clears a scheduler-noise floor — on very fast
|
||
runners p50 rounds to 0ms and the ratio becomes pure jitter.
|
||
|
||
Marked ``slow`` so a normal ``pytest`` run can skip it via
|
||
``-m 'not slow'``. CI matrix runs all marks.
|
||
"""
|
||
sys.path.insert(0, str(REPO_ROOT))
|
||
from scripts.replay_codex_ws_load import ( # noqa: E402
|
||
Frame,
|
||
Scenario,
|
||
boot_proxy,
|
||
replay_session,
|
||
warmup,
|
||
)
|
||
|
||
proxy = boot_proxy()
|
||
warmup_ms = warmup(proxy)
|
||
assert warmup_ms < 30_000, (
|
||
f"Warmup took {warmup_ms:.0f}ms — Kompress model failed to load? "
|
||
"Subsequent timing assertions are meaningless without a warm router."
|
||
)
|
||
|
||
# 12 sessions × 5 frames = 60 total. Uniform 4 KB plain-text
|
||
# payload — each frame's compute time should be identical modulo
|
||
# scheduler noise.
|
||
UNIFORM_FRAME = Frame(bytes_estimate=4096, text_shape="plain_text_like")
|
||
scenarios = [
|
||
Scenario(
|
||
request_id=f"stress-{i:02d}",
|
||
frames=[UNIFORM_FRAME] * 5,
|
||
)
|
||
for i in range(12)
|
||
]
|
||
|
||
results: list = []
|
||
started = time.perf_counter()
|
||
with concurrent.futures.ThreadPoolExecutor(max_workers=12) as pool:
|
||
futures = [pool.submit(replay_session, proxy, s, "gpt-4o-mini") for s in scenarios]
|
||
for fut in concurrent.futures.as_completed(futures):
|
||
results.extend(fut.result())
|
||
wall_s = time.perf_counter() - started
|
||
|
||
elapsed = sorted(r.elapsed_ms for r in results)
|
||
p50 = elapsed[len(elapsed) // 2]
|
||
p99 = elapsed[int(len(elapsed) * 0.99)]
|
||
errors = [r for r in results if r.error]
|
||
|
||
# Always print the distribution so CI logs show numbers for
|
||
# diagnosing failures and tracking drift across runs.
|
||
print(
|
||
f"\n[stress] frames={len(results)} wall={wall_s:.2f}s "
|
||
f"p50={p50:.0f}ms p99={p99:.0f}ms ratio={p99 / max(p50, 1):.2f}× errors={len(errors)}"
|
||
)
|
||
|
||
assert not errors, f"Got {len(errors)} errors; first: {errors[0].error}"
|
||
SEMAPHORE_P99_CEILING_MS = 1_000.0
|
||
assert p99 < SEMAPHORE_P99_CEILING_MS, (
|
||
f"p99 is {p99:.0f}ms; expected < {SEMAPHORE_P99_CEILING_MS:.0f}ms on "
|
||
"uniform-size workload. The pre-fix semaphore baseline was ~2433ms."
|
||
)
|
||
# Do not add a p99/p50 wall-clock ratio here. A hosted runner can park one
|
||
# worker independently of this code path, making an otherwise healthy
|
||
# 2ms/76ms distribution look like a 35x contention tail. The property is
|
||
# covered structurally above (the semaphore and nested executor must stay
|
||
# absent), while this absolute ceiling still rejects the measured 2433ms
|
||
# pre-fix behavior without pretending scheduler jitter is product state.
|