fix(proxy): quarantine compression only once half the pool is stuck (#3932)

## Description

The timeout-debt quarantine in
`HeadroomProxy._run_compression_in_executor` refused every new
compression call while even one timed-out worker was still running. On
an auto-sized pool (one worker per CPU, 8 on a typical laptop) a single
straggler forwarded every other request uncompressed for up to the 60s
cap while 7 of 8 workers sat idle. A common trigger is the cold-start
fast pass missing its deliberately short 10s fail-open budget.

The quarantine exists so timed-out work cannot saturate the executor
(#2292, #2360). That risk starts near saturation, not at the first
straggler, so the check now engages only once timed-out workers hold
half the pool (`max(1, max_workers // 2)`). Pools of 1-3 workers keep
today's behaviour.

Closes #3931

Related, not closed by this PR: #3711 (what makes one Kompress stage
overrun its budget; this PR narrows how far one overrun spreads, not the
overrun itself) and #3653 (thread/fd reclamation for stuck workers; see
Additional Notes for why this change does not add to it).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py`: new `_compression_quarantine_threshold`,
defaulting to `max(1, compression_max_workers // 2)` and overridable
with `HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD` (read with the existing
`_get_env_int`, `min_value=1`; `=1` restores the old any-straggler
behaviour).
- The quarantine check and its 60s time-cap release both key on
`timed_out_in_flight >= threshold` instead of `> 0`. The cap is still
re-armed on every fresh timeout, as before.
- `quarantine_activations_total` (and the `activated` Prometheus event
and warning) now fire on the transition into the quarantined state, i.e.
when debt reaches the threshold, or when a fresh timeout re-arms it
after the time cap released it. Previously a re-arm after a release
while debt still stood was not counted.
- "Quarantine cleared" is logged when debt drops back below the
threshold, with the number of timed-out workers still running.
- `/health` `runtime.compression_executor.quarantine_active` now reports
the condition the check enforces (debt at or above the threshold and the
cap not lapsed). It used to report `timed_out_workers > 0`, which was
already wrong after a time-cap release.
- Tests: `test_quarantine_waits_for_half_the_pool_to_be_stuck` (8-worker
pool: 1 and 3 stuck still compress, 4 stuck refuses, `/health` matches,
re-arm after release counts as a second activation) and
`test_quarantine_threshold_is_half_the_pool_unless_overridden` (default
sizing and the env override). Two tests in
`tests/test_tokenizer_count_offload.py` that forced a quarantine with a
debt of 1 on an auto-sized pool now set the debt to the threshold.

## Testing

- [x] Unit tests pass (`pytest`) - targeted files only, listed below;
the full suite was not run
- [x] Linting passes (`ruff check .`) - on the changed files, plus `ruff
format --check`
- [x] Type checking passes (`mypy headroom`) - `mypy
headroom/proxy/server.py`
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

Before the fix (new tests against unmodified `server.py`; the two
tokenizer tests fail only because they read the new attribute):

```text
$ python -m pytest tests/test_proxy_compression_executor.py tests/test_tokenizer_count_offload.py -q -p no:cacheprovider
FAILED tests/test_proxy_compression_executor.py::test_quarantine_waits_for_half_the_pool_to_be_stuck
FAILED tests/test_proxy_compression_executor.py::test_quarantine_threshold_is_half_the_pool_unless_overridden[8-None-4]
FAILED tests/test_proxy_compression_executor.py::test_quarantine_threshold_is_half_the_pool_unless_overridden[3-None-1]
FAILED tests/test_proxy_compression_executor.py::test_quarantine_threshold_is_half_the_pool_unless_overridden[1-None-1]
FAILED tests/test_proxy_compression_executor.py::test_quarantine_threshold_is_half_the_pool_unless_overridden[8-1-1]
FAILED tests/test_proxy_compression_executor.py::test_quarantine_threshold_is_half_the_pool_unless_overridden[8-0-4]
FAILED tests/test_tokenizer_count_offload.py::test_count_tokens_offloaded_fails_open_on_executor_quarantine
FAILED tests/test_tokenizer_count_offload.py::test_count_tokens_offloaded_returns_count_text_capable_tokenizer
=================== 8 failed, 20 passed, 1 warning in 20.73s ===================
```

The behavioural failure on main is:

```text
tests/test_proxy_compression_executor.py:336: in test_quarantine_waits_for_half_the_pool_to_be_stuck
E   headroom.proxy.server.CompressionQuarantinedError: compression quarantined: 1 timed-out worker(s) still running
```

After the fix:

```text
$ python -m pytest tests/test_proxy_compression_executor.py tests/test_tokenizer_count_offload.py tests/test_platform_stabilization_functional.py tests/test_prometheus_obs_counters.py -q -p no:cacheprovider
======================== 51 passed, 1 warning in 28.02s ========================

$ ruff check headroom/proxy/server.py tests/test_proxy_compression_executor.py tests/test_tokenizer_count_offload.py
All checks passed!
$ ruff format --check headroom/proxy/server.py tests/test_proxy_compression_executor.py tests/test_tokenizer_count_offload.py
3 files already formatted
$ mypy headroom/proxy/server.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.11, upstream main `d5318ac2`
(before) and the same commit plus this branch (after). Proxy booted
through `headroom.proxy.server.run_server(ProxyConfig(port=...,
compression_max_workers=8, disable_kompress=True, cache_enabled=False,
rate_limit_enabled=False, log_requests=False))`, which is what `headroom
proxy` calls after argument parsing (the local venv lacked the
`mcp`/`magika`/`onnxruntime` extras that the CLI's dependency preflight
requires). Env: `HEADROOM_SKIP_UPSTREAM_CHECK=1
HEADROOM_COMPRESSION_TIMEOUT_SECONDS=3 HEADROOM_TELEMETRY=off`, a fresh
`HEADROOM_WORKSPACE_DIR` per run.
- Exact command / steps: over HTTP, one warm-up `/v1/compress` (200-row
JSON `tool_result`, 10,623 tokens), one oversized `/v1/compress` (40,000
rows, about 22s of work against the 3s budget, so its worker keeps
running after the request fails open), then three small `/v1/compress`
calls while that worker is still running, reading `/health`
`runtime.compression_executor` in between. A second run fires four
oversized calls concurrently to strand 4 of 8 workers, sends a small
call, waits until one stuck worker exits, and sends another.
- Observed result: on main one straggler refuses every following
request; on this branch the other 7 workers keep compressing, and half
the pool stuck still quarantines and clears again (output below).

Before (main), one straggler refuses everything:

```text
health after big timed out: {'max_workers': 8, 'running': 1, 'timed_out_workers': 1, 'quarantine_active': True, 'quarantine_activations_total': 1, 'quarantine_skips_total': 0}
small request 1: 0.02s tokens_before=0 tokens_saved=0 skipped=True reason=compression_timeout
small request 2: 0.01s tokens_before=0 tokens_saved=0 skipped=True reason=compression_timeout
small request 3: 0.01s tokens_before=0 tokens_saved=0 skipped=True reason=compression_timeout
health after small requests: {'max_workers': 8, 'running': 1, 'timed_out_workers': 1, 'quarantine_active': True, 'quarantine_activations_total': 1, 'quarantine_skips_total': 3}
```

After, the same straggler leaves the other 7 workers serving:

```text
health after big timed out: {'max_workers': 8, 'running': 1, 'timed_out_workers': 1, 'quarantine_active': False, 'quarantine_activations_total': 0, 'quarantine_skips_total': 0}
small request 1: 0.12s tokens_before=10623 tokens_saved=4411 skipped=False reason=None
small request 2: 0.12s tokens_before=10623 tokens_saved=4411 skipped=False reason=None
small request 3: 0.18s tokens_before=10623 tokens_saved=4411 skipped=False reason=None
health after small requests: {'max_workers': 8, 'running': 1, 'timed_out_workers': 1, 'quarantine_active': False, 'quarantine_activations_total': 0, 'quarantine_skips_total': 0}
```

After, half the pool stuck still quarantines, and it clears when debt
drops below half:

```text
health after 4 concurrent big requests timed out: {'max_workers': 8, 'running': 4, 'timed_out_workers': 4, 'quarantine_active': True, 'quarantine_activations_total': 1, 'quarantine_skips_total': 0}
small request while 4 of 8 stuck: 0.10s tokens_before=0 tokens_saved=0 skipped=True reason=compression_timeout
health once a stuck worker exited: {'max_workers': 8, 'running': 3, 'timed_out_workers': 3, 'quarantine_active': False, 'quarantine_activations_total': 1, 'quarantine_skips_total': 1}
small request with 3 stuck: 0.19s tokens_before=10623 tokens_saved=4411 skipped=False reason=None
```

```text
WARNING - Compression worker exceeded its request deadline and is still running; new compression is quarantined until fewer than 4 timed-out workers remain
INFO - Compression quarantine cleared; 3 timed-out worker(s) still running
```

- Not tested: the `headroom proxy` CLI entry point itself (booted via
`run_server`, see Environment); the `/v1/messages`, `/v1/responses` and
Codex WS paths end to end (they share the same
`_run_compression_in_executor`, but I only drove `/v1/compress`);
Windows and Linux (the field evidence covers them, my repro does not);
the 60s time-cap release over HTTP (covered by
`test_compression_quarantine_releases_after_time_cap` and the new re-arm
assertion, not by a live run); a long soak sampling OS thread and fd
counts.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. The quarantine is not in
`rollout.py` FEATURES.
- Minimum rollout channel: n/a (applies on every channel).
- Stable/default behavior changed: yes. On pools of 4 or more workers,
compression keeps running while fewer than half the workers are held by
timed-out jobs; previously the first timed-out job refused all
compression for up to 60s. Pools of 1-3 workers are unchanged.
- Kill switch / disable path:
`HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD=1` restores the previous
behaviour on any pool size.
- Unsafe override required: no.
- Qualification impact: more requests get compressed while a straggler
runs, so `tokens_saved` rises and the `skipped` quarantine counter falls
on affected installs; no change to what compression does to a request it
runs on, and no change to provider cache behaviour beyond the requests
that are now compressed instead of forwarded raw.
- Rollback path: set `HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD=1`, or
revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` - it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Pool capacity and #3653. The compression executor is a
`ThreadPoolExecutor` with a fixed `max_workers`; it never creates
threads beyond that cap, and a stuck worker keeps the thread it already
holds. Admitting work while a straggler runs only puts that work on the
pool's idle threads, which finish and release their resources normally,
so this change cannot raise the process's thread count above
`max_workers` or add a leaked thread/fd per quarantine event. What it
does change is how many timed-out jobs can stand before new work is
refused: up to `threshold - 1` (3 on an 8-worker pool) instead of 0.
That bound is no looser than today's in practice: the old rule never
prevented requests that were already admitted from timing out together
(it only refuses calls that arrive after a timeout is recorded), and the
60s cap already lets work back in while debt stands. At `threshold`
stuck workers the quarantine engages exactly as before, so at least half
the pool stays free for work that is not stuck.

Field evidence that motivated this (Headroom Desktop fleet canary,
2026-09-21 to 2026-09-23): 9 hosts on macOS, Windows and Linux had
between 2.1% and 38.9% of requests refused compression by the
quarantine. On one machine's proxy logs, 211 of 262 refusals happened
with ONE stuck worker and none with more than 3 of 8 stuck; on 0.37.0
that machine forwarded 42 of 180 requests (23%) uncompressed in a day
while `/health` showed `max_workers` 8, `queued_max` 1 and
`queue_wait_seconds_max` 0.23, i.e. an idle pool.

Headroom Desktop has shipped an equivalent downstream patch (standing
the quarantine down below half the pool) since 2026-09-23 (desktop
0.9.22-rc.1); this PR implements it in the check itself so it can be
dropped there.

Documentation: `HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD` follows the
existing undocumented `HEADROOM_COMPRESSION_QUARANTINE_MAX_SECONDS`, so
no docs page was updated; happy to add both to the proxy env reference
if wanted.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3932"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
gglucass
2026-10-04 13:56:54 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JD Davis
parent 1132a54b21
commit be2b205c51
3 changed files with 139 additions and 30 deletions
+47 -26
View File
@@ -1276,11 +1276,19 @@ class HeadroomProxy(
# timeout. Stuck-thread leak indicator.
self._compression_leaked_threads: int = 0
# Timeout-debt quarantine. Python cannot preempt a worker after its
# asyncio waiter times out, so accepting more compression while that
# worker is still running can multiply one slow call into a saturated
# executor. New work raises immediately until every known post-timeout
# worker has genuinely exited, then follows the caller's existing
# compression-failure policy.
# asyncio waiter times out, so accepting more compression while those
# workers are still running can multiply slow calls into a saturated
# executor. Once half the pool is held by post-timeout workers, new work
# raises immediately until the debt drops below that again, and callers
# follow their existing compression-failure policy. One straggler on a
# mostly idle pool does not saturate it, so it does not refuse work;
# pools of 1-3 workers still quarantine on the first timeout.
# HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD=1 restores that everywhere.
self._compression_quarantine_threshold: int = _get_env_int(
"HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD",
max(1, _compression_max // 2),
min_value=1,
)
self._compression_timed_out_in_flight: int = 0
self._compression_timed_out_in_flight_max: int = 0
self._compression_quarantine_activations: int = 0
@@ -1531,10 +1539,11 @@ class HeadroomProxy(
worker keeps running to completion, ignored. We detect this by
marking the call timed out on the asyncio side and incrementing
``_compression_leaked_threads`` from the worker's ``finally``
block after it eventually finishes. While any such worker remains,
new calls raise :class:`CompressionQuarantinedError` immediately so
callers apply the existing compression-failure policy instead of
filling the rest of the pool with the same timeout debt. Jobs that are
block after it eventually finishes. While such workers hold half the
pool (``_compression_quarantine_threshold``), new calls raise
:class:`CompressionQuarantinedError` immediately so callers apply the
existing compression-failure policy instead of filling the rest of the
pool with the same timeout debt. Jobs that are
successfully cancelled before a worker starts are removed from the
queued gauge and do not activate the quarantine. If cancellation races
with worker startup, the now-running job is tracked as timeout debt and
@@ -1556,20 +1565,19 @@ class HeadroomProxy(
``asyncio.TimeoutError`` if the callable doesn't return within
``timeout``. Any exception raised by ``fn`` propagates
unchanged. :class:`CompressionQuarantinedError` (an
``asyncio.TimeoutError`` subclass) if a prior timed-out worker is
still running.
``asyncio.TimeoutError`` subclass) if prior timed-out workers still
hold half the pool.
"""
now = time.monotonic()
with self._compression_metrics_lock:
timed_out_in_flight = self._compression_timed_out_in_flight
quarantined = timed_out_in_flight > 0 and now < self._compression_quarantine_deadline
saturating = timed_out_in_flight >= self._compression_quarantine_threshold
quarantined = saturating and now < self._compression_quarantine_deadline
# Debt outlived the cap: presume the worker leaked/hung and stop
# blocking on it. Count the release once per lapse (while debt stands
# and the deadline has passed) so operators can see it happened.
released = (
timed_out_in_flight > 0
and not quarantined
and self._compression_quarantine_deadline > 0.0
saturating and not quarantined and self._compression_quarantine_deadline > 0.0
)
if quarantined:
self._compression_quarantine_skips += 1
@@ -1613,7 +1621,7 @@ class HeadroomProxy(
"""Record a still-running post-timeout worker; lock must be held.
Returns ``True`` only when this worker transitions the executor
from clear to quarantined.
from clear (or released by the time cap) to quarantined.
"""
if (
not state["timed_out"]
@@ -1622,7 +1630,12 @@ class HeadroomProxy(
or state["timeout_debt_recorded"]
):
return False
was_clear = self._compression_timed_out_in_flight == 0
now = time.monotonic()
threshold = self._compression_quarantine_threshold
was_quarantined = (
self._compression_timed_out_in_flight >= threshold
and now < self._compression_quarantine_deadline
)
self._compression_timed_out_in_flight += 1
self._compression_timed_out_in_flight_max = max(
self._compression_timed_out_in_flight_max,
@@ -1631,19 +1644,19 @@ class HeadroomProxy(
# (Re)arm the quarantine time cap on every fresh timeout, so ongoing
# slowness keeps quarantining while a single leaked worker cannot
# hold it past the cap (#2360).
self._compression_quarantine_deadline = (
time.monotonic() + self._compression_quarantine_max_seconds
)
self._compression_quarantine_deadline = now + self._compression_quarantine_max_seconds
state["timeout_debt_recorded"] = True
if was_clear:
activated = not was_quarantined and self._compression_timed_out_in_flight >= threshold
if activated:
self._compression_quarantine_activations += 1
return was_clear
return activated
def _announce_quarantine() -> None:
self.metrics.record_compression_quarantine("activated")
logger.warning(
"Compression worker exceeded its request deadline and is still running; "
"new compression is quarantined until timed-out workers exit"
"new compression is quarantined until fewer than %d timed-out workers remain",
self._compression_quarantine_threshold,
)
def _wrapped(): # noqa: ANN202
@@ -1678,11 +1691,15 @@ class HeadroomProxy(
if state["timeout_debt_recorded"]:
self._compression_timed_out_in_flight -= 1
state["timeout_debt_recorded"] = False
quarantine_cleared = self._compression_timed_out_in_flight == 0
remaining = self._compression_timed_out_in_flight
quarantine_cleared = remaining == self._compression_quarantine_threshold - 1
else:
quarantine_cleared = False
if quarantine_cleared:
logger.info("Compression quarantine cleared after all timed-out workers exited")
logger.info(
"Compression quarantine cleared; %d timed-out worker(s) still running",
remaining,
)
future = loop.run_in_executor(self._compression_executor, _wrapped)
try:
@@ -3542,6 +3559,10 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
_comp_run_max = proxy._compression_run_seconds_max
_comp_leaked = proxy._compression_leaked_threads
_comp_timed_out_in_flight = proxy._compression_timed_out_in_flight
_comp_quarantine_active = (
_comp_timed_out_in_flight >= proxy._compression_quarantine_threshold
and time.monotonic() < proxy._compression_quarantine_deadline
)
_comp_timed_out_in_flight_max = proxy._compression_timed_out_in_flight_max
_comp_quarantine_activations = proxy._compression_quarantine_activations
_comp_quarantine_skips = proxy._compression_quarantine_skips
@@ -3572,7 +3593,7 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
"run_seconds_total": _comp_run_total,
"run_seconds_max": _comp_run_max,
"leaked_threads_total": _comp_leaked,
"quarantine_active": _comp_timed_out_in_flight > 0,
"quarantine_active": _comp_quarantine_active,
"timed_out_workers": _comp_timed_out_in_flight,
"timed_out_workers_max": _comp_timed_out_in_flight_max,
"quarantine_activations_total": _comp_quarantine_activations,
+87
View File
@@ -294,6 +294,93 @@ def test_timeout_quarantines_new_work_until_timed_out_worker_finishes() -> None:
assert 'headroom_compression_quarantine_total{event="skipped"} 1' in prometheus_text
def test_quarantine_waits_for_half_the_pool_to_be_stuck() -> None:
"""One straggler must not refuse compression while most workers are idle.
The quarantine exists so timed-out work cannot saturate the executor. On
an 8-worker pool, one or three stuck workers leave the pool mostly idle,
so compression keeps running; at four (half the pool) the quarantine
engages, is counted once, and ``/health`` reports it. A timeout after the
time cap released it counts as a fresh activation.
"""
from fastapi.testclient import TestClient
from headroom.proxy.server import CompressionQuarantinedError
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
compression_max_workers=8,
)
app = create_app(config)
proxy = app.state.proxy
release = threading.Event()
async def _strand(count: int) -> None:
for _ in range(count):
with pytest.raises(asyncio.TimeoutError):
await proxy._run_compression_in_executor(release.wait, timeout=0.05)
def _executor_health() -> dict:
with TestClient(app) as client:
return client.get("/health").json()["runtime"]["compression_executor"]
try:
asyncio.run(_strand(1))
assert asyncio.run(proxy._run_compression_in_executor(lambda: "ran", timeout=1.0)) == "ran"
health = _executor_health()
assert health["timed_out_workers"] == 1
assert health["quarantine_active"] is False
asyncio.run(_strand(2))
assert asyncio.run(proxy._run_compression_in_executor(lambda: "ran", timeout=1.0)) == "ran"
with proxy._compression_metrics_lock:
assert proxy._compression_quarantine_activations == 0
assert proxy._compression_quarantine_skips == 0
assert proxy._compression_quarantine_releases == 0
asyncio.run(_strand(1))
with pytest.raises(CompressionQuarantinedError):
asyncio.run(proxy._run_compression_in_executor(lambda: "unused", timeout=1.0))
health = _executor_health()
assert health["timed_out_workers"] == 4
assert health["quarantine_active"] is True
assert health["quarantine_activations_total"] == 1
assert health["quarantine_skips_total"] == 1
proxy._compression_quarantine_deadline = time.monotonic() - 1.0
assert asyncio.run(proxy._run_compression_in_executor(lambda: "ran", timeout=1.0)) == "ran"
assert proxy._compression_quarantine_releases == 1
asyncio.run(_strand(1))
with pytest.raises(CompressionQuarantinedError):
asyncio.run(proxy._run_compression_in_executor(lambda: "unused", timeout=1.0))
assert proxy._compression_quarantine_activations == 2
finally:
release.set()
@pytest.mark.parametrize(
("workers", "env", "expected"),
[(8, None, 4), (3, None, 1), (1, None, 1), (8, "1", 1), (8, "0", 4)],
)
def test_quarantine_threshold_is_half_the_pool_unless_overridden(
monkeypatch, workers: int, env: str | None, expected: int
) -> None:
if env is None:
monkeypatch.delenv("HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD", raising=False)
else:
monkeypatch.setenv("HEADROOM_COMPRESSION_QUARANTINE_THRESHOLD", env)
proxy = _make_proxy(compression_max_workers=workers)
assert proxy._compression_quarantine_threshold == expected
def test_timeout_before_worker_start_does_not_leak_in_flight() -> None:
"""If a queued job times out before a worker starts, queued accounting
is cleaned up without touching the running gauge.
+5 -4
View File
@@ -179,8 +179,9 @@ async def test_count_tokens_offloaded_fails_open_on_executor_quarantine() -> Non
# quarantines the next call — no mock of the helper itself. Since the
# quarantine became time-capped (#2412), standing debt alone no longer
# quarantines: the deadline armed by the fresh timeout must still be in
# the future, so arm it the way a real timeout would.
proxy._compression_timed_out_in_flight = 1
# the future, so arm it the way a real timeout would. The debt must reach
# the quarantine threshold (half the auto-sized pool).
proxy._compression_timed_out_in_flight = proxy._compression_quarantine_threshold
proxy._compression_quarantine_deadline = time.monotonic() + 60.0
tokenizer, tokens = await proxy._count_tokens_offloaded(
@@ -198,8 +199,8 @@ async def test_count_tokens_offloaded_returns_count_text_capable_tokenizer() ->
proxy = _make_proxy()
# Quarantine forces the fail-open branch (an EstimatingTokenCounter).
# Post-#2412 the quarantine is time-capped, so the deadline must be armed
# alongside the standing debt.
proxy._compression_timed_out_in_flight = 1
# alongside the standing debt, which must reach the quarantine threshold.
proxy._compression_timed_out_in_flight = proxy._compression_quarantine_threshold
proxy._compression_quarantine_deadline = time.monotonic() + 60.0
# The empty-messages count is intentionally discarded by that handler