fix(kompress): skip the local model download when a remote endpoint is set (#4234)

## Description

When HEADROOM_KOMPRESS_ENDPOINT selects a remote compressor, skip
downloading an unused local Kompress model and avoid starting the local
model warmup thread. Preserve the remote backend identity in readiness
output and startup logs.

## Type of Change
- [x] Bug fix (remote Kompress startup)

## Changes Made
- Recognize the cached remote compressor during eager loading and report
it enabled with backend=remote.
- Skip local model artifact prefetch for that remote compressor.
- Carry backend metadata into the warmup registry.
- Report the remote endpoint accurately in the startup log.
- Cover remote startup, warmup, and readiness behavior.

## Testing
- [x] Maintainer warmup, health, background-warmup, and snapshot suites
passed after the current-main sync.
- [x] Whitespace check passed.

### Test Output
```text
python -m pytest tests/test_proxy_warmup.py tests/test_proxy_health.py tests/test_proxy/test_kompress_background_warmup.py tests/test_original_snapshot_in_place_hook.py -q
38 passed, 1 warning in 33.13s
```

Hosted CI is qualifying head 579e50cf57.
The maintainer used the available development native extension for app
startup; no newly built native binary is claimed.

## Real Behavior Proof
- Environment: isolated Windows/Python 3.12.13 maintainer worktree with
the actual proxy startup/readiness test fixtures.
- Exact command / steps: run the four pytest suites shown above,
including the remote-endpoint startup regression.
- Observed result: all 38 tests passed after incorporating current main,
including checks that a remote compressor does not start local artifact
prefetch and retains its backend in readiness status.
- Not tested: independent maintainer live remote inference or an
empty-cache artifact-download comparison. The author's real-proxy
before/after proof is preserved below and remains attributed to the
author.

## Runtime Rollout Safety
- Rollout-managed feature(s): no new feature or rollout toggle
introduced.
- Minimum rollout channel: existing remote-Kompress deployment policy
applies.
- Stable/default behavior changed: a configured remote endpoint no
longer causes an unused local model download/warmup; local-compressor
deployments retain their existing path.
- Kill switch / disable path: existing Kompress enable/disable controls
apply; there is no dedicated toggle for this startup correction.
- Unsafe override required: no.
- Qualification impact: remote startup/readiness checks are included;
hosted CI and human review remain required.
- Rollback path: revert the change or retain the prior qualified
release.

## Review Readiness
- [ ] I have performed a self-review
- [ ] This PR is ready for human review

Maintainer verification is documented above; author readiness
attestations have not been filled on the author's behalf.

## Additional Notes

The original author description and supplied evidence are preserved
below. The original follow-up exclusions remain unchanged.
#### The test we ran

A Daytona sandbox running Headroom Enterprise 0.4.3 (core 0.39.1), with
Codex CLI 0.162.1 and Claude Code connected to real GitHub, Context7 and
Fetch MCP servers, working through the workshop tasks. Kompress pointed
at the hosted endpoint on Modal (`HEADROOM_KOMPRESS_ENDPOINT`).

#### What we found

The proxy still downloaded the **~262 MB local Kompress ONNX model**,
even though all inference went to the remote endpoint. The startup log
said:

```
Kompress model preload deferred until first request
Kompress: DEFERRED (warming in the background after startup)
```

Here is why. `ContentRouter._get_kompress()` correctly returns the
`RemoteKompressCompressor`. But `eager_load_compressors()` only checked
`hasattr(compressor, "preload")`, and the remote class has a no-op
`preload`. So startup:

1. marked Kompress `deferred`;
2. called `_prefetch_kompress_artifacts_async(compressor.config)`, which
fetched the default local model files because the remote's config
carries the default `model_id`;
3. had the server start the `kompress-warmup` thread.

**Local fallback:** core has no design for falling back to the local
model when the remote fails. `RemoteKompressCompressor` fails open to
pass-through. So there was nothing to keep, and the local model is
simply not needed when an endpoint is set.

#### The fix

- `eager_load_compressors()`: when the compressor is the router's cached
remote one, report `kompress=enabled` with `kompress_backend=remote`,
and skip the artifact prefetch. Because the status is no longer
`deferred`, the server doesn't start the background warm-up.
- `WarmupRegistry.merge_transform_status` now carries `backend=remote`
into the kompress slot. Without this, `/readyz` would show `backend:
null`: once the slot is marked loaded at startup,
`_reconcile_kompress_health` returns early and never fills it in.
- The startup log now says `Kompress: ENABLED (remote endpoint; no local
model)` instead of "ModernBERT token compressor".

#### Before / after (proxy startup with `HEADROOM_KOMPRESS_ENDPOINT` set
and `optimize=True`)

| | origin/main | this PR |
|---|---|---|
| Local model artifact prefetch started | yes
(`chopratejas/kompress-v2-base`) | no |
| `kompress-warmup` thread | started | not started |
| `/readyz` `checks.kompress.backend` | `remote` | `remote` |

#### Tests

-
`tests/test_proxy_health.py::test_remote_kompress_endpoint_skips_local_model_at_startup`
boots the real app with the endpoint set. Fails on origin/main (the
prefetch ran), passes here.
- Related suites pass: kompress preload deferral, proxy warmup, eager
preload bind, debug endpoints, background warm-up, remote drop-in and
remote endpoint (115 passed).
- ruff check, ruff format --check and mypy pass.

#### Not in this PR (follow-up)

`CodeAwareCompressor._fallback_compress`
(`headroom/transforms/code_compressor.py`) builds its own local
`KompressCompressor()` for unknown languages or AST failures, and
ignores `HEADROOM_KOMPRESS_ENDPOINT`. With `[ml]` installed it can still
load the local model on the first such code block.
`headroom/compression/universal.py` does the same. Fixing that needs the
router's Kompress getter threaded into the code compressor, which is a
separate change.

#### Real behavior proof

- **Setup:** macOS 26.7, Python 3.12.13, `[ml]` installed. Two proxies,
one on origin/main and one on this branch, each with an empty `HF_HOME`
and `HEADROOM_KOMPRESS_ENDPOINT=https://kompress.invalid`. Command:
`headroom proxy --port <p>`, left running about 50 s, then `curl
/readyz`.
- **origin/main:**
  ```
  Kompress: using remote endpoint https://kompress.invalid/compress
Kompress: prefetching model artifacts for chopratejas/kompress-v2-base
...
  Kompress model preload deferred until first request
  Kompress: DEFERRED (warming in the background after startup)
  Kompress: warmed in the background in 0 ms (backend remote)
Kompress: artifact prefetch complete for chopratejas/kompress-v2-base;
...
$ du -sh $HF_HOME -> 261M
(hub/models--chopratejas--kompress-v2-base/.../kompress-int8-wo.onnx)
/readyz checks.kompress -> {'enabled': True, 'ready': True, 'backend':
'remote', ...}
  ```
- **This branch:**
  ```
  Kompress: using remote endpoint https://kompress.invalid/compress
  Kompress: ENABLED (remote endpoint; no local model)
  $ du -sh $HF_HOME   ->  0B
/readyz checks.kompress -> {'enabled': True, 'ready': True, 'backend':
'remote', ...}
  ```
- **Not tested:** compression requests against the real Modal endpoint
after the patch. The request path is unchanged: `_get_kompress()`
already returned the remote compressor. The code-compressor fallback
path described above is also untested.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-10-16 14:31:59 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent 8cedf7be4d
commit ff36931653
4 changed files with 45 additions and 5 deletions
+4 -1
View File
@@ -2229,7 +2229,10 @@ class HeadroomProxy(
# Log component status
if self._kompress_status == "enabled":
logger.info("Kompress: ENABLED (ModernBERT token compressor)")
if eager_status.get("kompress_backend") == "remote":
logger.info("Kompress: ENABLED (remote endpoint; no local model)")
else:
logger.info("Kompress: ENABLED (ModernBERT token compressor)")
elif self._kompress_status == "deferred":
if self._start_kompress_background_warmup():
logger.info("Kompress: DEFERRED (warming in the background after startup)")
+9 -3
View File
@@ -111,21 +111,27 @@ class WarmupRegistry:
``error`` statuses must be set explicitly by the caller.
"""
def _apply(slot: WarmupSlot, value: str | None) -> None:
def _apply(slot: WarmupSlot, value: str | None, **info: Any) -> None:
if value is None:
return
v = value.strip().lower()
if v in {"enabled", "ready"} or v.startswith("loaded"):
# Preserve whatever handle/info was set already.
if slot.status != "loaded":
slot.mark_loaded(handle=slot.handle, source_status=value)
slot.mark_loaded(handle=slot.handle, source_status=value, **info)
else:
slot.info.setdefault("source_status", value)
else:
if slot.status != "loaded":
slot.info["source_status"] = value
_apply(self.kompress, status.get("kompress"))
# A remote endpoint is ready at startup; /readyz reports its backend.
kompress_backend = status.get("kompress_backend")
_apply(
self.kompress,
status.get("kompress"),
**({"backend": kompress_backend} if kompress_backend == "remote" else {}),
)
_apply(self.magika, status.get("magika"))
_apply(self.code_aware, status.get("code_aware"))
_apply(self.tree_sitter, status.get("tree_sitter"))
+6 -1
View File
@@ -5198,7 +5198,12 @@ class ContentRouter(Transform):
if self.config.enable_kompress:
compressor = self._get_kompress()
if compressor:
if not hasattr(compressor, "preload"):
if compressor is self._get_remote_kompress():
# HEADROOM_KOMPRESS_ENDPOINT runs the model; there is no
# local model to download or warm.
status["kompress"] = "enabled"
status["kompress_backend"] = "remote"
elif not hasattr(compressor, "preload"):
status["kompress"] = "enabled"
status["kompress_backend"] = "unknown"
else:
+26
View File
@@ -323,3 +323,29 @@ def test_readyz_kompress_state_matrix(monkeypatch, slot_status, compressor, disa
payload = TestClient(app).get("/readyz").json()["checks"]["kompress"]
assert payload == expected
def test_remote_kompress_endpoint_skips_local_model_at_startup(monkeypatch):
"""With HEADROOM_KOMPRESS_ENDPOINT set, startup neither fetches nor warms the
local model, and /readyz still names the remote backend."""
import threading
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
monkeypatch.setenv("HEADROOM_KOMPRESS_ENDPOINT", "https://kompress.example.test")
monkeypatch.setenv("HEADROOM_KOMPRESS_WARMUP", "1")
monkeypatch.setenv("HEADROOM_KOMPRESS_WARMUP_DELAY_SECONDS", "3600")
prefetched: list[str] = []
monkeypatch.setattr(
kompress_compressor,
"ensure_background_prefetch",
lambda model_id: (prefetched.append(model_id), True)[1],
)
app = create_app(ProxyConfig(optimize=True, cache_enabled=False, rate_limit_enabled=False))
with TestClient(app) as client:
app.state.proxy.http_client = object()
payload = client.get("/readyz").json()
assert prefetched == []
assert not any(t.name == "kompress-warmup" for t in threading.enumerate())
assert payload["checks"]["kompress"]["backend"] == "remote"