Commit Graph
6 Commits
Author SHA1 Message Date
2de582875b fix(kompress): pin ModernBERT tokenizer and encoder revisions (#3808)
## Description

The Kompress ONNX artifacts are pinned to a commit SHA in
`_PINNED_REVISIONS`, but the tokenizer they were exported against
(`answerdotai/ModernBERT-base`) was loaded from the floating default
ref, and so was the torch-path encoder base. The export script that
produces those artifacts also loaded both the Kompress checkpoint and
the tokenizer from floating refs. An upstream change could shift token
ids under the shipped weights, or produce an artifact production never
loads, with no code change on our side.

Revised 2026-10-05 after review: the ONNX export script now loads the
checkpoint, encoder and tokenizer only at immutable commits defaulting
to the production pins, and cache-only PyTorch loads no longer fetch the
encoder.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `answerdotai/ModernBERT-base @
8949b909ec900327062f0ebf497f51aef5e6f0c8` (the snapshot the export cache
holds; upstream `main` at the time) to `_PINNED_REVISIONS`.
- `_load_modernbert_tokenizer` resolves the pin via `_resolve_revision`
and passes `revision=` on both the cache-only and download paths.
`HEADROOM_HF_PIN=off` floats it exactly as it floats the weights.
- `HeadroomCompressorModel` takes `revision=` and `allow_download=`; it
passes the resolved revision and `local_files_only=not allow_download`
to `AutoModel.from_pretrained`. `_load_kompress_pytorch` forwards
`allow_download` and turns a missing encoder snapshot into
`KompressModelNotCached` in cache-only mode (Devin finding).
- `scripts/export_kompress_v2_onnx.py`: new `--revision` (checkpoint)
and `--base-revision` (ModernBERT encoder + tokenizer). Both default to
the production pins in `_PINNED_REVISIONS` (not affected by
`HEADROOM_HF_PIN`); anything other than a 40-hex commit, or an unpinned
`--model-id` with no `--revision`, is rejected. Every Hub/Transformers
load (`hf_hub_download`, encoder, `AutoTokenizer`) passes them. A model
upgrade is now a reviewed pin bump plus a regenerated artifact.

No behaviour change for a warm cache at the same snapshot; a cache
populated from a different revision now downloads the pinned one (when
downloads are allowed) instead of using whatever is present.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`) — on touched files, plus `ruff
format --check`
- [x] Type checking passes (`mypy headroom`) — `mypy
headroom/transforms/kompress_compressor.py`
- [x] New tests added for new functionality
- [ ] Manual testing performed

New tests (all fail on the pre-fix code, pass now; no network):
- `tests/test_scripts/test_export_kompress_v2_onnx_pins.py`: AST check
that every `from_pretrained` / `hf_hub_download` / `snapshot_download` /
model-class load in the export script passes `revision=`; defaults are
the 40-hex production pins even with `HEADROOM_HF_PIN=off`; `main`, a
tag, or an unpinned repo are rejected; `_build_core` threads both
revisions.
- `tests/test_hf_revision_pinning.py`: tokenizer pin present /
cache-only path / download path / bypass, and
`test_cache_only_pytorch_load_does_not_fetch_encoder`.

### Test Output

```text
$ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --frozen --extra dev --extra proxy pytest -q tests/test_hf_revision_pinning.py tests/test_scripts/test_export_kompress_v2_onnx_pins.py tests/test_kompress_preload_deferral.py tests/test_transforms/test_kompress_compressor.py
78 passed in 62.67s
# With the source changes reverted: 7 failed, 15 passed (all 7 new tests fail)
```

## Real Behavior Proof

- Environment: macOS, Python 3.14, `uv run --frozen --extra dev --extra
proxy`.
- Exact command / steps: the pytest command above; tests drive the real
loader and export entry points with Hub/torch boundaries faked.
- Observed result: every Hub load in the export path receives a 40-hex
revision; cache-only PyTorch load raises `KompressModelNotCached`
without attempting a download.
- Not tested: a real ONNX export / upload against the Hub (needs torch +
HF write auth); no artifact is regenerated by this PR.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: the ModernBERT tokenizer/encoder load
at the pinned commit instead of the floating ref; cache-only PyTorch
loads defer instead of downloading the encoder.
- Kill switch / disable path: `HEADROOM_HF_PIN=off` floats the runtime
loads, as for every other pinned repo.
- Unsafe override required: no.
- Qualification impact: none expected; the pin equals the snapshot the
artifacts were exported against.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Uploading a newly exported artifact creates a new commit in the model
repo; production picks it up only after the
`chopratejas/kompress-v2-base` pin is bumped in a reviewed PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:29:20 -05:00
Adryan Eka Vandra b71b659e1b fix(ci): restore repro harness test in dev installs
Add websockets to the dev extra so the repro harness smoke test can import
its websocket client dependency in the CI test matrix. Also apply ruff
formatting to the files the formatter check was rejecting so the 3.12 lint
job passes.
2026-04-20 22:12:01 +07:00
Adryan Eka Vandra c438268d8a feat(scripts): improve repro harness CLI contract and backoff
- Route --json output cleanly: JSON goes to stdout (machine-readable),
  human-readable summary goes to stderr. Without --json, the human
  summary stays on stdout as before. Makes piping the harness into
  jq / other tools trivial without losing operator visibility.
- Add distinct exit codes so CI and shell wrappers can branch on the
  failure class: 0 success, 1 crash, 2 proxy_unreachable, 3 livez
  threshold exceeded, 4 warmup failed, 130 SIGINT (already correct).
  Smoke test updated to assert EXIT_PROXY_UNREACHABLE instead of 1.
- Replace flat 50-250ms jitter in the Anthropic-client retry loop with
  exponential backoff + 50-150% jitter
  (base=250ms, max=5000ms, attempt counter). Matches the proxy's own
  jitter_delay_ms helper; inlined to keep the script free of proxy
  package imports.
2026-04-20 22:09:23 +07:00
Adryan Eka Vandra 30e7b7c498 test(scripts): restore real websockets module before repro smoke test
Some earlier tests in the suite replace sys.modules['websockets'] with
a stub. When the smoke test ran after those, uvicorn's
websockets-sansio backend tried to import websockets.server at connect
time, resolved through the stub, and failed with
'No module named websockets.server; websockets is not a package',
preventing the mock proxy from starting.

Add an autouse fixture that drops stub websockets.* entries from
sys.modules, re-imports the real package + websockets.asyncio.server,
and restores prior state on teardown. Keeps the smoke test
deterministic regardless of collection order.
2026-04-20 22:02:40 +07:00
Adryan Eka Vandra ea0d519677 fix(tests): pin repro smoke server to websockets-sansio backend
uvicorn's auto-select imports websockets_impl.py which requires the
legacy API (websockets<14). With websockets 16.0 installed in the
local venv, auto-select errored before its fallback could trigger,
failing the smoke test with 'No module named websockets.legacy'.

Explicitly selecting the websockets-sansio backend (shipped with
uvicorn) works against websockets 16.0 and keeps the smoke test
representative of the real production WS path.
2026-04-20 22:02:40 +07:00
Adryan Eka Vandra c482d75a73 feat(scripts): add Codex proxy reconnect-storm repro harness
Reproducibly exercises the multi-agent Codex reconnect/retry storm
(origin §"Latest Correction") against a local proxy: concurrent Codex WS
sessions + parallel Anthropic /v1/messages replays, with /livez probed
every 250ms to detect event-loop starvation. Exits non-zero if /livez
p99 exceeds the configured threshold.

- scripts/repro_codex_replay.py — CLI harness (asyncio + websockets +
  httpx, no new pip deps).
- scripts/fixtures/{anthropic_replay_body,codex_response_create_frame}.json
  — hand-crafted, fully synthetic fixtures shaped like real traffic.
- scripts/README.md — "Reproducing the reconnect storm" section.
- tests/test_scripts/test_repro_codex_replay_smoke.py — FastAPI/uvicorn
  mock server; runs the harness end-to-end in < 3s.
- .gitignore — allowlist the new checked-in scripts (existing rule
  previously ignored everything under scripts/ except install.sh/.ps1).
2026-04-20 22:02:02 +07:00