mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description The Kompress ONNX artifacts are pinned to a commit SHA in `_PINNED_REVISIONS`, but the tokenizer they were exported against (`answerdotai/ModernBERT-base`) was loaded from the floating default ref, and so was the torch-path encoder base. The export script that produces those artifacts also loaded both the Kompress checkpoint and the tokenizer from floating refs. An upstream change could shift token ids under the shipped weights, or produce an artifact production never loads, with no code change on our side. Revised 2026-10-05 after review: the ONNX export script now loads the checkpoint, encoder and tokenizer only at immutable commits defaulting to the production pins, and cache-only PyTorch loads no longer fetch the encoder. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `answerdotai/ModernBERT-base @ 8949b909ec900327062f0ebf497f51aef5e6f0c8` (the snapshot the export cache holds; upstream `main` at the time) to `_PINNED_REVISIONS`. - `_load_modernbert_tokenizer` resolves the pin via `_resolve_revision` and passes `revision=` on both the cache-only and download paths. `HEADROOM_HF_PIN=off` floats it exactly as it floats the weights. - `HeadroomCompressorModel` takes `revision=` and `allow_download=`; it passes the resolved revision and `local_files_only=not allow_download` to `AutoModel.from_pretrained`. `_load_kompress_pytorch` forwards `allow_download` and turns a missing encoder snapshot into `KompressModelNotCached` in cache-only mode (Devin finding). - `scripts/export_kompress_v2_onnx.py`: new `--revision` (checkpoint) and `--base-revision` (ModernBERT encoder + tokenizer). Both default to the production pins in `_PINNED_REVISIONS` (not affected by `HEADROOM_HF_PIN`); anything other than a 40-hex commit, or an unpinned `--model-id` with no `--revision`, is rejected. Every Hub/Transformers load (`hf_hub_download`, encoder, `AutoTokenizer`) passes them. A model upgrade is now a reviewed pin bump plus a regenerated artifact. No behaviour change for a warm cache at the same snapshot; a cache populated from a different revision now downloads the pinned one (when downloads are allowed) instead of using whatever is present. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) — on touched files, plus `ruff format --check` - [x] Type checking passes (`mypy headroom`) — `mypy headroom/transforms/kompress_compressor.py` - [x] New tests added for new functionality - [ ] Manual testing performed New tests (all fail on the pre-fix code, pass now; no network): - `tests/test_scripts/test_export_kompress_v2_onnx_pins.py`: AST check that every `from_pretrained` / `hf_hub_download` / `snapshot_download` / model-class load in the export script passes `revision=`; defaults are the 40-hex production pins even with `HEADROOM_HF_PIN=off`; `main`, a tag, or an unpinned repo are rejected; `_build_core` threads both revisions. - `tests/test_hf_revision_pinning.py`: tokenizer pin present / cache-only path / download path / bypass, and `test_cache_only_pytorch_load_does_not_fetch_encoder`. ### Test Output ```text $ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --frozen --extra dev --extra proxy pytest -q tests/test_hf_revision_pinning.py tests/test_scripts/test_export_kompress_v2_onnx_pins.py tests/test_kompress_preload_deferral.py tests/test_transforms/test_kompress_compressor.py 78 passed in 62.67s # With the source changes reverted: 7 failed, 15 passed (all 7 new tests fail) ``` ## Real Behavior Proof - Environment: macOS, Python 3.14, `uv run --frozen --extra dev --extra proxy`. - Exact command / steps: the pytest command above; tests drive the real loader and export entry points with Hub/torch boundaries faked. - Observed result: every Hub load in the export path receives a 40-hex revision; cache-only PyTorch load raises `KompressModelNotCached` without attempting a download. - Not tested: a real ONNX export / upload against the Hub (needs torch + HF write auth); no artifact is regenerated by this PR. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: n/a. - Stable/default behavior changed: the ModernBERT tokenizer/encoder load at the pinned commit instead of the floating ref; cache-only PyTorch loads defer instead of downloading the encoder. - Kill switch / disable path: `HEADROOM_HF_PIN=off` floats the runtime loads, as for every other pinned repo. - Unsafe override required: no. - Qualification impact: none expected; the pin equals the snapshot the artifacts were exported against. - Rollback path: revert this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Uploading a newly exported artifact creates a new commit in the model repo; production picks it up only after the `chopratejas/kompress-v2-base` pin is bumped in a reviewed PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>