fix(kompress): pin ModernBERT tokenizer and encoder revisions (#3808)

## Description

The Kompress ONNX artifacts are pinned to a commit SHA in
`_PINNED_REVISIONS`, but the tokenizer they were exported against
(`answerdotai/ModernBERT-base`) was loaded from the floating default
ref, and so was the torch-path encoder base. The export script that
produces those artifacts also loaded both the Kompress checkpoint and
the tokenizer from floating refs. An upstream change could shift token
ids under the shipped weights, or produce an artifact production never
loads, with no code change on our side.

Revised 2026-10-05 after review: the ONNX export script now loads the
checkpoint, encoder and tokenizer only at immutable commits defaulting
to the production pins, and cache-only PyTorch loads no longer fetch the
encoder.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `answerdotai/ModernBERT-base @
8949b909ec900327062f0ebf497f51aef5e6f0c8` (the snapshot the export cache
holds; upstream `main` at the time) to `_PINNED_REVISIONS`.
- `_load_modernbert_tokenizer` resolves the pin via `_resolve_revision`
and passes `revision=` on both the cache-only and download paths.
`HEADROOM_HF_PIN=off` floats it exactly as it floats the weights.
- `HeadroomCompressorModel` takes `revision=` and `allow_download=`; it
passes the resolved revision and `local_files_only=not allow_download`
to `AutoModel.from_pretrained`. `_load_kompress_pytorch` forwards
`allow_download` and turns a missing encoder snapshot into
`KompressModelNotCached` in cache-only mode (Devin finding).
- `scripts/export_kompress_v2_onnx.py`: new `--revision` (checkpoint)
and `--base-revision` (ModernBERT encoder + tokenizer). Both default to
the production pins in `_PINNED_REVISIONS` (not affected by
`HEADROOM_HF_PIN`); anything other than a 40-hex commit, or an unpinned
`--model-id` with no `--revision`, is rejected. Every Hub/Transformers
load (`hf_hub_download`, encoder, `AutoTokenizer`) passes them. A model
upgrade is now a reviewed pin bump plus a regenerated artifact.

No behaviour change for a warm cache at the same snapshot; a cache
populated from a different revision now downloads the pinned one (when
downloads are allowed) instead of using whatever is present.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`) — on touched files, plus `ruff
format --check`
- [x] Type checking passes (`mypy headroom`) — `mypy
headroom/transforms/kompress_compressor.py`
- [x] New tests added for new functionality
- [ ] Manual testing performed

New tests (all fail on the pre-fix code, pass now; no network):
- `tests/test_scripts/test_export_kompress_v2_onnx_pins.py`: AST check
that every `from_pretrained` / `hf_hub_download` / `snapshot_download` /
model-class load in the export script passes `revision=`; defaults are
the 40-hex production pins even with `HEADROOM_HF_PIN=off`; `main`, a
tag, or an unpinned repo are rejected; `_build_core` threads both
revisions.
- `tests/test_hf_revision_pinning.py`: tokenizer pin present /
cache-only path / download path / bypass, and
`test_cache_only_pytorch_load_does_not_fetch_encoder`.

### Test Output

```text
$ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --frozen --extra dev --extra proxy pytest -q tests/test_hf_revision_pinning.py tests/test_scripts/test_export_kompress_v2_onnx_pins.py tests/test_kompress_preload_deferral.py tests/test_transforms/test_kompress_compressor.py
78 passed in 62.67s
# With the source changes reverted: 7 failed, 15 passed (all 7 new tests fail)
```

## Real Behavior Proof

- Environment: macOS, Python 3.14, `uv run --frozen --extra dev --extra
proxy`.
- Exact command / steps: the pytest command above; tests drive the real
loader and export entry points with Hub/torch boundaries faked.
- Observed result: every Hub load in the export path receives a 40-hex
revision; cache-only PyTorch load raises `KompressModelNotCached`
without attempting a download.
- Not tested: a real ONNX export / upload against the Hub (needs torch +
HF write auth); no artifact is regenerated by this PR.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: the ModernBERT tokenizer/encoder load
at the pinned commit instead of the floating ref; cache-only PyTorch
loads defer instead of downloading the encoder.
- Kill switch / disable path: `HEADROOM_HF_PIN=off` floats the runtime
loads, as for every other pinned repo.
- Unsafe override required: no.
- Qualification impact: none expected; the pin equals the snapshot the
artifacts were exported against.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Uploading a newly exported artifact creates a new commit in the model
repo; production picks it up only after the
`chopratejas/kompress-v2-base` pin is bumped in a reviewed PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-10-05 08:29:20 -05:00
committed by GitHub
co-authored by Claude Fable 5.1 JD Davis
parent 84f0e84dcc
commit 2de582875b
5 changed files with 320 additions and 16 deletions
+4
View File
@@ -71,6 +71,10 @@ def onnx_thread_spinning_enabled() -> bool:
_PINNED_REVISIONS: dict[str, str] = {
# chopratejas/kompress-v2-base @ 2026-06-10
"chopratejas/kompress-v2-base": "b1563631b35bfdcee37587ad530147497d820d4c",
# Kompress tokenizer + torch-path encoder base. The ONNX artifacts were
# exported against this snapshot; a silently updated tokenizer would shift
# token ids under the shipped weights.
"answerdotai/ModernBERT-base": "8949b909ec900327062f0ebf497f51aef5e6f0c8",
"chopratejas/technique-router-onnx": "27b0b4bfa510a1cff66d888072c0b807082721a8",
"chopratejas/siglip-image-encoder-onnx": "d0a9fbd66d4bd8c761bff592d44831f7c2ae184e",
# Third-party repo — pinning matters most here.
+40 -7
View File
@@ -30,6 +30,7 @@ from typing import Any, Literal
from ..config import TransformResult
from ..onnx_runtime import (
ONNX_CPU_ARENA_ENV,
_resolve_revision,
create_cpu_session_options,
hf_entry_known_absent,
hf_hub_download_local_first,
@@ -600,9 +601,22 @@ def _get_model_class() -> type:
class HeadroomCompressorModel(nn.Module):
"""Dual-head ModernBERT: token classification + span importance CNN."""
def __init__(self, model_name: str = "answerdotai/ModernBERT-base"):
def __init__(
self,
model_name: str = "answerdotai/ModernBERT-base",
*,
revision: str | None = None,
allow_download: bool = True,
):
super().__init__()
self.encoder = AutoModel.from_pretrained(model_name, attn_implementation="eager")
# Same pin as the tokenizer: the fine-tuned heads were trained on
# this encoder snapshot. Unknown repos resolve to None (floating).
self.encoder = AutoModel.from_pretrained(
model_name,
attn_implementation="eager",
revision=_resolve_revision(model_name, revision),
local_files_only=not allow_download,
)
hidden_size = self.encoder.config.hidden_size # 768
# Head 1: Token keep/discard
@@ -831,6 +845,9 @@ def _load_kompress_onnx(
return model, tokenizer, backend
_MODERNBERT_TOKENIZER_REPO = "answerdotai/ModernBERT-base"
def _load_modernbert_tokenizer(auto_tokenizer: Any, *, allow_download: bool) -> Any:
"""Load the ModernBERT tokenizer, cache-only when ``allow_download`` is False.
@@ -845,14 +862,25 @@ def _load_modernbert_tokenizer(auto_tokenizer: Any, *, allow_download: bool) ->
changes whether the Hub is consulted to confirm what is already on disk.
Mirrors ``onnx_runtime.hf_hub_download_local_first``, which the ONNX half of
this loader already uses.
The load is pinned to the same immutable revision as the model artifacts
(``onnx_runtime._PINNED_REVISIONS``): the ONNX export was produced against
that tokenizer snapshot, so a floating ``main`` could shift token ids under
the shipped weights without any code change. ``HEADROOM_HF_PIN=off`` floats
it, exactly as for the weights.
"""
revision = _resolve_revision(_MODERNBERT_TOKENIZER_REPO, None)
try:
return auto_tokenizer.from_pretrained("answerdotai/ModernBERT-base", local_files_only=True)
return auto_tokenizer.from_pretrained(
_MODERNBERT_TOKENIZER_REPO, revision=revision, local_files_only=True
)
except _NOT_CACHED_ERRORS as exc:
if not allow_download:
raise KompressModelNotCached("answerdotai/ModernBERT-base") from exc
# Genuine cache miss and downloading is permitted: fetch it.
return auto_tokenizer.from_pretrained("answerdotai/ModernBERT-base", local_files_only=False)
raise KompressModelNotCached(_MODERNBERT_TOKENIZER_REPO) from exc
# Genuine cache miss and downloading is permitted: fetch it at the pin.
return auto_tokenizer.from_pretrained(
_MODERNBERT_TOKENIZER_REPO, revision=revision, local_files_only=False
)
# Sub-state-dict keys inside a merged v2-style checkpoint (see
@@ -971,7 +999,12 @@ def _load_kompress_pytorch(
logger.info("Downloading Kompress PyTorch model from %s ...", model_id)
HeadroomCompressorModel = _get_model_class()
model = HeadroomCompressorModel()
try:
model = HeadroomCompressorModel(allow_download=allow_download)
except _NOT_CACHED_ERRORS as exc:
if allow_download:
raise
raise KompressModelNotCached(_MODERNBERT_TOKENIZER_REPO) from exc
_load_pytorch_weights(model, model_id, allow_download=allow_download)
+56 -9
View File
@@ -31,12 +31,22 @@ Usage
# Convert, verify, and upload back to the HF repo (needs `huggingface-cli login`):
python scripts/export_kompress_v2_onnx.py --model-id chopratejas/kompress-v2-base --upload
Reproducibility
---------------
Every Hub load is pinned to an immutable commit: the checkpoint to
``--revision`` and the ModernBERT encoder + tokenizer to ``--base-revision``.
Both default to the production pins in ``headroom.onnx_runtime._PINNED_REVISIONS``
(``HEADROOM_HF_PIN`` does not apply here), so the exported artifact is built
from exactly what production loads. Upgrading a model is a reviewed pin bump
plus a regenerated artifact, never ambient Hub state.
"""
from __future__ import annotations
import argparse
import logging
import re
import sys
from pathlib import Path
@@ -47,8 +57,24 @@ logger = logging.getLogger("export_kompress_v2_onnx")
BASE_MODEL = "answerdotai/ModernBERT-base"
DEFAULT_MODEL_ID = "chopratejas/kompress-v2-base"
_COMMIT_SHA = re.compile(r"[0-9a-f]{40}")
def _build_core(model_id: str):
def _pinned_revision(repo_id: str, revision: str | None) -> str:
"""Return an immutable commit for ``repo_id``: ``revision`` if given, else
the production pin. Branches, tags and unpinned repos are rejected."""
from headroom.onnx_runtime import _PINNED_REVISIONS
revision = revision or _PINNED_REVISIONS.get(repo_id)
if revision is None or not _COMMIT_SHA.fullmatch(revision):
raise SystemExit(
f"{repo_id}: need a 40-hex commit SHA (got {revision!r}). "
"Pass one explicitly or pin the repo in headroom.onnx_runtime._PINNED_REVISIONS."
)
return revision
def _build_core(model_id: str, revision: str, base_revision: str):
"""Instantiate HeadroomCompressorModel and load the merged v2 weights.
The v2 repo's ``model.safetensors`` is the *unmerged* PEFT structure
@@ -67,7 +93,7 @@ def _build_core(model_id: str):
from headroom.transforms.kompress_compressor import _get_model_class
ckpt_path = hf_hub_download(model_id, "merged.pt")
ckpt_path = hf_hub_download(model_id, "merged.pt", revision=revision)
ckpt = torch.load(ckpt_path, map_location="cpu")
for key in ("encoder_state_dict", "token_head_state_dict", "span_conv_state_dict"):
if key not in ckpt:
@@ -76,7 +102,7 @@ def _build_core(model_id: str):
"This script targets the v2 'merged' checkpoint format."
)
core = _get_model_class()(model_name=BASE_MODEL)
core = _get_model_class()(model_name=BASE_MODEL, revision=base_revision)
def _strict_load(module, sd, label: str) -> None:
missing, unexpected = module.load_state_dict(sd, strict=False)
@@ -115,11 +141,18 @@ def _export_wrapper(core):
return ExportWrapper(core).eval()
def export(model_id: str, out_path: Path, opset: int, precision: str) -> None:
def export(
model_id: str,
out_path: Path,
opset: int,
precision: str,
revision: str,
base_revision: str,
) -> None:
import numpy as np
import torch
core = _build_core(model_id)
core = _build_core(model_id, revision, base_revision)
wrapper = _export_wrapper(core)
out_path.parent.mkdir(parents=True, exist_ok=True)
@@ -167,15 +200,15 @@ def export(model_id: str, out_path: Path, opset: int, precision: str) -> None:
)
trace_target.unlink(missing_ok=True)
_verify(model_id, core, out_path, np, torch)
_verify(core, out_path, base_revision, np, torch)
def _verify(model_id: str, core, out_path: Path, np, torch) -> None:
def _verify(core, out_path: Path, base_revision: str, np, torch) -> None:
"""Compare ONNX scores against PyTorch get_scores on a real tokenized sample."""
import onnxruntime as ort
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained(BASE_MODEL)
tok = AutoTokenizer.from_pretrained(BASE_MODEL, revision=base_revision)
sample = (
"The proxy compresses tool outputs before they reach the model. "
"Errors and stack traces should survive; boilerplate should not. "
@@ -236,6 +269,16 @@ def upload(model_id: str, out_path: Path) -> None:
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--model-id", default=DEFAULT_MODEL_ID)
ap.add_argument(
"--revision",
default=None,
help="Checkpoint commit SHA. Defaults to the production pin for --model-id.",
)
ap.add_argument(
"--base-revision",
default=None,
help=f"{BASE_MODEL} commit SHA (encoder + tokenizer). Defaults to the production pin.",
)
ap.add_argument(
"--precision",
choices=["fp32", "int8"],
@@ -256,8 +299,12 @@ def main() -> int:
)
args = ap.parse_args()
revision = _pinned_revision(args.model_id, args.revision)
base_revision = _pinned_revision(BASE_MODEL, args.base_revision)
logger.info("Pins: %s@%s, %s@%s", args.model_id, revision, BASE_MODEL, base_revision)
out_path = args.out or Path(f"onnx/kompress-{args.precision}.onnx")
export(args.model_id, out_path, args.opset, args.precision)
export(args.model_id, out_path, args.opset, args.precision, revision, base_revision)
if args.upload:
upload(args.model_id, out_path)
return 0
+96
View File
@@ -51,3 +51,99 @@ def test_pin_can_be_disabled_via_env(monkeypatch, value):
def test_pin_disabled_still_respects_explicit_revision(monkeypatch):
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
assert _resolve_revision("chopratejas/kompress-v2-base", "abc123") == "abc123"
def test_modernbert_tokenizer_base_is_pinned():
# The Kompress tokenizer (and torch-path encoder) load from this repo; the
# ONNX export was produced against the pinned snapshot, so it must not float.
assert "answerdotai/ModernBERT-base" in _PINNED_REVISIONS
def test_tokenizer_loader_passes_pinned_revision(monkeypatch):
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
from headroom.transforms import kompress_compressor as kc
calls: list[dict] = []
class _Tok:
@staticmethod
def from_pretrained(repo, **kwargs):
calls.append({"repo": repo, **kwargs})
return object()
kc._load_modernbert_tokenizer(_Tok, allow_download=True)
assert calls == [
{
"repo": "answerdotai/ModernBERT-base",
"revision": _PINNED_REVISIONS["answerdotai/ModernBERT-base"],
"local_files_only": True,
}
]
def test_tokenizer_loader_downloads_at_pin_on_cache_miss(monkeypatch):
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
from headroom.transforms import kompress_compressor as kc
calls: list[dict] = []
class _Tok:
@staticmethod
def from_pretrained(repo, **kwargs):
calls.append({"repo": repo, **kwargs})
if kwargs.get("local_files_only"):
raise OSError("not cached")
return object()
kc._load_modernbert_tokenizer(_Tok, allow_download=True)
assert [c["local_files_only"] for c in calls] == [True, False]
assert {c["revision"] for c in calls} == {_PINNED_REVISIONS["answerdotai/ModernBERT-base"]}
def test_tokenizer_loader_floats_when_pin_disabled(monkeypatch):
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
from headroom.transforms import kompress_compressor as kc
calls: list[dict] = []
class _Tok:
@staticmethod
def from_pretrained(repo, **kwargs):
calls.append(kwargs)
return object()
kc._load_modernbert_tokenizer(_Tok, allow_download=True)
assert calls[0]["revision"] is None
def test_cache_only_pytorch_load_does_not_fetch_encoder(monkeypatch):
# A cache-only load must not download the pinned encoder on the request
# path; a missing snapshot defers the load like any other cache miss.
import sys
import types
from headroom.transforms import kompress_compressor as kc
calls: list[dict] = []
class _AutoModel:
@staticmethod
def from_pretrained(repo, **kwargs):
calls.append(kwargs)
if kwargs.get("local_files_only"):
raise OSError("pinned encoder snapshot not cached")
raise AssertionError("cache-only load reached the network")
nn = types.SimpleNamespace(Module=object)
monkeypatch.setitem(sys.modules, "torch", types.SimpleNamespace(nn=nn))
monkeypatch.setitem(sys.modules, "torch.nn", nn)
monkeypatch.setitem(
sys.modules,
"transformers",
types.SimpleNamespace(AutoModel=_AutoModel, AutoTokenizer=object),
)
monkeypatch.setattr(kc, "_kompress_cache", {})
with pytest.raises(kc.KompressModelNotCached):
kc._load_kompress_pytorch("chopratejas/kompress-v2-base", allow_download=False)
assert [c["local_files_only"] for c in calls] == [True]
@@ -0,0 +1,124 @@
"""The Kompress ONNX export must build from the same immutable Hub snapshots
production loads, so an exported artifact can never mix floating inputs."""
from __future__ import annotations
import ast
import importlib.util
import re
import sys
import types
from pathlib import Path
import pytest
from headroom.onnx_runtime import _PINNED_REVISIONS
SCRIPT = Path(__file__).resolve().parents[2] / "scripts" / "export_kompress_v2_onnx.py"
SHA = re.compile(r"[0-9a-f]{40}")
BASE = "answerdotai/ModernBERT-base"
CKPT = "chopratejas/kompress-v2-base"
def _load_script():
spec = importlib.util.spec_from_file_location("export_kompress_v2_onnx", SCRIPT)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
def _call_name(call: ast.Call) -> str:
func = call.func
if isinstance(func, ast.Attribute):
return func.attr
if isinstance(func, ast.Name):
return func.id
if isinstance(func, ast.Call): # _get_model_class()(...)
return _call_name(func) + "()"
return ""
def test_every_hub_load_in_export_script_passes_revision():
loaders = {"from_pretrained", "hf_hub_download", "snapshot_download", "_get_model_class()"}
calls = [
node
for node in ast.walk(ast.parse(SCRIPT.read_text()))
if isinstance(node, ast.Call) and _call_name(node) in loaders
]
assert {_call_name(c) for c in calls} >= {
"from_pretrained",
"hf_hub_download",
"_get_model_class()",
}
unpinned = [
f"line {c.lineno}: {_call_name(c)}"
for c in calls
if not any(kw.arg == "revision" for kw in c.keywords)
]
assert unpinned == []
def test_export_defaults_to_production_pins_even_with_pinning_disabled(monkeypatch):
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
mod = _load_script()
seen: dict = {}
monkeypatch.setattr(mod, "export", lambda *a: seen.setdefault("args", a))
monkeypatch.setattr(sys, "argv", ["export"])
assert mod.main() == 0
revision, base_revision = seen["args"][-2:]
assert revision == _PINNED_REVISIONS[CKPT]
assert base_revision == _PINNED_REVISIONS[BASE]
assert SHA.fullmatch(revision) and SHA.fullmatch(base_revision)
@pytest.mark.parametrize(
"argv",
[
["--revision", "main"],
["--base-revision", "v1.0"],
["--model-id", "someone/unpinned-model"],
],
)
def test_export_rejects_floating_or_missing_revisions(monkeypatch, argv):
mod = _load_script()
monkeypatch.setattr(mod, "export", lambda *a: pytest.fail("export ran unpinned"))
monkeypatch.setattr(sys, "argv", ["export", *argv])
with pytest.raises(SystemExit, match="40-hex commit SHA"):
mod.main()
def test_build_core_threads_both_revisions(monkeypatch):
mod = _load_script()
calls: list[tuple] = []
def fake_download(repo, filename, **kwargs):
calls.append(("download", repo, kwargs.get("revision")))
return "merged.pt"
keys = ("encoder_state_dict", "token_head_state_dict", "span_conv_state_dict")
monkeypatch.setitem(
sys.modules, "torch", types.SimpleNamespace(load=lambda *a, **k: {k: {} for k in keys})
)
monkeypatch.setitem(
sys.modules, "huggingface_hub", types.SimpleNamespace(hf_hub_download=fake_download)
)
class _Part:
def load_state_dict(self, sd, strict):
return [], []
class _Core:
def __init__(self, model_name, **kwargs):
calls.append(("encoder", model_name, kwargs.get("revision")))
self.encoder = self.token_head = self.span_conv = _Part()
def eval(self):
return self
from headroom.transforms import kompress_compressor as kc
monkeypatch.setattr(kc, "_get_model_class", lambda: _Core)
mod._build_core(CKPT, "a" * 40, "b" * 40)
assert calls == [("download", CKPT, "a" * 40), ("encoder", BASE, "b" * 40)]