mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(kompress): pin ModernBERT tokenizer and encoder revisions (#3808)
## Description The Kompress ONNX artifacts are pinned to a commit SHA in `_PINNED_REVISIONS`, but the tokenizer they were exported against (`answerdotai/ModernBERT-base`) was loaded from the floating default ref, and so was the torch-path encoder base. The export script that produces those artifacts also loaded both the Kompress checkpoint and the tokenizer from floating refs. An upstream change could shift token ids under the shipped weights, or produce an artifact production never loads, with no code change on our side. Revised 2026-10-05 after review: the ONNX export script now loads the checkpoint, encoder and tokenizer only at immutable commits defaulting to the production pins, and cache-only PyTorch loads no longer fetch the encoder. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `answerdotai/ModernBERT-base @ 8949b909ec900327062f0ebf497f51aef5e6f0c8` (the snapshot the export cache holds; upstream `main` at the time) to `_PINNED_REVISIONS`. - `_load_modernbert_tokenizer` resolves the pin via `_resolve_revision` and passes `revision=` on both the cache-only and download paths. `HEADROOM_HF_PIN=off` floats it exactly as it floats the weights. - `HeadroomCompressorModel` takes `revision=` and `allow_download=`; it passes the resolved revision and `local_files_only=not allow_download` to `AutoModel.from_pretrained`. `_load_kompress_pytorch` forwards `allow_download` and turns a missing encoder snapshot into `KompressModelNotCached` in cache-only mode (Devin finding). - `scripts/export_kompress_v2_onnx.py`: new `--revision` (checkpoint) and `--base-revision` (ModernBERT encoder + tokenizer). Both default to the production pins in `_PINNED_REVISIONS` (not affected by `HEADROOM_HF_PIN`); anything other than a 40-hex commit, or an unpinned `--model-id` with no `--revision`, is rejected. Every Hub/Transformers load (`hf_hub_download`, encoder, `AutoTokenizer`) passes them. A model upgrade is now a reviewed pin bump plus a regenerated artifact. No behaviour change for a warm cache at the same snapshot; a cache populated from a different revision now downloads the pinned one (when downloads are allowed) instead of using whatever is present. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) — on touched files, plus `ruff format --check` - [x] Type checking passes (`mypy headroom`) — `mypy headroom/transforms/kompress_compressor.py` - [x] New tests added for new functionality - [ ] Manual testing performed New tests (all fail on the pre-fix code, pass now; no network): - `tests/test_scripts/test_export_kompress_v2_onnx_pins.py`: AST check that every `from_pretrained` / `hf_hub_download` / `snapshot_download` / model-class load in the export script passes `revision=`; defaults are the 40-hex production pins even with `HEADROOM_HF_PIN=off`; `main`, a tag, or an unpinned repo are rejected; `_build_core` threads both revisions. - `tests/test_hf_revision_pinning.py`: tokenizer pin present / cache-only path / download path / bypass, and `test_cache_only_pytorch_load_does_not_fetch_encoder`. ### Test Output ```text $ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --frozen --extra dev --extra proxy pytest -q tests/test_hf_revision_pinning.py tests/test_scripts/test_export_kompress_v2_onnx_pins.py tests/test_kompress_preload_deferral.py tests/test_transforms/test_kompress_compressor.py 78 passed in 62.67s # With the source changes reverted: 7 failed, 15 passed (all 7 new tests fail) ``` ## Real Behavior Proof - Environment: macOS, Python 3.14, `uv run --frozen --extra dev --extra proxy`. - Exact command / steps: the pytest command above; tests drive the real loader and export entry points with Hub/torch boundaries faked. - Observed result: every Hub load in the export path receives a 40-hex revision; cache-only PyTorch load raises `KompressModelNotCached` without attempting a download. - Not tested: a real ONNX export / upload against the Hub (needs torch + HF write auth); no artifact is regenerated by this PR. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: n/a. - Stable/default behavior changed: the ModernBERT tokenizer/encoder load at the pinned commit instead of the floating ref; cache-only PyTorch loads defer instead of downloading the encoder. - Kill switch / disable path: `HEADROOM_HF_PIN=off` floats the runtime loads, as for every other pinned repo. - Unsafe override required: no. - Qualification impact: none expected; the pin equals the snapshot the artifacts were exported against. - Rollback path: revert this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Uploading a newly exported artifact creates a new commit in the model repo; production picks it up only after the `chopratejas/kompress-v2-base` pin is bumped in a reviewed PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
JD Davis
parent
84f0e84dcc
commit
2de582875b
@@ -71,6 +71,10 @@ def onnx_thread_spinning_enabled() -> bool:
|
||||
_PINNED_REVISIONS: dict[str, str] = {
|
||||
# chopratejas/kompress-v2-base @ 2026-06-10
|
||||
"chopratejas/kompress-v2-base": "b1563631b35bfdcee37587ad530147497d820d4c",
|
||||
# Kompress tokenizer + torch-path encoder base. The ONNX artifacts were
|
||||
# exported against this snapshot; a silently updated tokenizer would shift
|
||||
# token ids under the shipped weights.
|
||||
"answerdotai/ModernBERT-base": "8949b909ec900327062f0ebf497f51aef5e6f0c8",
|
||||
"chopratejas/technique-router-onnx": "27b0b4bfa510a1cff66d888072c0b807082721a8",
|
||||
"chopratejas/siglip-image-encoder-onnx": "d0a9fbd66d4bd8c761bff592d44831f7c2ae184e",
|
||||
# Third-party repo — pinning matters most here.
|
||||
|
||||
@@ -30,6 +30,7 @@ from typing import Any, Literal
|
||||
from ..config import TransformResult
|
||||
from ..onnx_runtime import (
|
||||
ONNX_CPU_ARENA_ENV,
|
||||
_resolve_revision,
|
||||
create_cpu_session_options,
|
||||
hf_entry_known_absent,
|
||||
hf_hub_download_local_first,
|
||||
@@ -600,9 +601,22 @@ def _get_model_class() -> type:
|
||||
class HeadroomCompressorModel(nn.Module):
|
||||
"""Dual-head ModernBERT: token classification + span importance CNN."""
|
||||
|
||||
def __init__(self, model_name: str = "answerdotai/ModernBERT-base"):
|
||||
def __init__(
|
||||
self,
|
||||
model_name: str = "answerdotai/ModernBERT-base",
|
||||
*,
|
||||
revision: str | None = None,
|
||||
allow_download: bool = True,
|
||||
):
|
||||
super().__init__()
|
||||
self.encoder = AutoModel.from_pretrained(model_name, attn_implementation="eager")
|
||||
# Same pin as the tokenizer: the fine-tuned heads were trained on
|
||||
# this encoder snapshot. Unknown repos resolve to None (floating).
|
||||
self.encoder = AutoModel.from_pretrained(
|
||||
model_name,
|
||||
attn_implementation="eager",
|
||||
revision=_resolve_revision(model_name, revision),
|
||||
local_files_only=not allow_download,
|
||||
)
|
||||
hidden_size = self.encoder.config.hidden_size # 768
|
||||
|
||||
# Head 1: Token keep/discard
|
||||
@@ -831,6 +845,9 @@ def _load_kompress_onnx(
|
||||
return model, tokenizer, backend
|
||||
|
||||
|
||||
_MODERNBERT_TOKENIZER_REPO = "answerdotai/ModernBERT-base"
|
||||
|
||||
|
||||
def _load_modernbert_tokenizer(auto_tokenizer: Any, *, allow_download: bool) -> Any:
|
||||
"""Load the ModernBERT tokenizer, cache-only when ``allow_download`` is False.
|
||||
|
||||
@@ -845,14 +862,25 @@ def _load_modernbert_tokenizer(auto_tokenizer: Any, *, allow_download: bool) ->
|
||||
changes whether the Hub is consulted to confirm what is already on disk.
|
||||
Mirrors ``onnx_runtime.hf_hub_download_local_first``, which the ONNX half of
|
||||
this loader already uses.
|
||||
|
||||
The load is pinned to the same immutable revision as the model artifacts
|
||||
(``onnx_runtime._PINNED_REVISIONS``): the ONNX export was produced against
|
||||
that tokenizer snapshot, so a floating ``main`` could shift token ids under
|
||||
the shipped weights without any code change. ``HEADROOM_HF_PIN=off`` floats
|
||||
it, exactly as for the weights.
|
||||
"""
|
||||
revision = _resolve_revision(_MODERNBERT_TOKENIZER_REPO, None)
|
||||
try:
|
||||
return auto_tokenizer.from_pretrained("answerdotai/ModernBERT-base", local_files_only=True)
|
||||
return auto_tokenizer.from_pretrained(
|
||||
_MODERNBERT_TOKENIZER_REPO, revision=revision, local_files_only=True
|
||||
)
|
||||
except _NOT_CACHED_ERRORS as exc:
|
||||
if not allow_download:
|
||||
raise KompressModelNotCached("answerdotai/ModernBERT-base") from exc
|
||||
# Genuine cache miss and downloading is permitted: fetch it.
|
||||
return auto_tokenizer.from_pretrained("answerdotai/ModernBERT-base", local_files_only=False)
|
||||
raise KompressModelNotCached(_MODERNBERT_TOKENIZER_REPO) from exc
|
||||
# Genuine cache miss and downloading is permitted: fetch it at the pin.
|
||||
return auto_tokenizer.from_pretrained(
|
||||
_MODERNBERT_TOKENIZER_REPO, revision=revision, local_files_only=False
|
||||
)
|
||||
|
||||
|
||||
# Sub-state-dict keys inside a merged v2-style checkpoint (see
|
||||
@@ -971,7 +999,12 @@ def _load_kompress_pytorch(
|
||||
logger.info("Downloading Kompress PyTorch model from %s ...", model_id)
|
||||
|
||||
HeadroomCompressorModel = _get_model_class()
|
||||
model = HeadroomCompressorModel()
|
||||
try:
|
||||
model = HeadroomCompressorModel(allow_download=allow_download)
|
||||
except _NOT_CACHED_ERRORS as exc:
|
||||
if allow_download:
|
||||
raise
|
||||
raise KompressModelNotCached(_MODERNBERT_TOKENIZER_REPO) from exc
|
||||
|
||||
_load_pytorch_weights(model, model_id, allow_download=allow_download)
|
||||
|
||||
|
||||
@@ -31,12 +31,22 @@ Usage
|
||||
|
||||
# Convert, verify, and upload back to the HF repo (needs `huggingface-cli login`):
|
||||
python scripts/export_kompress_v2_onnx.py --model-id chopratejas/kompress-v2-base --upload
|
||||
|
||||
Reproducibility
|
||||
---------------
|
||||
Every Hub load is pinned to an immutable commit: the checkpoint to
|
||||
``--revision`` and the ModernBERT encoder + tokenizer to ``--base-revision``.
|
||||
Both default to the production pins in ``headroom.onnx_runtime._PINNED_REVISIONS``
|
||||
(``HEADROOM_HF_PIN`` does not apply here), so the exported artifact is built
|
||||
from exactly what production loads. Upgrading a model is a reviewed pin bump
|
||||
plus a regenerated artifact, never ambient Hub state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
@@ -47,8 +57,24 @@ logger = logging.getLogger("export_kompress_v2_onnx")
|
||||
BASE_MODEL = "answerdotai/ModernBERT-base"
|
||||
DEFAULT_MODEL_ID = "chopratejas/kompress-v2-base"
|
||||
|
||||
_COMMIT_SHA = re.compile(r"[0-9a-f]{40}")
|
||||
|
||||
def _build_core(model_id: str):
|
||||
|
||||
def _pinned_revision(repo_id: str, revision: str | None) -> str:
|
||||
"""Return an immutable commit for ``repo_id``: ``revision`` if given, else
|
||||
the production pin. Branches, tags and unpinned repos are rejected."""
|
||||
from headroom.onnx_runtime import _PINNED_REVISIONS
|
||||
|
||||
revision = revision or _PINNED_REVISIONS.get(repo_id)
|
||||
if revision is None or not _COMMIT_SHA.fullmatch(revision):
|
||||
raise SystemExit(
|
||||
f"{repo_id}: need a 40-hex commit SHA (got {revision!r}). "
|
||||
"Pass one explicitly or pin the repo in headroom.onnx_runtime._PINNED_REVISIONS."
|
||||
)
|
||||
return revision
|
||||
|
||||
|
||||
def _build_core(model_id: str, revision: str, base_revision: str):
|
||||
"""Instantiate HeadroomCompressorModel and load the merged v2 weights.
|
||||
|
||||
The v2 repo's ``model.safetensors`` is the *unmerged* PEFT structure
|
||||
@@ -67,7 +93,7 @@ def _build_core(model_id: str):
|
||||
|
||||
from headroom.transforms.kompress_compressor import _get_model_class
|
||||
|
||||
ckpt_path = hf_hub_download(model_id, "merged.pt")
|
||||
ckpt_path = hf_hub_download(model_id, "merged.pt", revision=revision)
|
||||
ckpt = torch.load(ckpt_path, map_location="cpu")
|
||||
for key in ("encoder_state_dict", "token_head_state_dict", "span_conv_state_dict"):
|
||||
if key not in ckpt:
|
||||
@@ -76,7 +102,7 @@ def _build_core(model_id: str):
|
||||
"This script targets the v2 'merged' checkpoint format."
|
||||
)
|
||||
|
||||
core = _get_model_class()(model_name=BASE_MODEL)
|
||||
core = _get_model_class()(model_name=BASE_MODEL, revision=base_revision)
|
||||
|
||||
def _strict_load(module, sd, label: str) -> None:
|
||||
missing, unexpected = module.load_state_dict(sd, strict=False)
|
||||
@@ -115,11 +141,18 @@ def _export_wrapper(core):
|
||||
return ExportWrapper(core).eval()
|
||||
|
||||
|
||||
def export(model_id: str, out_path: Path, opset: int, precision: str) -> None:
|
||||
def export(
|
||||
model_id: str,
|
||||
out_path: Path,
|
||||
opset: int,
|
||||
precision: str,
|
||||
revision: str,
|
||||
base_revision: str,
|
||||
) -> None:
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
core = _build_core(model_id)
|
||||
core = _build_core(model_id, revision, base_revision)
|
||||
wrapper = _export_wrapper(core)
|
||||
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
@@ -167,15 +200,15 @@ def export(model_id: str, out_path: Path, opset: int, precision: str) -> None:
|
||||
)
|
||||
trace_target.unlink(missing_ok=True)
|
||||
|
||||
_verify(model_id, core, out_path, np, torch)
|
||||
_verify(core, out_path, base_revision, np, torch)
|
||||
|
||||
|
||||
def _verify(model_id: str, core, out_path: Path, np, torch) -> None:
|
||||
def _verify(core, out_path: Path, base_revision: str, np, torch) -> None:
|
||||
"""Compare ONNX scores against PyTorch get_scores on a real tokenized sample."""
|
||||
import onnxruntime as ort
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(BASE_MODEL)
|
||||
tok = AutoTokenizer.from_pretrained(BASE_MODEL, revision=base_revision)
|
||||
sample = (
|
||||
"The proxy compresses tool outputs before they reach the model. "
|
||||
"Errors and stack traces should survive; boilerplate should not. "
|
||||
@@ -236,6 +269,16 @@ def upload(model_id: str, out_path: Path) -> None:
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("--model-id", default=DEFAULT_MODEL_ID)
|
||||
ap.add_argument(
|
||||
"--revision",
|
||||
default=None,
|
||||
help="Checkpoint commit SHA. Defaults to the production pin for --model-id.",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--base-revision",
|
||||
default=None,
|
||||
help=f"{BASE_MODEL} commit SHA (encoder + tokenizer). Defaults to the production pin.",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--precision",
|
||||
choices=["fp32", "int8"],
|
||||
@@ -256,8 +299,12 @@ def main() -> int:
|
||||
)
|
||||
args = ap.parse_args()
|
||||
|
||||
revision = _pinned_revision(args.model_id, args.revision)
|
||||
base_revision = _pinned_revision(BASE_MODEL, args.base_revision)
|
||||
logger.info("Pins: %s@%s, %s@%s", args.model_id, revision, BASE_MODEL, base_revision)
|
||||
|
||||
out_path = args.out or Path(f"onnx/kompress-{args.precision}.onnx")
|
||||
export(args.model_id, out_path, args.opset, args.precision)
|
||||
export(args.model_id, out_path, args.opset, args.precision, revision, base_revision)
|
||||
if args.upload:
|
||||
upload(args.model_id, out_path)
|
||||
return 0
|
||||
|
||||
@@ -51,3 +51,99 @@ def test_pin_can_be_disabled_via_env(monkeypatch, value):
|
||||
def test_pin_disabled_still_respects_explicit_revision(monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
|
||||
assert _resolve_revision("chopratejas/kompress-v2-base", "abc123") == "abc123"
|
||||
|
||||
|
||||
def test_modernbert_tokenizer_base_is_pinned():
|
||||
# The Kompress tokenizer (and torch-path encoder) load from this repo; the
|
||||
# ONNX export was produced against the pinned snapshot, so it must not float.
|
||||
assert "answerdotai/ModernBERT-base" in _PINNED_REVISIONS
|
||||
|
||||
|
||||
def test_tokenizer_loader_passes_pinned_revision(monkeypatch):
|
||||
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
|
||||
from headroom.transforms import kompress_compressor as kc
|
||||
|
||||
calls: list[dict] = []
|
||||
|
||||
class _Tok:
|
||||
@staticmethod
|
||||
def from_pretrained(repo, **kwargs):
|
||||
calls.append({"repo": repo, **kwargs})
|
||||
return object()
|
||||
|
||||
kc._load_modernbert_tokenizer(_Tok, allow_download=True)
|
||||
assert calls == [
|
||||
{
|
||||
"repo": "answerdotai/ModernBERT-base",
|
||||
"revision": _PINNED_REVISIONS["answerdotai/ModernBERT-base"],
|
||||
"local_files_only": True,
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_tokenizer_loader_downloads_at_pin_on_cache_miss(monkeypatch):
|
||||
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
|
||||
from headroom.transforms import kompress_compressor as kc
|
||||
|
||||
calls: list[dict] = []
|
||||
|
||||
class _Tok:
|
||||
@staticmethod
|
||||
def from_pretrained(repo, **kwargs):
|
||||
calls.append({"repo": repo, **kwargs})
|
||||
if kwargs.get("local_files_only"):
|
||||
raise OSError("not cached")
|
||||
return object()
|
||||
|
||||
kc._load_modernbert_tokenizer(_Tok, allow_download=True)
|
||||
assert [c["local_files_only"] for c in calls] == [True, False]
|
||||
assert {c["revision"] for c in calls} == {_PINNED_REVISIONS["answerdotai/ModernBERT-base"]}
|
||||
|
||||
|
||||
def test_tokenizer_loader_floats_when_pin_disabled(monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
|
||||
from headroom.transforms import kompress_compressor as kc
|
||||
|
||||
calls: list[dict] = []
|
||||
|
||||
class _Tok:
|
||||
@staticmethod
|
||||
def from_pretrained(repo, **kwargs):
|
||||
calls.append(kwargs)
|
||||
return object()
|
||||
|
||||
kc._load_modernbert_tokenizer(_Tok, allow_download=True)
|
||||
assert calls[0]["revision"] is None
|
||||
|
||||
|
||||
def test_cache_only_pytorch_load_does_not_fetch_encoder(monkeypatch):
|
||||
# A cache-only load must not download the pinned encoder on the request
|
||||
# path; a missing snapshot defers the load like any other cache miss.
|
||||
import sys
|
||||
import types
|
||||
|
||||
from headroom.transforms import kompress_compressor as kc
|
||||
|
||||
calls: list[dict] = []
|
||||
|
||||
class _AutoModel:
|
||||
@staticmethod
|
||||
def from_pretrained(repo, **kwargs):
|
||||
calls.append(kwargs)
|
||||
if kwargs.get("local_files_only"):
|
||||
raise OSError("pinned encoder snapshot not cached")
|
||||
raise AssertionError("cache-only load reached the network")
|
||||
|
||||
nn = types.SimpleNamespace(Module=object)
|
||||
monkeypatch.setitem(sys.modules, "torch", types.SimpleNamespace(nn=nn))
|
||||
monkeypatch.setitem(sys.modules, "torch.nn", nn)
|
||||
monkeypatch.setitem(
|
||||
sys.modules,
|
||||
"transformers",
|
||||
types.SimpleNamespace(AutoModel=_AutoModel, AutoTokenizer=object),
|
||||
)
|
||||
monkeypatch.setattr(kc, "_kompress_cache", {})
|
||||
|
||||
with pytest.raises(kc.KompressModelNotCached):
|
||||
kc._load_kompress_pytorch("chopratejas/kompress-v2-base", allow_download=False)
|
||||
assert [c["local_files_only"] for c in calls] == [True]
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
"""The Kompress ONNX export must build from the same immutable Hub snapshots
|
||||
production loads, so an exported artifact can never mix floating inputs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import importlib.util
|
||||
import re
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.onnx_runtime import _PINNED_REVISIONS
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[2] / "scripts" / "export_kompress_v2_onnx.py"
|
||||
SHA = re.compile(r"[0-9a-f]{40}")
|
||||
BASE = "answerdotai/ModernBERT-base"
|
||||
CKPT = "chopratejas/kompress-v2-base"
|
||||
|
||||
|
||||
def _load_script():
|
||||
spec = importlib.util.spec_from_file_location("export_kompress_v2_onnx", SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
def _call_name(call: ast.Call) -> str:
|
||||
func = call.func
|
||||
if isinstance(func, ast.Attribute):
|
||||
return func.attr
|
||||
if isinstance(func, ast.Name):
|
||||
return func.id
|
||||
if isinstance(func, ast.Call): # _get_model_class()(...)
|
||||
return _call_name(func) + "()"
|
||||
return ""
|
||||
|
||||
|
||||
def test_every_hub_load_in_export_script_passes_revision():
|
||||
loaders = {"from_pretrained", "hf_hub_download", "snapshot_download", "_get_model_class()"}
|
||||
calls = [
|
||||
node
|
||||
for node in ast.walk(ast.parse(SCRIPT.read_text()))
|
||||
if isinstance(node, ast.Call) and _call_name(node) in loaders
|
||||
]
|
||||
assert {_call_name(c) for c in calls} >= {
|
||||
"from_pretrained",
|
||||
"hf_hub_download",
|
||||
"_get_model_class()",
|
||||
}
|
||||
unpinned = [
|
||||
f"line {c.lineno}: {_call_name(c)}"
|
||||
for c in calls
|
||||
if not any(kw.arg == "revision" for kw in c.keywords)
|
||||
]
|
||||
assert unpinned == []
|
||||
|
||||
|
||||
def test_export_defaults_to_production_pins_even_with_pinning_disabled(monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
|
||||
mod = _load_script()
|
||||
seen: dict = {}
|
||||
monkeypatch.setattr(mod, "export", lambda *a: seen.setdefault("args", a))
|
||||
monkeypatch.setattr(sys, "argv", ["export"])
|
||||
|
||||
assert mod.main() == 0
|
||||
revision, base_revision = seen["args"][-2:]
|
||||
assert revision == _PINNED_REVISIONS[CKPT]
|
||||
assert base_revision == _PINNED_REVISIONS[BASE]
|
||||
assert SHA.fullmatch(revision) and SHA.fullmatch(base_revision)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"argv",
|
||||
[
|
||||
["--revision", "main"],
|
||||
["--base-revision", "v1.0"],
|
||||
["--model-id", "someone/unpinned-model"],
|
||||
],
|
||||
)
|
||||
def test_export_rejects_floating_or_missing_revisions(monkeypatch, argv):
|
||||
mod = _load_script()
|
||||
monkeypatch.setattr(mod, "export", lambda *a: pytest.fail("export ran unpinned"))
|
||||
monkeypatch.setattr(sys, "argv", ["export", *argv])
|
||||
with pytest.raises(SystemExit, match="40-hex commit SHA"):
|
||||
mod.main()
|
||||
|
||||
|
||||
def test_build_core_threads_both_revisions(monkeypatch):
|
||||
mod = _load_script()
|
||||
calls: list[tuple] = []
|
||||
|
||||
def fake_download(repo, filename, **kwargs):
|
||||
calls.append(("download", repo, kwargs.get("revision")))
|
||||
return "merged.pt"
|
||||
|
||||
keys = ("encoder_state_dict", "token_head_state_dict", "span_conv_state_dict")
|
||||
monkeypatch.setitem(
|
||||
sys.modules, "torch", types.SimpleNamespace(load=lambda *a, **k: {k: {} for k in keys})
|
||||
)
|
||||
monkeypatch.setitem(
|
||||
sys.modules, "huggingface_hub", types.SimpleNamespace(hf_hub_download=fake_download)
|
||||
)
|
||||
|
||||
class _Part:
|
||||
def load_state_dict(self, sd, strict):
|
||||
return [], []
|
||||
|
||||
class _Core:
|
||||
def __init__(self, model_name, **kwargs):
|
||||
calls.append(("encoder", model_name, kwargs.get("revision")))
|
||||
self.encoder = self.token_head = self.span_conv = _Part()
|
||||
|
||||
def eval(self):
|
||||
return self
|
||||
|
||||
from headroom.transforms import kompress_compressor as kc
|
||||
|
||||
monkeypatch.setattr(kc, "_get_model_class", lambda: _Core)
|
||||
|
||||
mod._build_core(CKPT, "a" * 40, "b" * 40)
|
||||
assert calls == [("download", CKPT, "a" * 40), ("encoder", BASE, "b" * 40)]
|
||||
Reference in New Issue
Block a user