mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(relevance): load the default embedder from one pinned snapshot (#4152)
## Description The supply-chain pin on the default relevance embedder never took effect. `EmbeddingScorer` passed `revision="52398278…"` to fastembed's `TextEmbedding`, which accepts it and then drops it before `snapshot_download` (fastembed 0.8.0 and 0.9.0: `OnnxTextEmbedding.__init__` calls `download_model(...)` without `**kwargs`). So the model floated on `Qdrant/bge-small-en-v1.5-onnx-Q@main`, cached under the system temp dir. Today the floating `main` (`aa8f8b0`) differs from the pin only in `README.md`, so no unreviewed weights have shipped. The pin just wasn't protecting anything. This PR makes the Python default model load the pinned snapshot and adds tests that would have caught the dropped revision. The Rust `EmbeddingScorer::try_new` change that was in this PR is now its own PR, stacked on this one: #4190. No linked issue. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/onnx_runtime.py`: pin `Qdrant/bge-small-en-v1.5-onnx-Q` at `52398278842ec682c6f32300af41344b1c0b0bb2` in `_PINNED_REVISIONS`, so `HEADROOM_HF_PIN=off` applies to it like every other pinned repo. - `headroom/relevance/embedding.py`: - The default model resolves its 5 files through `hf_hub_download_local_first`, which is pinned, cache-first and guarded on a miss. - fastembed then gets only the snapshot directory as `specific_model_path`. That construction runs with the Hub forced offline, so a future fastembed that ignored the path would fail instead of downloading an unpinned copy. - Other catalog models keep the existing `_load_text_embedding` path. - The air-gapped error for the default model names the exact snapshot and the command that seeds it, and says a fastembed cache is no longer read (details under Release note). - The private `_DEFAULT_MODEL_PINNED_REVISION` / `_pinned_revision` are removed. - `_load_text_embedding` now types its kwargs `dict[str, Any]`, fixing 8 mypy errors that only appear with fastembed installed. - `pyproject.toml` / `uv.lock`: `fastembed>=0.6.0`. `specific_model_path` first appears in fastembed v0.6.0; it isn't in v0.4.x or v0.5.x. The lock still resolves 0.8.0; only the `requires-dist` specifier changes. - `.github/workflows/ci.yml` (`test-extras`): - The prefetch warms the pinned snapshot through the loader's own resolver, and the cache entry's key carries the SHA. The old prefetch warmed fastembed's own cache, which the offline relevance tests would now miss: `conftest.py:199` turns that into silent skips. - The job runs the new test file, and the unused `FASTEMBED_CACHE_PATH` is gone. - Tests: - New `tests/test_fastembed_pinned_snapshot.py`: - the pinned revision reaches every file download; - fastembed gets only the snapshot directory, built with the Hub forced offline; - `HEADROOM_HF_PIN=off` floats; - files resolved from two different snapshots are refused; - a cold cache while offline degrades without a download, and the error names `repo@sha` and `--revision <sha>`; - real fastembed skips the Hub when given `specific_model_path`. - `tests/test_stateless_writers.py`: the three `_pinned_revision` tests are removed with the helper; `tests/test_hf_revision_pinning.py` covers `_resolve_revision`. - `tests/test_offline_egress_chokepoint.py`: `relevance/embedding.py`'s cache-only count goes from 1 to 2 for the `_load_default_model` construction. ## Release note > **relevance:** the default embedder (`BAAI/bge-small-en-v1.5`) now loads from the pinned snapshot `Qdrant/bge-small-en-v1.5-onnx-Q@52398278842ec682c6f32300af41344b1c0b0bb2` in the HuggingFace hub cache (`HF_HUB_CACHE` / `HF_HOME/hub` / `~/.cache/huggingface/hub`), not from fastembed's cache. Existing installs download ~67 MB once. Air-gapped hosts that pre-seeded the fastembed cache must re-seed, e.g. `hf download Qdrant/bge-small-en-v1.5-onnx-Q model_optimized.onnx tokenizer.json tokenizer_config.json special_tokens_map.json config.json --revision 52398278842ec682c6f32300af41344b1c0b0bb2`. Without that, relevance scoring falls back to BM25, and the error message says how to re-seed. `CHANGELOG.md` is generated from the PR title, so this text is here for whoever writes the release notes. ## Dependency Change (`fastembed>=0.4.0` → `>=0.6.0`) This raises the minimum version of an existing dependency; no package is added. `uv.lock` still resolves `fastembed 0.8.0`. The only lock change is the `requires-dist` specifier, so the locked install set is byte-for-byte the same. - **Why this package:** it's already the `relevance` extra's embedder (since Stage 3c.1). This PR adds no package and swaps none. - **Who maintains it:** Qdrant (`qdrant/fastembed`), Apache-2.0, actively released (0.6.0 on 2025-02-26 through 0.9.0 on 2026-10-07). The GitHub Advisory Database has no advisories for `fastembed` (pip). - **Install surface:** unchanged for the locked install (0.8.0). Installs that resolve their own versions lose access to 0.4.x/0.5.x. Runtime network shrinks: given `specific_model_path`, fastembed makes no Hub call, and the only download is Headroom's own pinned, offline-guarded `hf_hub_download_local_first`. - **Why this version:** `specific_model_path` on `TextEmbedding` first appears in v0.6.0, and the pinned-snapshot load depends on it. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ HOME=$(mktemp -d) pytest -q tests/test_fastembed_pinned_snapshot.py tests/test_offline_egress_chokepoint.py \ tests/test_relevance.py tests/test_relevance_extra.py tests/test_stateless_writers.py tests/test_hf_revision_pinning.py 7 failed, 209 passed, 3 skipped # The 7 failures are TestOtlpExporterOffline / TestLangfuseExporterOffline / TestStartupRefusal / # TestHeadroomCloudCompressionOffline: the OTel and Cloud extras aren't in this venv. The same 7 fail on # origin/main (f1bc2609) here. $ mypy headroom # with fastembed 0.8.0 installed Success: no issues found in 569 source files $ ruff check . # All checks passed! $ ruff format --check . # 1790 files already formatted ``` ## Real Behavior Proof - Exact command / steps: additional maintainer verification on Windows/Python 3.12.13: python -m pytest tests/test_fastembed_pinned_snapshot.py tests/test_offline_egress_chokepoint.py tests/test_stateless_writers.py -q. - Observed result: the additional maintainer run passed all 160 selected tests onb3eb7f349c, using the available development native extension for startup; no fresh Rust implementation qualification is claimed. - Environment: macOS arm64, repo `.venv` (fastembed 0.8.0 = `uv.lock`), rebased on `origin/main` `f1bc2609`. - Default load: `EmbeddingScorer().score("authentication failed for user admin", "login error")` loads from `~/.cache/huggingface/hub/models--Qdrant--bge-small-en-v1.5-onnx-Q/snapshots/52398278842ec682c6f32300af41344b1c0b0bb2` and scores 0.7389. `origin/main` loads `<tmp>/fastembed_cache/…/snapshots/aa8f8b06…` (the floating `main`, under `$TMPDIR`). - Air-gapped re-seed, from an empty `HF_HOME`: 1. `HEADROOM_OFFLINE=1` gives `Embedding model 'BAAI/bge-small-en-v1.5' is unavailable: HEADROOM_OFFLINE is set: … Pre-seed the HuggingFace hub cache with Qdrant/bge-small-en-v1.5-onnx-Q@52398278… (`hf download … --revision 52398278…`), or use the BM25-only scorer. A fastembed cache seeded for an older Headroom is not read for this model.` 2. I ran the `hf download` command from that message as printed. 3. `HEADROOM_OFFLINE=1 HF_HUB_OFFLINE=1` then loads and scores 0.7389. 4. With `HEADROOM_HF_PIN=off`, the message names `@main` and `--revision main` instead. - Not tested: Windows, and Linux outside CI. fastembed 0.6.x/0.7.x at runtime (the 0.6.0 floor comes from reading the v0.6.0 tag's source). ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A. This is the default code path, with no flag. - Stable/default behavior changed: yes. The default relevance model loads the pinned snapshot from the standard HF hub cache, not fastembed's temp-dir cache, so existing installs download ~67 MB once. The model files are byte-identical to what `main` served; only `README.md` differs. - Kill switch / disable path: `HEADROOM_HF_PIN=off` floats it on `main`. Air-gapped hosts re-seed as described in the Release note. - Unsafe override required: no. - Qualification impact: none expected. Same weights and tokenizer as the current floating `main`. - Rollback path: revert this PR. Nothing persistent changes beyond an extra snapshot in the HF cache. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A. ## Additional Notes - `_load_text_embedding` was typed `kwargs: dict[str, str]`, which made mypy report 8 `arg-type` errors on `TextEmbedding(**kwargs)` whenever fastembed is installed in the type-checking environment. CI's mypy run doesn't install fastembed, so it never saw them. - Follow-ups, not in this PR: Rust's Kompress and tokenizer Hub fetches are still unpinned (there's no Rust `HEADROOM_HF_PIN`). Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
This commit is contained in:
co-authored by
Kenneth Wong
parent
62b3bedaba
commit
f380692b46
@@ -443,8 +443,6 @@ jobs:
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
env:
|
||||
FASTEMBED_CACHE_PATH: ${{ github.workspace }}/.fastembed-cache
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/setup-python@v7
|
||||
@@ -459,8 +457,11 @@ jobs:
|
||||
- name: Cache fastembed model
|
||||
uses: actions/cache@v6
|
||||
with:
|
||||
path: ${{ github.workspace }}/.fastembed-cache
|
||||
key: ${{ runner.os }}-fastembed-bge-small-v1
|
||||
# The default relevance model loads from its pinned snapshot in the
|
||||
# HuggingFace hub cache (headroom/onnx_runtime.py _PINNED_REVISIONS);
|
||||
# the key carries the pinned SHA so a bump re-fetches.
|
||||
path: ~/.cache/huggingface/hub/models--Qdrant--bge-small-en-v1.5-onnx-Q
|
||||
key: ${{ runner.os }}-hf-bge-small-en-v1.5-onnx-Q-52398278842e
|
||||
- name: Download prebuilt wheel
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
@@ -481,7 +482,7 @@ jobs:
|
||||
HF_HUB_DISABLE_TELEMETRY: "1"
|
||||
run: |
|
||||
for i in 1 2 3 4 5; do
|
||||
if python -c "from fastembed import TextEmbedding; TextEmbedding('BAAI/bge-small-en-v1.5')"; then exit 0; fi
|
||||
if python -c "from headroom.relevance.embedding import _resolve_default_model_snapshot as r; print(r())"; then exit 0; fi
|
||||
echo "::warning::fastembed fetch attempt $i failed; backing off"; sleep $((i * 20))
|
||||
done
|
||||
echo "::error::could not fetch fastembed model from HuggingFace"; exit 1
|
||||
@@ -491,7 +492,7 @@ jobs:
|
||||
env:
|
||||
HF_HUB_OFFLINE: "1"
|
||||
TRANSFORMERS_OFFLINE: "1"
|
||||
run: pytest tests/test_relevance.py -v
|
||||
run: pytest tests/test_relevance.py tests/test_fastembed_pinned_snapshot.py -v
|
||||
|
||||
- name: Run HTML extraction and tool reversibility tests
|
||||
run: |
|
||||
|
||||
@@ -81,6 +81,9 @@ _PINNED_REVISIONS: dict[str, str] = {
|
||||
"chopratejas/siglip-image-encoder-onnx": "d0a9fbd66d4bd8c761bff592d44831f7c2ae184e",
|
||||
# Third-party repo — pinning matters most here.
|
||||
"Qdrant/all-MiniLM-L6-v2-onnx": "5f1b8cd78bc4fb444dd171e59b18f3a3af89a079",
|
||||
# Default relevance embedder (BAAI/bge-small-en-v1.5), third-party too.
|
||||
# relevance/embedding.py downloads each of its files at this SHA.
|
||||
"Qdrant/bge-small-en-v1.5-onnx-Q": "52398278842ec682c6f32300af41344b1c0b0bb2",
|
||||
}
|
||||
|
||||
|
||||
|
||||
+118
-47
@@ -1,10 +1,9 @@
|
||||
"""Embedding-based relevance scorer for Headroom SDK.
|
||||
|
||||
This module provides semantic relevance scoring using `fastembed`
|
||||
(BAAI/bge-small-en-v1.5 by default — 33M params, 384 dims, ~30 MB
|
||||
int8-quantized ONNX). Same library + same model used by the Rust
|
||||
SmartCrusher (fastembed-rs crate) so embeddings agree byte-for-byte
|
||||
across the language boundary.
|
||||
(BAAI/bge-small-en-v1.5 by default — 33M params, 384 dims, ~67 MB
|
||||
quantized ONNX), loaded from a pinned Qdrant/bge-small-en-v1.5-onnx-Q
|
||||
snapshot in the HuggingFace hub cache.
|
||||
|
||||
Key features:
|
||||
- Semantic understanding ("errors" matches "failed", "issues")
|
||||
@@ -28,9 +27,13 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
from typing import TYPE_CHECKING
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from headroom.offline import OfflineEgressBlocked, guard_egress
|
||||
from headroom.onnx_runtime import _resolve_revision, hf_hub_download_local_first
|
||||
|
||||
from .base import RelevanceScore, RelevanceScorer
|
||||
|
||||
@@ -60,7 +63,31 @@ if TYPE_CHECKING:
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_text_embedding(kwargs: dict[str, str]) -> TextEmbedding:
|
||||
@contextmanager
|
||||
def _hf_hub_forced_offline() -> Iterator[None]:
|
||||
"""Force the HuggingFace stack offline for the duration of the block.
|
||||
|
||||
huggingface_hub reads HF_HUB_OFFLINE once, at import, into
|
||||
``constants.HF_HUB_OFFLINE``; if it was imported before this call the env
|
||||
var alone changes nothing. Force both, and restore both on exit.
|
||||
"""
|
||||
from huggingface_hub import constants as hf_constants
|
||||
|
||||
previous = os.environ.get("HF_HUB_OFFLINE")
|
||||
previous_constant = hf_constants.HF_HUB_OFFLINE
|
||||
os.environ["HF_HUB_OFFLINE"] = "1"
|
||||
hf_constants.HF_HUB_OFFLINE = True
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
hf_constants.HF_HUB_OFFLINE = previous_constant
|
||||
if previous is None:
|
||||
os.environ.pop("HF_HUB_OFFLINE", None)
|
||||
else:
|
||||
os.environ["HF_HUB_OFFLINE"] = previous
|
||||
|
||||
|
||||
def _load_text_embedding(kwargs: dict[str, Any]) -> TextEmbedding:
|
||||
"""Build a fastembed ``TextEmbedding``, cache first and network second.
|
||||
|
||||
``onnx_runtime.hf_hub_download_local_first`` can hang the air-gap guard on
|
||||
@@ -83,25 +110,12 @@ def _load_text_embedding(kwargs: dict[str, str]) -> TextEmbedding:
|
||||
never sent somewhere it would not otherwise go.
|
||||
"""
|
||||
from fastembed import TextEmbedding
|
||||
from huggingface_hub import constants as hf_constants
|
||||
|
||||
# huggingface_hub reads HF_HUB_OFFLINE once, at import, into
|
||||
# ``constants.HF_HUB_OFFLINE``; if it was imported before this call the
|
||||
# env var alone changes nothing. Force both.
|
||||
previous = os.environ.get("HF_HUB_OFFLINE")
|
||||
previous_constant = hf_constants.HF_HUB_OFFLINE
|
||||
os.environ["HF_HUB_OFFLINE"] = "1"
|
||||
hf_constants.HF_HUB_OFFLINE = True
|
||||
try:
|
||||
return TextEmbedding(**kwargs)
|
||||
with _hf_hub_forced_offline():
|
||||
return TextEmbedding(**kwargs)
|
||||
except Exception as cache_miss: # noqa: BLE001 - any local-lookup failure
|
||||
logger.debug("fastembed cache lookup failed, falling back to network: %s", cache_miss)
|
||||
finally:
|
||||
hf_constants.HF_HUB_OFFLINE = previous_constant
|
||||
if previous is None:
|
||||
os.environ.pop("HF_HUB_OFFLINE", None)
|
||||
else:
|
||||
os.environ["HF_HUB_OFFLINE"] = previous
|
||||
|
||||
guard_egress(
|
||||
f"fastembed weight download for {kwargs['model_name']}",
|
||||
@@ -113,25 +127,77 @@ def _load_text_embedding(kwargs: dict[str, str]) -> TextEmbedding:
|
||||
# Default model name. Same string used by the Rust embedding scorer.
|
||||
DEFAULT_MODEL_NAME = "BAAI/bge-small-en-v1.5"
|
||||
|
||||
# Pinned revision of fastembed's underlying HF repo for the default model
|
||||
# (fastembed resolves "BAAI/bge-small-en-v1.5" -> "qdrant/bge-small-en-v1.5-onnx-q").
|
||||
# fastembed's TextEmbedding signature omits ``revision`` but forwards **kwargs to
|
||||
# huggingface_hub.snapshot_download, so passing it pins the download for
|
||||
# supply-chain integrity. Only the default model is pinned; custom models float.
|
||||
# Set HEADROOM_HF_PIN=off to bypass (mirrors headroom.onnx_runtime pinning).
|
||||
_DEFAULT_MODEL_PINNED_REVISION = "52398278842ec682c6f32300af41344b1c0b0bb2"
|
||||
# The HF repo fastembed resolves DEFAULT_MODEL_NAME to, and every file a
|
||||
# fastembed load of it reads. Its revision is pinned in
|
||||
# ``onnx_runtime._PINNED_REVISIONS`` (HEADROOM_HF_PIN=off floats it).
|
||||
DEFAULT_MODEL_REPO = "Qdrant/bge-small-en-v1.5-onnx-Q"
|
||||
DEFAULT_MODEL_FILES = (
|
||||
"model_optimized.onnx",
|
||||
"tokenizer.json",
|
||||
"tokenizer_config.json",
|
||||
"special_tokens_map.json",
|
||||
"config.json",
|
||||
)
|
||||
|
||||
|
||||
def _pinned_revision(model_name: str) -> str | None:
|
||||
"""Return the pinned HF revision for ``model_name`` (default model only).
|
||||
def _resolve_default_model_snapshot() -> str:
|
||||
"""Return the local directory holding the pinned default-model snapshot.
|
||||
|
||||
Returns ``None`` for custom models or when ``HEADROOM_HF_PIN=off``.
|
||||
fastembed cannot pin this itself: ``TextEmbedding`` accepts ``revision``
|
||||
but drops it before ``snapshot_download`` (fastembed 0.9.0), so passing it
|
||||
left the download floating on the repo's ``main``. Each file is resolved
|
||||
here through ``hf_hub_download_local_first`` instead, which applies the
|
||||
pinned SHA, tries the cache first and guards the network fallback.
|
||||
|
||||
The files land in the standard HuggingFace hub cache rather than
|
||||
fastembed's default under the system temp dir, which macOS purges after a
|
||||
few idle days.
|
||||
|
||||
Raises:
|
||||
OfflineEgressBlocked: ``HEADROOM_OFFLINE`` is set and a file is not cached.
|
||||
RuntimeError: the files resolved to more than one snapshot, which only
|
||||
happens when ``main`` moved mid-download with pinning turned off.
|
||||
"""
|
||||
if os.environ.get("HEADROOM_HF_PIN", "").strip().lower() in ("off", "0", "false", "no"):
|
||||
return None
|
||||
if model_name == DEFAULT_MODEL_NAME:
|
||||
return _DEFAULT_MODEL_PINNED_REVISION
|
||||
return None
|
||||
paths = [
|
||||
Path(hf_hub_download_local_first(DEFAULT_MODEL_REPO, name)) for name in DEFAULT_MODEL_FILES
|
||||
]
|
||||
snapshot_dirs = {path.parent for path in paths}
|
||||
if len(snapshot_dirs) != 1:
|
||||
raise RuntimeError(
|
||||
f"{DEFAULT_MODEL_REPO} files resolved to more than one snapshot: "
|
||||
f"{sorted(str(d) for d in snapshot_dirs)}"
|
||||
)
|
||||
return str(snapshot_dirs.pop())
|
||||
|
||||
|
||||
def _default_model_seed_hint() -> str:
|
||||
"""How to pre-seed DEFAULT_MODEL_NAME on a host that cannot download it.
|
||||
|
||||
Names the exact snapshot the load looks up, because a fastembed cache
|
||||
seeded for an older Headroom no longer satisfies it.
|
||||
"""
|
||||
revision = _resolve_revision(DEFAULT_MODEL_REPO, None) or "main"
|
||||
return (
|
||||
f"Pre-seed the HuggingFace hub cache with {DEFAULT_MODEL_REPO}@{revision} "
|
||||
f"(`hf download {DEFAULT_MODEL_REPO} {' '.join(DEFAULT_MODEL_FILES)} "
|
||||
f"--revision {revision}`), or use the BM25-only scorer. A fastembed cache "
|
||||
"seeded for an older Headroom is not read for this model."
|
||||
)
|
||||
|
||||
|
||||
def _load_default_model() -> TextEmbedding:
|
||||
"""Load DEFAULT_MODEL_NAME from its pinned snapshot, with no fastembed lookup.
|
||||
|
||||
Given ``specific_model_path`` fastembed loads that directory without
|
||||
touching the Hub; the construction still runs with the Hub forced offline
|
||||
so that a fastembed that stopped honouring it fails here instead of
|
||||
downloading an unpinned copy.
|
||||
"""
|
||||
from fastembed import TextEmbedding
|
||||
|
||||
snapshot = _resolve_default_model_snapshot()
|
||||
with _hf_hub_forced_offline():
|
||||
return TextEmbedding(model_name=DEFAULT_MODEL_NAME, specific_model_path=snapshot)
|
||||
|
||||
|
||||
def _cosine_similarity(a, b) -> float:
|
||||
@@ -160,8 +226,8 @@ class EmbeddingScorer(RelevanceScorer):
|
||||
"""Semantic relevance scorer using fastembed (ONNX-backed).
|
||||
|
||||
Default model: BAAI/bge-small-en-v1.5 (33M params, 384 dims).
|
||||
Auto-downloads from HuggingFace Hub on first use (~30 MB
|
||||
int8-quantized ONNX).
|
||||
Auto-downloads its pinned snapshot from HuggingFace Hub on first
|
||||
use (~67 MB quantized ONNX).
|
||||
|
||||
Example:
|
||||
scorer = EmbeddingScorer()
|
||||
@@ -223,23 +289,28 @@ class EmbeddingScorer(RelevanceScorer):
|
||||
)
|
||||
|
||||
if self._model is None:
|
||||
revision = _pinned_revision(self.model_name)
|
||||
kwargs = {"model_name": self.model_name}
|
||||
if revision is not None:
|
||||
# fastembed forwards **kwargs to snapshot_download(revision=...).
|
||||
kwargs["revision"] = revision
|
||||
try:
|
||||
self._model = _load_text_embedding(kwargs)
|
||||
if self.model_name == DEFAULT_MODEL_NAME:
|
||||
self._model = _load_default_model()
|
||||
else:
|
||||
# Other catalog models are not pinned: fastembed resolves
|
||||
# them on the repo's main, cache first.
|
||||
self._model = _load_text_embedding({"model_name": self.model_name})
|
||||
except OfflineEgressBlocked as blocked:
|
||||
# Translate at the boundary that owns the degradation, the way
|
||||
# the Kompress and ONNX loaders do. Public model weights are
|
||||
# not data leaving the box, so what the caller needs to hear
|
||||
# is "this model is not available here and why", not an
|
||||
# air-gap type it has never seen.
|
||||
if self.model_name == DEFAULT_MODEL_NAME:
|
||||
seed_hint = _default_model_seed_hint()
|
||||
else:
|
||||
seed_hint = (
|
||||
"Pre-seed the fastembed cache on this host, or use the BM25-only scorer."
|
||||
)
|
||||
raise RuntimeError(
|
||||
f"Embedding model {self.model_name!r} is unavailable: {blocked}. "
|
||||
"Pre-seed the fastembed/HuggingFace cache on this host, or "
|
||||
"use the BM25-only scorer."
|
||||
f"Embedding model {self.model_name!r} is unavailable: "
|
||||
f"{str(blocked).rstrip('.')}. {seed_hint}"
|
||||
) from None
|
||||
return self._model
|
||||
|
||||
|
||||
+4
-4
@@ -172,13 +172,13 @@ pytorch-mps = [
|
||||
]
|
||||
# Semantic relevance scoring with embeddings.
|
||||
# Uses `fastembed` (BAAI/bge-small-en-v1.5 by default — 33M params,
|
||||
# 384 dims, ~30 MB int8-quantized ONNX). Same library + model used by
|
||||
# the Rust SmartCrusher (`fastembed` crate), giving byte-equal embeddings
|
||||
# across the language boundary. Replaced sentence-transformers in
|
||||
# 384 dims, ~67 MB quantized ONNX), loaded from a pinned
|
||||
# Qdrant/bge-small-en-v1.5-onnx-Q snapshot. Replaced sentence-transformers in
|
||||
# Stage 3c.1 — fastembed is faster (~2-3x), smaller (no torch
|
||||
# dependency), and outranks all-MiniLM-L6-v2 on MTEB by ~6 points.
|
||||
# >=0.6.0 for `specific_model_path`, which the pinned-snapshot load needs.
|
||||
relevance = [
|
||||
"fastembed>=0.4.0; python_version<'3.14' or sys_platform != 'darwin' or platform_machine != 'x86_64'",
|
||||
"fastembed>=0.6.0; python_version<'3.14' or sys_platform != 'darwin' or platform_machine != 'x86_64'",
|
||||
"numpy>=1.24.0",
|
||||
]
|
||||
# Image compression (ML-based routing + OCR)
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
"""The default relevance embedder loads one pinned snapshot.
|
||||
|
||||
fastembed's ``TextEmbedding`` accepts ``revision`` but drops it before
|
||||
``snapshot_download``, so the old pin never reached the Hub and the model
|
||||
floated on ``main``. These tests hold the replacement in place: the pinned
|
||||
SHA reaches every file download and fastembed only gets the resolved
|
||||
directory.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.offline import OFFLINE_ENV
|
||||
from headroom.onnx_runtime import _PINNED_REVISIONS
|
||||
from headroom.relevance import embedding
|
||||
from headroom.relevance.embedding import (
|
||||
DEFAULT_MODEL_FILES,
|
||||
DEFAULT_MODEL_NAME,
|
||||
DEFAULT_MODEL_REPO,
|
||||
EmbeddingScorer,
|
||||
)
|
||||
|
||||
PIN = _PINNED_REVISIONS[DEFAULT_MODEL_REPO]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_hub(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> list[dict[str, object]]:
|
||||
"""Stand in for ``huggingface_hub.hf_hub_download``: record each call and
|
||||
return the path the real cache layout would give for that revision."""
|
||||
import huggingface_hub
|
||||
|
||||
calls: list[dict[str, object]] = []
|
||||
|
||||
def _download(repo_id: str, filename: str, **kwargs: object) -> str:
|
||||
calls.append({"repo_id": repo_id, "filename": filename, **kwargs})
|
||||
commit = kwargs.get("revision") or "main-commit"
|
||||
return str(tmp_path / "snapshots" / str(commit) / filename)
|
||||
|
||||
monkeypatch.setattr(huggingface_hub, "hf_hub_download", _download)
|
||||
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
|
||||
return calls
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_fastembed(monkeypatch: pytest.MonkeyPatch) -> list[dict[str, object]]:
|
||||
"""Install a stub fastembed whose constructor records its kwargs."""
|
||||
constructed: list[dict[str, object]] = []
|
||||
|
||||
class _TextEmbedding:
|
||||
def __init__(self, **kwargs: object) -> None:
|
||||
constructed.append(kwargs)
|
||||
|
||||
monkeypatch.setitem(sys.modules, "fastembed", SimpleNamespace(TextEmbedding=_TextEmbedding))
|
||||
return constructed
|
||||
|
||||
|
||||
def test_every_file_is_downloaded_at_the_pinned_revision(fake_hub, fake_fastembed, tmp_path):
|
||||
EmbeddingScorer()._get_model()
|
||||
|
||||
assert {c["filename"] for c in fake_hub} == set(DEFAULT_MODEL_FILES)
|
||||
assert {c["repo_id"] for c in fake_hub} == {DEFAULT_MODEL_REPO}
|
||||
assert {c["revision"] for c in fake_hub} == {PIN}
|
||||
|
||||
|
||||
def test_fastembed_gets_the_snapshot_directory_not_a_revision(fake_hub, fake_fastembed, tmp_path):
|
||||
EmbeddingScorer()._get_model()
|
||||
|
||||
assert fake_fastembed == [
|
||||
{
|
||||
"model_name": DEFAULT_MODEL_NAME,
|
||||
"specific_model_path": str(tmp_path / "snapshots" / PIN),
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_fastembed_is_constructed_with_the_hub_forced_offline(fake_hub, monkeypatch):
|
||||
"""If fastembed ever stopped honouring ``specific_model_path`` it would
|
||||
try the Hub; forcing it offline makes that a load failure, not an
|
||||
unpinned download."""
|
||||
from huggingface_hub import constants as hf_constants
|
||||
|
||||
seen: list[tuple[str | None, bool]] = []
|
||||
|
||||
class _TextEmbedding:
|
||||
def __init__(self, **kwargs: object) -> None:
|
||||
seen.append((os.environ.get("HF_HUB_OFFLINE"), hf_constants.HF_HUB_OFFLINE))
|
||||
|
||||
monkeypatch.setitem(sys.modules, "fastembed", SimpleNamespace(TextEmbedding=_TextEmbedding))
|
||||
monkeypatch.setenv("HF_HUB_OFFLINE", "0")
|
||||
monkeypatch.setattr(hf_constants, "HF_HUB_OFFLINE", False)
|
||||
|
||||
EmbeddingScorer()._get_model()
|
||||
|
||||
assert seen == [("1", True)]
|
||||
assert os.environ["HF_HUB_OFFLINE"] == "0"
|
||||
assert hf_constants.HF_HUB_OFFLINE is False
|
||||
|
||||
|
||||
def test_pin_off_floats_on_main(fake_hub, fake_fastembed, monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
|
||||
EmbeddingScorer()._get_model()
|
||||
|
||||
assert {c["revision"] for c in fake_hub} == {None}
|
||||
|
||||
|
||||
def test_files_from_two_snapshots_are_refused(monkeypatch, fake_fastembed, tmp_path):
|
||||
import huggingface_hub
|
||||
|
||||
def _download(repo_id: str, filename: str, **kwargs: object) -> str:
|
||||
commit = "a" if filename == DEFAULT_MODEL_FILES[0] else "b"
|
||||
return str(tmp_path / commit / filename)
|
||||
|
||||
monkeypatch.setattr(huggingface_hub, "hf_hub_download", _download)
|
||||
with pytest.raises(RuntimeError, match="more than one snapshot"):
|
||||
EmbeddingScorer()._get_model()
|
||||
assert fake_fastembed == []
|
||||
|
||||
|
||||
def test_a_cold_cache_offline_degrades_without_a_download(monkeypatch, fake_fastembed):
|
||||
import huggingface_hub
|
||||
from huggingface_hub.errors import LocalEntryNotFoundError
|
||||
|
||||
def _download(repo_id: str, filename: str, **kwargs: object) -> str:
|
||||
if not kwargs.get("local_files_only"):
|
||||
raise AssertionError("the network fallback ran while offline")
|
||||
raise LocalEntryNotFoundError("not cached")
|
||||
|
||||
monkeypatch.setattr(huggingface_hub, "hf_hub_download", _download)
|
||||
monkeypatch.setenv(OFFLINE_ENV, "1")
|
||||
with pytest.raises(RuntimeError, match="is unavailable") as excinfo:
|
||||
EmbeddingScorer()._get_model()
|
||||
assert fake_fastembed == []
|
||||
# The hint names the exact snapshot to seed, not the fastembed cache an
|
||||
# older Headroom read.
|
||||
assert f"{DEFAULT_MODEL_REPO}@{PIN}" in str(excinfo.value)
|
||||
assert f"--revision {PIN}" in str(excinfo.value)
|
||||
|
||||
|
||||
def test_other_models_keep_fastembed_resolution(monkeypatch, fake_hub):
|
||||
seen: list[dict[str, str]] = []
|
||||
monkeypatch.setattr(embedding, "_load_text_embedding", lambda kwargs: seen.append(kwargs))
|
||||
monkeypatch.setitem(sys.modules, "fastembed", SimpleNamespace())
|
||||
|
||||
EmbeddingScorer("intfloat/e5-small-v2")._get_model()
|
||||
|
||||
assert seen == [{"model_name": "intfloat/e5-small-v2"}]
|
||||
assert fake_hub == []
|
||||
|
||||
|
||||
def test_fastembed_loads_specific_model_path_without_a_download(monkeypatch, tmp_path):
|
||||
"""The contract the fix rests on, against the real fastembed: given
|
||||
``specific_model_path`` it uses that directory and never reaches the Hub."""
|
||||
pytest.importorskip("fastembed")
|
||||
import fastembed.common.model_management as model_management
|
||||
from fastembed import TextEmbedding
|
||||
|
||||
def _no_hub(*args: object, **kwargs: object) -> None:
|
||||
raise AssertionError("fastembed reached the Hub despite specific_model_path")
|
||||
|
||||
monkeypatch.setattr(model_management, "snapshot_download", _no_hub)
|
||||
model = TextEmbedding(
|
||||
model_name=DEFAULT_MODEL_NAME,
|
||||
specific_model_path=str(tmp_path),
|
||||
cache_dir=str(tmp_path / "unused-cache"),
|
||||
lazy_load=True,
|
||||
)
|
||||
assert Path(model.model._model_dir) == tmp_path
|
||||
@@ -2351,13 +2351,17 @@ _EGRESS_ALLOWLIST: dict[str, tuple[int, str]] = {
|
||||
"calls guard_egress first.",
|
||||
),
|
||||
"relevance/embedding.py": (
|
||||
1,
|
||||
2,
|
||||
"cache-only: the first of the two fastembed loads in "
|
||||
"_load_text_embedding runs with HF_HUB_OFFLINE forced to 1, so "
|
||||
"huggingface_hub resolves from the local cache or raises without "
|
||||
"opening a socket. The network retry directly below it calls "
|
||||
"guard_egress, which is why a pre-seeded air-gapped host still loads "
|
||||
"the model and a cold one refuses instead of dialling.",
|
||||
"the model and a cold one refuses instead of dialling. The load in "
|
||||
"_load_default_model runs under the same forced HF_HUB_OFFLINE and "
|
||||
"passes specific_model_path, which fastembed loads without any Hub "
|
||||
"lookup; its files come through hf_hub_download_local_first, guarded "
|
||||
"on the miss.",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@@ -13,11 +13,6 @@ pytest.importorskip("fastapi")
|
||||
|
||||
from headroom import paths
|
||||
from headroom.proxy.output_savings import SavingsRecorder
|
||||
from headroom.relevance.embedding import (
|
||||
_DEFAULT_MODEL_PINNED_REVISION,
|
||||
DEFAULT_MODEL_NAME,
|
||||
_pinned_revision,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -100,23 +95,3 @@ def test_memory_disabled_under_stateless(tmp_path, monkeypatch):
|
||||
proxy = app.state.proxy
|
||||
assert proxy.memory_handler is None
|
||||
assert not workspace.exists()
|
||||
|
||||
|
||||
# ---- fastembed model pinning ----------------------------------------------
|
||||
|
||||
|
||||
def test_fastembed_default_model_is_pinned_to_sha(monkeypatch):
|
||||
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
|
||||
rev = _pinned_revision(DEFAULT_MODEL_NAME)
|
||||
assert rev == _DEFAULT_MODEL_PINNED_REVISION
|
||||
assert len(rev) == 40 and all(c in "0123456789abcdef" for c in rev)
|
||||
|
||||
|
||||
def test_fastembed_custom_model_not_pinned(monkeypatch):
|
||||
monkeypatch.delenv("HEADROOM_HF_PIN", raising=False)
|
||||
assert _pinned_revision("intfloat/e5-small-v2") is None
|
||||
|
||||
|
||||
def test_fastembed_pin_can_be_disabled(monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_HF_PIN", "off")
|
||||
assert _pinned_revision(DEFAULT_MODEL_NAME) is None
|
||||
|
||||
@@ -1813,7 +1813,7 @@ requires-dist = [
|
||||
{ name = "datasets", marker = "extra == 'voice-train'", specifier = ">=5.0.1" },
|
||||
{ name = "fastapi", marker = "extra == 'dev'", specifier = ">=0.100.0" },
|
||||
{ name = "fastapi", marker = "extra == 'proxy'", specifier = ">=0.100.0" },
|
||||
{ name = "fastembed", marker = "(python_full_version < '3.14' and extra == 'relevance') or (platform_machine != 'x86_64' and extra == 'relevance') or (sys_platform != 'darwin' and extra == 'relevance')", specifier = ">=0.4.0" },
|
||||
{ name = "fastembed", marker = "(python_full_version < '3.14' and extra == 'relevance') or (platform_machine != 'x86_64' and extra == 'relevance') or (sys_platform != 'darwin' and extra == 'relevance')", specifier = ">=0.6.0" },
|
||||
{ name = "gunicorn", marker = "sys_platform != 'win32' and extra == 'proxy-prod'", specifier = ">=21.0.0" },
|
||||
{ name = "headroom-ai", extras = ["proxy"], marker = "extra == 'proxy-prod'" },
|
||||
{ name = "headroom-ai", extras = ["proxy", "code", "ml", "memory", "relevance", "image", "reports", "otel", "evals", "voice", "html", "mcp", "spreadsheet"], marker = "extra == 'all'" },
|
||||
|
||||
Reference in New Issue
Block a user