Files
9bb4b130ff fix(relevance): load the Rust default embedder from Python's pinned snapshot (#4190)
## Description

Parent status: #4152 has merged. The pinned-loader and batch-cap merge
resolution was reviewed at e3a495cbc5. The
original split/stack history is retained below.

**Rebased onto main (`7e0094d6`), 2026-11-03.** The branch is now four
linear commits on top of main, with no merge commits:

- `bfae4ade` fix(relevance): load the Rust default embedder from
Python's pinned snapshot
- `6fbae380` fix(relevance): look up and download the Rust embedder in
one HF cache dir
- `70afece8` fix(relevance): expand ~ and $VAR in the Rust HF cache path
like huggingface_hub
- `62b792f0` fix(relevance): keep an empty HF cache variable, as
huggingface_hub does

What the rebase changed:

- The #4152 commit (`cb1c3a33`) is gone from the branch: main already
has it as `f380692b`. Its files matched main; the only difference was
`uv.lock`, where main's later `filelock>=4.2.1` entry was kept.
- The six main-sync merges (`185dc9b7`, `052977b9`, `029c72ac`,
`4405b12c`, `8b0a10e8`, and the reviewed `e3a495cb`) are linearized
away, so those SHAs are no longer on the branch.
- `embedding.rs` conflicted again with main's batch-size cap (#4507). I
resolved it to the same content as the reviewed merge: every file this
PR touches is byte-identical between `e3a495cb` and the new head
`62b792f0` (`git diff e3a495cb 62b792f0 -- <files changed vs main>` is
empty). The review and qualification below still apply to the content;
only the history and SHAs changed.

Rust `EmbeddingScorer::try_new` used `EmbeddingModel::BGESmallENV15`.
fastembed-rs maps that to a different repo and file from what Python
runs: `Xenova/bge-small-en-v1.5` `onnx/model.onnx` (fp32), always on
`main`. So the two languages never ran the same ONNX file, and the
"byte-equal across languages" claim in both modules was wrong.

This PR makes `try_new` load the same pinned
`Qdrant/bge-small-en-v1.5-onnx-Q` snapshot that #4152 makes Python load.
It adds a cross-language parity fixture checked from both sides, and
replaces the parity claim with measured numbers.

Nothing in production calls `EmbeddingScorer::try_new` today:
`create_scorer` and `HybridScorer` use `default()`, which never loads a
model. This keeps the Rust scorer correct and pinned for when it is
wired in.

No linked issue.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

(This PR's own commits only.)

- `crates/headroom-core/src/relevance/embedding.rs`:
- `try_new` loads the snapshot through
`TextEmbedding::try_new_from_user_defined`, with CLS pooling and static
quantization, which is what fastembed-rs uses for `BGESmallENV15Q`.
- It uses exactly one hub cache directory, resolved like Python's
`huggingface_hub.constants.HF_HUB_CACHE`: `HF_HUB_CACHE` →
`HUGGINGFACE_HUB_CACHE` → `HF_HOME/hub` →
`XDG_CACHE_HOME/huggingface/hub` → `~/.cache/huggingface/hub`, with a
leading `~` expanded. It looks up and downloads only there, so a stale
snapshot in a lower-priority cache is never used. The home directory
comes from `std::env::home_dir`, which falls back to the user database
when `HOME` is unset, as Python's `expanduser` does.
- It finds the snapshot as `snapshots/<sha>` in that directory.
`hf_hub::Cache` resolves through `refs/<rev>`, which Python's
`huggingface_hub` doesn't write for a commit SHA.
- On a miss it downloads with `Repo::with_revision`. `guard_egress` sits
in front of the Hub client and runs only on a cache miss, so a
pre-seeded air-gapped host now loads; previously Rust refused even with
a warm cache.
- `HEADROOM_HF_PIN=off` floats it on `main`, with the same values as
Python's `_resolve_revision`.
  - `try_new_with_model` is unchanged.
- Parity fixture: `tests/parity/record_embedding.py` and
`tests/parity/fixtures/embedding/bge_small_en_v15_pinned.json` (6 texts
× 384 dims, recorded on ORT 1.31.0). The fixture is written one
embedding row per line, so it's 22 lines. Python checks it in CI
(`test-extras` sets `RUN_FASTEMBED_TESTS=1`); Rust checks it under
`RUN_FASTEMBED_TESTS=1`.
- `tests/test_fastembed_pinned_snapshot.py`: the Rust constants name the
same repo, SHA and files as Python; the fixture was recorded at the pin;
plus the gated Python parity check.
- 10 new Rust unit tests in `embedding.rs`: the lookup ignores other
commits and needs every file; the download URL requests the pinned
revision; the offline guard applies on a miss only;
`HEADROOM_HF_PIN=off` floats; the cache directory follows
`huggingface_hub`'s order variable by variable (including an empty
variable and `~`); with no `HOME` it still resolves through the user
database; and the review case: an empty explicit `HF_HUB_CACHE` next to
a complete pinned snapshot and an older `refs/main` under `HOME` finds
neither, for the pinned revision or `main`, while both are found once
`HF_HUB_CACHE` is unset.
- `tests/test_offline_egress_chokepoint.py`: the Rust sweep's regex now
also matches `ApiBuilder::from_cache` / `from_env`. It missed the new
download client, which is guarded; it flags nothing else in `crates/`.
The user-defined model load, which only reads bytes already on disk, is
allowlisted as `cache-only`.
- Docs: the "byte-equal" wording in both embedder modules,
`pyproject.toml` and `onnx_runtime.py` is replaced with measured numbers
and the cross-language keep-equal notes.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

After the rebase, on `62b792f0` (macOS arm64, no `ORT_DYLIB_PATH`, so
the gated model tests skip):

```text
$ cargo test -p headroom-core --lib
test result: ok. 991 passed; 0 failed; 1 ignored
$ cargo clippy -p headroom-core --all-targets -- -D warnings                  # clean
$ cargo clippy -p headroom-core --lib --no-default-features -- -D warnings    # clean
$ cargo fmt --all -- --check                                                  # clean
$ pytest -q tests/test_fastembed_pinned_snapshot.py tests/test_offline_egress_chokepoint.py tests/test_hf_revision_pinning.py
235 passed, 3 skipped
$ ruff check / ruff format --check on the PR's Python files (ruff 0.17.2)     # clean
```

The 7 offline-egress failures noted below no longer reproduce here.

Before the rebase:

```text
$ cargo test -p headroom-core --lib
test result: ok. 985 passed; 0 failed; 1 ignored      # see the note on #4151 below

$ cargo clippy -p headroom-core --all-targets -- -D warnings                  # clean
$ cargo clippy -p headroom-core --lib --no-default-features -- -D warnings    # clean
$ cargo fmt --all -- --check                                                  # clean

$ RUN_FASTEMBED_TESTS=1 ORT_DYLIB_PATH=<onnxruntime 1.31.0 dylib> cargo test -p headroom-core --lib relevance::embedding
test relevance::embedding::tests::fastembed_matches_parity_fixture ... ok
test result: ok. 24 passed; 0 failed

$ RUN_FASTEMBED_TESTS=1 pytest -q tests/test_fastembed_pinned_snapshot.py tests/test_relevance.py \
    tests/test_stateless_writers.py tests/test_hf_revision_pinning.py
71 passed

$ pytest -q tests/test_offline_egress_chokepoint.py
7 failed, 132 passed, 3 skipped
# The 7 are the OTel / Langfuse / Headroom Cloud offline tests, whose extras aren't in this venv;
# the same 7 fail on origin/main here. The Rust sweep tests pass.
```

**Note on #4151:** with `ORT_DYLIB_PATH` set on macOS, a full `cargo
test -p headroom-core --lib` fails
`tokenizer::hf_impl::tests::from_pretrained_invalid_repo_returns_hub_error`
most of the time (5/5 on `afdb60d3`, before the cache change too).
`a_cached_snapshot_is_not_refused_offline` holds `HEADROOM_OFFLINE=1`
under the env lock while ONNX Runtime loads, which is slow there, and
the tokenizer test reads the variable without the lock. That's the race
#4151 fixes; with its commit applied, 6/6 full runs pass. Linux CI
passes either way.

The cache-precedence test was checked against the bug: with the lookup
changed back to also scanning `~/.cache`, it fails on the first case
(pinned revision, empty `HF_HUB_CACHE`).

## Real Behavior Proof
- Exact command / steps: additional maintainer verification on the
main-synced head: cargo test -p headroom-core --locked hf_hub_cache_dir;
cargo test -p headroom-core --locked
expand_vars_matches_python_posixpath_and_ntpath; python -m pytest
tests/test_fastembed_pinned_snapshot.py -q; repeat its fixture cases
with huggingface-hub==2.2.0.
- Observed result: three Rust tests passed across both POSIX and Windows
expansion fixtures and cache-directory selection; Python passed 46 tests
with one gated skip using Hugging Face Hub 1.16.1, and all 36 shared
fixture cases passed separately with locked Hub 2.2.0.

- Environment: macOS arm64; Rust fastembed 5.17.3 with hf-hub 0.5.0;
repo `.venv` (fastembed 0.8.0, onnxruntime 1.23.2) and a second
interpreter on onnxruntime 1.31.0.
- A cold-cache Rust load with `HF_HUB_CACHE=<empty dir>` wrote
`snapshots/52398278…` plus `refs/52398278…` and passed the parity check.
Per-component max |Δ| against the fixture:
  - Rust vs Python on the same ORT (1.31.0): ≤ 1e-6.
  - Python on ORT 1.23.2 vs 1.31.0: ≤ 4.1e-4.
  - The fixture's tolerance is 1e-3.
- For reference, Xenova's fp32 export (what Rust used to load) is ≤
4.7e-4 from the fixture. The old cross-language gap was real but
numerically small, and this fixture bounds drift; it cannot tell the two
exports apart. Model identity is enforced by the pinned SHA and the unit
tests.
- The fixture's new one-row-per-line layout was re-emitted from the
recorded data, not re-recorded: the parsed JSON is identical.
- Not tested: Windows, and Linux outside CI. Rust's gated tests in CI
(they need the model and an ORT dylib, so they stay manual behind
`RUN_FASTEMBED_TESTS=1`). Rust `HybridScorer` thresholds against the
Qdrant export.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no production path.
`EmbeddingScorer::try_new` loads the Qdrant export instead of Xenova's,
and now loads offline from a warm cache instead of refusing, but nothing
in production calls it today.
- Kill switch / disable path: `HEADROOM_HF_PIN=off` floats it on `main`.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A.

## Additional Notes

- Follow-ups, not in this PR: Rust's Kompress and tokenizer Hub fetches
are still unpinned (there's no Rust `HEADROOM_HF_PIN`), and Rust's gated
embedding tests could get a CI job once one ships an ORT dylib.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4190"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer qualification

The environment-reference mismatch from the previous review is
addressed. The shared corpus covers the reported explicit-cache case,
platform-specific variable syntax, and the two-stage home/hub expansion
ordering. The Rust test runs both POSIX and Windows expansion rules;
Python validates the same corpus against posixpath, ntpath, and fresh
huggingface_hub.constants imports.

A clean current-main sync was published as
052977b9e88ae3bbda96797615b64ee350a2f695; it does not change the
contributor's resolver or fixture files. Qualification used Windows,
Hugging Face Hub 1.16.1 and an isolated locked 2.2.0 target. No model
download or numerical embedding-parity result is claimed by these path
tests; the pre-existing gated numerical-parity test remained skipped.

Hosted checks on the pushed head must finish before approval. No
contributor CI action requested or PR merged.

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-03 21:31:59 -06:00
..