fix: prevent HF tokenizer downloads in offline mode (#3783)

## Description

`HEADROOM_OFFLINE=1` must not fall through from a cold HuggingFace
tokenizer cache to a network download. This change applies the existing
offline environment defaults before importing Transformers and returns
to estimation after a cache miss when offline mode is active.

Closes #3777

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Apply `apply_offline_env()` before the Transformers import when
`HEADROOM_OFFLINE` is enabled.
- Skip the network fallback and use estimation after an offline cache
miss.
- Add regression coverage for cold-cache no-egress behavior, explicit
offline-flag overrides, and warm-cache loading.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
22 passed in 1.09s
2 files already formatted
All checks passed!
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.13.15, `uv` 0.12.15, Transformers
5.x, empty temporary `HF_HOME`, and a loopback HTTP server used as
`HF_ENDPOINT`; no provider or HuggingFace credentials.
- Exact command / steps: ran `env -u HF_HUB_OFFLINE -u
TRANSFORMERS_OFFLINE PYTHONPATH=. uv run --no-project --with
'transformers>=5.5,<6' --with opentelemetry-api --with pydantic --with
tiktoken --with click --with rich --with pyyaml --with tomlkit python -`
with a script that started the loopback server, set `HF_HOME` to an
empty temporary directory, set `HEADROOM_OFFLINE=1`, called
`_load_tokenizer("Qwen/Qwen2.5-7B")`, and recorded requests and cache
entries.
- Observed result: `_load_tokenizer` returned `None` (estimation
fallback), the loopback HuggingFace endpoint received no requests, the
empty `HF_HOME` stayed empty, and both `HF_HUB_OFFLINE` and
`TRANSFORMERS_OFFLINE` were set to `1`:

```text
{"cache_entries": [], "hf_hub_offline": "1", "loader_http_requests": [], "result": null, "transformers_offline": "1"}
```

- Not tested: a real external `huggingface.co` endpoint, a full proxy
request, or a real warm tokenizer cache; those paths are covered by the
existing loader tests and the local proof isolates the changed behavior.

## Runtime Rollout Safety

- Rollout-managed feature(s): HuggingFace tokenizer network fallback.
- Minimum rollout channel: next patch release.
- Stable/default behavior changed: only when `HEADROOM_OFFLINE` is
enabled; online behavior remains unchanged.
- Kill switch / disable path: unset `HEADROOM_OFFLINE` and restart the
process; existing explicit `HF_HUB_OFFLINE` and `TRANSFORMERS_OFFLINE`
overrides remain respected by `apply_offline_env()`.
- Unsafe override required: none.
- Qualification impact: no provider, model, or API qualification
changes.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title

## Screenshots (if applicable)

Not applicable.

## Additional Notes

No dependencies or generated files were added. The implementation reuses
the existing offline-mode contract rather than introducing a second
configuration switch.


<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3783"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
yi111
2026-10-04 13:55:13 -07:00
committed by GitHub
co-authored by JD Davis
parent 0d841feb89
commit d503c575ea
2 changed files with 52 additions and 1 deletions
+13
View File
@@ -226,6 +226,12 @@ def _load_tokenizer(tokenizer_name: str):
# vetted, not a variant the caller chose.
tokenizer_name = repo
from headroom.offline import apply_offline_env, is_offline
offline = is_offline()
if offline:
apply_offline_env()
from transformers import AutoTokenizer
try:
@@ -242,6 +248,13 @@ def _load_tokenizer(tokenizer_name: str):
except Exception:
pass # Not in the local cache — try the network below, bounded.
if offline:
logger.warning(
f"Tokenizer {tokenizer_name} not in local HF cache and network "
f"loading is disabled (HEADROOM_OFFLINE); using estimation"
)
return None
timeout = _load_timeout_secs()
if timeout <= 0:
logger.warning(
+39 -1
View File
@@ -29,7 +29,10 @@ from headroom.tokenizers.huggingface import (
@pytest.fixture(autouse=True)
def _fresh_cache():
def _fresh_cache(monkeypatch: pytest.MonkeyPatch):
monkeypatch.delenv("HEADROOM_OFFLINE", raising=False)
monkeypatch.delenv("HF_HUB_OFFLINE", raising=False)
monkeypatch.delenv("TRANSFORMERS_OFFLINE", raising=False)
_load_tokenizer.cache_clear()
yield
_load_tokenizer.cache_clear()
@@ -106,6 +109,41 @@ def test_timeout_zero_disables_network_loading(monkeypatch: pytest.MonkeyPatch)
assert _load_tokenizer("google/gemma-7b") is None
def test_offline_mode_prevents_network_loading(monkeypatch: pytest.MonkeyPatch) -> None:
calls: list[dict[str, Any]] = []
def fake_from_pretrained(name: str, **kwargs: Any):
calls.append(kwargs)
if kwargs.get("local_files_only"):
raise OSError("not in cache")
raise AssertionError("network load attempted in offline mode")
_install_fake_transformers(monkeypatch, fake_from_pretrained)
monkeypatch.setenv("HEADROOM_OFFLINE", "1")
monkeypatch.setenv("HF_HUB_OFFLINE", "0")
monkeypatch.setenv("TRANSFORMERS_OFFLINE", "0")
monkeypatch.setenv("HEADROOM_HF_TOKENIZER_LOAD_TIMEOUT_SECS", "5")
assert _load_tokenizer("Qwen/Qwen2.5-7B") is None
assert len(calls) == 1
assert calls[0].get("local_files_only") is True
def test_offline_mode_allows_cached_tokenizer(monkeypatch: pytest.MonkeyPatch) -> None:
calls: list[dict[str, Any]] = []
def fake_from_pretrained(name: str, **kwargs: Any):
calls.append(kwargs)
return "cached-tokenizer"
_install_fake_transformers(monkeypatch, fake_from_pretrained)
monkeypatch.setenv("HEADROOM_OFFLINE", "1")
assert _load_tokenizer("Qwen/Qwen2.5-7B") == "cached-tokenizer"
assert len(calls) == 1
assert calls[0].get("local_files_only") is True
def test_count_messages_fails_open_to_estimation(monkeypatch: pytest.MonkeyPatch) -> None:
def fake_from_pretrained(name: str, **kwargs: Any):
raise OSError("unavailable")