mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(cli): pass configured SSL context to stdlib urlopen calls (#4011)
## Description Fixes #3976. On `uv tool` installs and other environments where certifi is not wired into the stdlib OpenSSL trust path, `headroom update` and bundled-binary downloads can fail behind corporate TLS inspection. These urllib calls now use the configured Headroom trust context, matching the proxy's TLS policy. The lazy helper import still falls back to urllib's default when the optional trust module cannot be imported. Errors while constructing an explicitly configured context now propagate instead of silently replacing the requested trust roots with stdlib defaults. The update checker retains its outer error-handling behavior and returns no update on failure. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - Pass `build_urlopen_context()` to the PyPI version check and release-binary download paths. - Catch only `ImportError` around the lazy helper import; let trust-context construction errors reach the caller. - Add valid and invalid replacement-bundle tests for both helpers and verify the update check still handles the error through its existing outer handler. - Update mocked `urlopen` functions to accept the `context` keyword used by the production call path. ## Testing - [x] Targeted unit tests pass - [ ] Full test suite run - [x] Ruff lint and format checks pass for changed files - [x] New tests added for the requested behavior - [ ] Type checking run - [ ] Manual live-network testing performed ### Test Output ```text python -m pytest --disable-plugin-autoload -p pytest_asyncio.plugin tests/test_ssl_context.py tests/test_binaries.py tests/test_update_check.py 95 passed in 5.10s python -m ruff check headroom/binaries.py headroom/update_check.py tests/test_ssl_context.py tests/test_binaries.py All checks passed! python -m ruff format --check headroom/binaries.py headroom/update_check.py tests/test_ssl_context.py tests/test_binaries.py 4 files already formatted git diff --check passed ``` ## Real Behavior Proof **Original reproduction:** On macOS behind a corporate TLS-inspecting network, `headroom update` failed with `CERTIFICATE_VERIFY_FAILED` when the inspection root was present in the OS trust store but absent from Python's stdlib OpenSSL bundle. The same machine's proxy reached its upstream through the configured root. The original fix routed the direct urllib calls through `build_urlopen_context()`. **Review regression checks:** On Windows with Python 3.13.13, tests using a valid `REQUESTS_CA_BUNDLE` confirm both helpers return a verifying `SSLContext`. With an existing file containing invalid PEM, both helpers now raise `ssl.SSLError`; the update-check caller still returns `None` through its outer error handler. All 95 tests in the affected TLS, binary-download, and update-check suites pass. **Not tested:** A live corporate TLS-inspection connection, Windows `uv tool` end-to-end behavior behind such a network, the full repository test suite, and the separate eval dataset download path. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Not applicable. - Stable/default behavior changed: An invalid explicitly configured trust bundle now fails visibly rather than silently falling back to stdlib roots; unconfigured default behavior is unchanged. - Kill switch / disable path: No new switch; existing TLS environment settings remain available. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this PR's changes. ## Review Readiness - [x] I have performed a self-review. - [x] This PR is ready for human review. ## Checklist - [x] My code follows the project's style guidelines. - [x] I have performed a self-review of my code. - [ ] I have added comments where the code needs additional explanation. - [ ] I have made corresponding documentation changes (none needed for this internal error-handling change). - [x] The changed files pass Ruff checks and the focused tests add no warnings. - [x] I have added tests that prove the requested behavior. - [ ] The full repository unit-test suite was run. - [x] I did not edit `CHANGELOG.md`. --------- Co-authored-by: dajiaohuang <dajiaohuang@users.noreply.github.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
dajiaohuang
JD Davis
parent
a21879bd54
commit
22d0905a73
+32
-2
@@ -48,7 +48,10 @@ import urllib.request
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import ssl
|
||||
|
||||
from headroom._subprocess import run
|
||||
from headroom.offline import guard_egress
|
||||
@@ -264,6 +267,25 @@ def _mirror_url(url: str) -> str:
|
||||
# ---------- Download + verify --------------------------------------------- #
|
||||
|
||||
|
||||
def _ssl_context() -> ssl.SSLContext | None:
|
||||
"""Return Headroom's configured SSL context for urllib.request.urlopen, or
|
||||
``None`` (urlopen's default) when the trust helper cannot be loaded.
|
||||
|
||||
This ensures binary downloads from GitHub releases (and any configured
|
||||
mirror) trust the same roots as the proxy — including corporate
|
||||
TLS-inspection roots installed in the OS store and bundles referenced via
|
||||
``HEADROOM_CA_BUNDLE`` / ``SSL_CERT_FILE``. Without this, ``uv tool``
|
||||
installs (where certifi may not yet be wired to the stdlib path) raise
|
||||
``CERTIFICATE_VERIFY_FAILED`` even though certifi is already a dependency.
|
||||
"""
|
||||
try:
|
||||
from headroom.proxy.ssl_context import build_urlopen_context
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
return build_urlopen_context()
|
||||
|
||||
|
||||
def _download(url: str, dest: Path, *, progress: bool = True) -> None:
|
||||
if os.environ.get("HEADROOM_BINARIES_OFFLINE"):
|
||||
raise OfflineError(f"offline mode (HEADROOM_BINARIES_OFFLINE=1) but fetch required: {url}")
|
||||
@@ -281,10 +303,18 @@ def _download(url: str, dest: Path, *, progress: bool = True) -> None:
|
||||
if not final_url.startswith("https://"):
|
||||
raise BinaryFetchError(f"refusing non-https download URL: {final_url!r}")
|
||||
req = urllib.request.Request(final_url, headers={"User-Agent": "headroom-binaries/1"})
|
||||
try:
|
||||
ctx = _ssl_context()
|
||||
except Exception as e:
|
||||
raise BinaryFetchError(f"failed to configure TLS trust for binary download: {e}") from e
|
||||
attempts = 3
|
||||
for attempt in range(1, attempts + 1):
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=60) as resp: # noqa: S310 (https)
|
||||
if ctx is not None:
|
||||
resp = urllib.request.urlopen(req, timeout=60, context=ctx) # noqa: S310 (https)
|
||||
else:
|
||||
resp = urllib.request.urlopen(req, timeout=60) # noqa: S310 (https)
|
||||
with resp:
|
||||
total = int(resp.headers.get("Content-Length") or 0)
|
||||
_stream_to(resp, dest, total, label=dest.name, show_progress=progress)
|
||||
return
|
||||
|
||||
@@ -26,7 +26,10 @@ import threading
|
||||
import time
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import ssl
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -187,17 +190,46 @@ def _select_latest(data: dict[str, Any], *, allow_pre: bool) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _urlopen_ssl_context() -> ssl.SSLContext | None:
|
||||
"""Return an SSL context for urllib.request.urlopen that honors Headroom's
|
||||
trust policy (OS store + certifi, HEADROOM_CA_BUNDLE, etc).
|
||||
|
||||
Imported lazily so this module's dependency footprint stays at stdlib +
|
||||
packaging when the trust helpers aren't yet importable (e.g. during early
|
||||
startup or in minimal test environments). Falls back to ``None`` (urlopen's
|
||||
default) when the helper cannot be loaded.
|
||||
"""
|
||||
try:
|
||||
from headroom.proxy.ssl_context import build_urlopen_context
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
return build_urlopen_context()
|
||||
|
||||
|
||||
def fetch_latest_version(*, allow_pre: bool = False, timeout: float = 4.0) -> str | None:
|
||||
"""Query the PyPI JSON API for the latest release. Returns None on any error.
|
||||
|
||||
Uses ``urllib`` (stdlib) so the base CLI install needs no HTTP dependency.
|
||||
Passes Headroom's configured SSL context so corporate TLS-inspection roots
|
||||
(Zscaler, Netskope, ...) installed in the OS trust store — or referenced via
|
||||
``HEADROOM_CA_BUNDLE`` / ``SSL_CERT_FILE`` — are trusted the same way the
|
||||
proxy trusts them.
|
||||
"""
|
||||
try:
|
||||
from headroom.offline import guard_egress
|
||||
|
||||
guard_egress("Headroom update check", _PYPI_JSON_URL)
|
||||
req = urllib.request.Request(
|
||||
_PYPI_JSON_URL,
|
||||
headers={"Accept": "application/json", "User-Agent": "headroom-update-check"},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp: # noqa: S310 — fixed https URL
|
||||
ctx = _urlopen_ssl_context()
|
||||
if ctx is not None:
|
||||
resp = urllib.request.urlopen(req, timeout=timeout, context=ctx) # noqa: S310 — fixed https URL
|
||||
else:
|
||||
resp = urllib.request.urlopen(req, timeout=timeout) # noqa: S310 — fixed https URL
|
||||
with resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return _select_latest(data, allow_pre=allow_pre)
|
||||
except Exception:
|
||||
|
||||
+18
-3
@@ -7,6 +7,7 @@ from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import ssl
|
||||
import sys
|
||||
import tarfile
|
||||
import zipfile
|
||||
@@ -101,7 +102,7 @@ def fake_urlopen(monkeypatch):
|
||||
"""Install a fake urllib.request.urlopen that serves registered URLs."""
|
||||
served: dict[str, bytes] = {}
|
||||
|
||||
def fake(req, timeout=None): # noqa: ARG001
|
||||
def fake(req, timeout=None, context=None): # noqa: ARG001
|
||||
url = req.full_url if hasattr(req, "full_url") else req
|
||||
if url not in served:
|
||||
raise AssertionError(f"unexpected fetch for {url}")
|
||||
@@ -218,6 +219,20 @@ def test_mirror_substitution(monkeypatch):
|
||||
assert binaries._mirror_url("https://example.com/x") == "https://example.com/x"
|
||||
|
||||
|
||||
def test_download_wraps_tls_context_error_and_preserves_cause(monkeypatch, tmp_path):
|
||||
error = ssl.SSLError("invalid configured certificate bundle")
|
||||
monkeypatch.setattr(
|
||||
binaries,
|
||||
"_ssl_context",
|
||||
lambda: (_ for _ in ()).throw(error),
|
||||
)
|
||||
|
||||
with pytest.raises(binaries.BinaryFetchError, match="failed to configure TLS trust") as exc:
|
||||
binaries._download("https://example.invalid/tool.tar.gz", tmp_path / "tool.tar.gz")
|
||||
|
||||
assert exc.value.__cause__ is error
|
||||
|
||||
|
||||
def test_mirror_url_with_query_params_strips_them_from_download_filename(
|
||||
monkeypatch, fake_urlopen, allow_unverified
|
||||
):
|
||||
@@ -284,7 +299,7 @@ def test_download_retries_transient_network_failure(monkeypatch, tmp_path):
|
||||
attempts = 0
|
||||
sleeps: list[float] = []
|
||||
|
||||
def flaky_urlopen(req, timeout=None): # noqa: ARG001
|
||||
def flaky_urlopen(req, timeout=None, context=None): # noqa: ARG001
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
if attempts < 3:
|
||||
@@ -465,7 +480,7 @@ def test_ensure_tools_partial_failure_proxy_still_starts(monkeypatch, allow_unve
|
||||
scc_asset = binaries._registry()["tools"]["scc"]["assets"]["darwin-aarch64"]
|
||||
scc_tar = _make_tar_gz({"scc": b"ok"})
|
||||
|
||||
def selective_urlopen(req, timeout=None): # noqa: ARG001
|
||||
def selective_urlopen(req, timeout=None, context=None): # noqa: ARG001
|
||||
url = req.full_url if hasattr(req, "full_url") else req
|
||||
if url == scc_asset["url"]:
|
||||
return _FakeResponse(scc_tar)
|
||||
|
||||
@@ -96,6 +96,74 @@ def test_sg_command_reports_resolution_errors(
|
||||
assert expected in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("args", "expected", "exit_code"),
|
||||
[
|
||||
(("diff", "--version"), "error: failed to configure TLS trust", 2),
|
||||
(("tools", "install", "--tool", "difft"), "difft: failed to configure TLS trust", 1),
|
||||
],
|
||||
)
|
||||
def test_invalid_configured_ca_bundle_is_reported_without_fetching(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
runner: CliRunner,
|
||||
tmp_path: Path,
|
||||
args: tuple[str, ...],
|
||||
expected: str,
|
||||
exit_code: int,
|
||||
) -> None:
|
||||
invalid_bundle = tmp_path / "invalid.pem"
|
||||
invalid_bundle.write_text("not a PEM certificate", encoding="ascii")
|
||||
monkeypatch.setenv("REQUESTS_CA_BUNDLE", str(invalid_bundle))
|
||||
for variable in (
|
||||
"SSL_CERT_FILE",
|
||||
"NODE_EXTRA_CA_CERTS",
|
||||
"HEADROOM_CA_BUNDLE",
|
||||
"HEADROOM_TLS_STRICT",
|
||||
):
|
||||
monkeypatch.delenv(variable, raising=False)
|
||||
monkeypatch.setenv("HEADROOM_CERT_STORE", "bundled")
|
||||
monkeypatch.setattr(cli_tools.binaries, "cache_dir", lambda: tmp_path / "cache")
|
||||
monkeypatch.setattr(cli_tools.binaries, "_path_lookup", lambda _name: None)
|
||||
monkeypatch.setattr(
|
||||
cli_tools.binaries,
|
||||
"detect_platform",
|
||||
lambda: binaries.PlatformKey("linux", "x86_64", "gnu"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
cli_tools.binaries,
|
||||
"_registry",
|
||||
lambda: {
|
||||
"tools": {
|
||||
"difft": {
|
||||
"version": "test",
|
||||
"source": "github",
|
||||
"assets": {
|
||||
"linux-x86_64-gnu": {
|
||||
"url": "https://example.invalid/difft.tar.gz",
|
||||
"member": "difft",
|
||||
"sha256": "0" * 64,
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
urlopen_calls: list[object] = []
|
||||
|
||||
def unexpected_urlopen(*call_args, **call_kwargs):
|
||||
urlopen_calls.append((call_args, call_kwargs))
|
||||
raise AssertionError("invalid configured trust must fail before urlopen")
|
||||
|
||||
monkeypatch.setattr(cli_tools.binaries.urllib.request, "urlopen", unexpected_urlopen)
|
||||
|
||||
result = runner.invoke(main, list(args))
|
||||
|
||||
assert result.exit_code == exit_code
|
||||
assert expected in result.output
|
||||
assert "NO_CERTIFICATE_OR_CRL_FOUND" in result.output
|
||||
assert urlopen_calls == []
|
||||
|
||||
|
||||
def test_tools_list_renders_registry(monkeypatch: pytest.MonkeyPatch, runner: CliRunner) -> None:
|
||||
install_fake_rich(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
|
||||
@@ -57,6 +57,27 @@ PACKAGE_ROOT = REPO_ROOT / "headroom"
|
||||
# ──────────────────────────── socket booby trap ────────────────────────────
|
||||
|
||||
|
||||
class TestUpdateCheckOffline:
|
||||
def test_explicit_update_check_never_constructs_request(self, monkeypatch, offline, no_sockets):
|
||||
from click.testing import CliRunner
|
||||
|
||||
from headroom import update_check
|
||||
from headroom.cli import main
|
||||
|
||||
attempts = []
|
||||
|
||||
def refuse_request(*args, **kwargs):
|
||||
attempts.append((args, kwargs))
|
||||
raise SocketOpened("update check constructed a request while offline")
|
||||
|
||||
monkeypatch.setenv("HEADROOM_UPDATE_CHECK", "on")
|
||||
monkeypatch.setattr(update_check.urllib.request, "Request", refuse_request)
|
||||
result = CliRunner().invoke(main, ["update", "--check"])
|
||||
|
||||
assert attempts == []
|
||||
assert result.exit_code == 1
|
||||
|
||||
|
||||
class SocketOpened(AssertionError):
|
||||
"""Raised from the patched socket entry points.
|
||||
|
||||
@@ -2271,11 +2292,6 @@ _EGRESS_ALLOWLIST: dict[str, tuple[int, str]] = {
|
||||
"deployment that points Headroom at an on-prem model endpoint, which "
|
||||
"is the main reason such a deployment exists.",
|
||||
),
|
||||
"update_check.py": (
|
||||
1,
|
||||
"gated: is_update_check_enabled() returns False when is_offline(), so "
|
||||
"the request is never built. See headroom/update_check.py.",
|
||||
),
|
||||
"telemetry/session.py": (
|
||||
1,
|
||||
"gated: the beacon upload is reached only via is_telemetry_enabled(), "
|
||||
|
||||
@@ -17,6 +17,7 @@ import ssl
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom import binaries, update_check
|
||||
from headroom.proxy import ssl_context
|
||||
from headroom.proxy.ssl_context import (
|
||||
apply_global_tls_relaxation,
|
||||
@@ -341,6 +342,53 @@ class TestBuildUrlopenContext:
|
||||
assert created_context.alpn_protocols == ["http/1.1"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"context_helper",
|
||||
[
|
||||
pytest.param(binaries._ssl_context, id="binary-download"),
|
||||
pytest.param(update_check._urlopen_ssl_context, id="update-check"),
|
||||
],
|
||||
)
|
||||
def test_urlopen_context_helpers_use_valid_configured_bundle(
|
||||
monkeypatch, ca_pem_file, context_helper
|
||||
):
|
||||
_clean_env(monkeypatch)
|
||||
monkeypatch.setenv("REQUESTS_CA_BUNDLE", ca_pem_file)
|
||||
|
||||
context = context_helper()
|
||||
|
||||
assert isinstance(context, ssl.SSLContext)
|
||||
assert context.verify_mode == ssl.CERT_REQUIRED
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"context_helper",
|
||||
[
|
||||
pytest.param(binaries._ssl_context, id="binary-download"),
|
||||
pytest.param(update_check._urlopen_ssl_context, id="update-check"),
|
||||
],
|
||||
)
|
||||
def test_urlopen_context_helpers_propagate_invalid_configured_bundle(
|
||||
monkeypatch, tmp_path, context_helper
|
||||
):
|
||||
_clean_env(monkeypatch)
|
||||
invalid_bundle = tmp_path / "invalid.pem"
|
||||
invalid_bundle.write_text("not a PEM certificate", encoding="ascii")
|
||||
monkeypatch.setenv("REQUESTS_CA_BUNDLE", str(invalid_bundle))
|
||||
|
||||
with pytest.raises(ssl.SSLError):
|
||||
context_helper()
|
||||
|
||||
|
||||
def test_update_check_returns_none_when_configured_bundle_is_invalid(monkeypatch, tmp_path):
|
||||
_clean_env(monkeypatch)
|
||||
invalid_bundle = tmp_path / "invalid.pem"
|
||||
invalid_bundle.write_text("not a PEM certificate", encoding="ascii")
|
||||
monkeypatch.setenv("REQUESTS_CA_BUNDLE", str(invalid_bundle))
|
||||
|
||||
assert update_check.fetch_latest_version() is None
|
||||
|
||||
|
||||
class TestApplyGlobalTlsRelaxation:
|
||||
def test_noop_when_strict_on(self, monkeypatch):
|
||||
_clean_env(monkeypatch)
|
||||
|
||||
Reference in New Issue
Block a user