mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description Closes #3354. When Codex uses an OpenAI API key stored in auth.json, Headroom's generated provider block did not configure authentication. Codex therefore sent requests to the local Headroom proxy without an Authorization header and received 401 Missing bearer. This is separate from #3206: that issue covers ChatGPT OAuth authentication, while this change covers file-backed API-key authentication. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - Resolve API-key and OAuth credentials from `codex_home_dir()` for both project and user configurations, honoring `CODEX_HOME`. - Generate a private, config-specific helper beside `auth.json`; API keys stay out of generated TOML and Headroom environment variables. - Configure Codex's supported `auth.command` field for persistent install, init, and wrap. - Serialize command and argument paths with literal Unicode and validate generated TOML before writing provider configurations. - Abort provider updates with an actionable error if a helper collides with another file/symlink or cannot be created. - Use exclusive creation, 0600 permissions on Unix, and content checks for helper ownership. - Give project and user configurations independent helper lifetimes. Retain legacy shared helpers and helpers referenced by restored/remaining providers or unparseable configurations. - Preserve unrelated files and the original provider config when authentication setup fails. - Remove an unused config-specific helper on uninstall/unwrap. ## Testing - [x] Focused unit and integration tests pass - [x] Linting and formatting pass - [ ] Type checking - [x] New regression tests added - [x] Manual runtime testing performed ### Test Output ```text $ .venv/bin/python -m pytest tests/test_provider_codex_install.py tests/test_install/test_codex_install.py tests/test_cli/test_init_cli.py tests/test_install/test_providers.py tests/test_cli/test_wrap_codex.py -q 284 passed $ .venv/bin/ruff check . All checks passed! $ .venv/bin/ruff format --check . 1744 files already formatted ``` Ruff 0.16.8 matches this branch's CI pin. A focused coverage run reports 93% line/branch coverage for `headroom/providers/codex/install.py` and covers 44/45 newly added executable lines across the three production files in this follow-up. This is local focused coverage, not a claim about the hosted Codecov aggregate. Before the follow-up implementation, the added local-init/Unicode/helper-isolation regressions produced 6 failures while 118 existing tests passed. ## Real Behavior Proof - Environment: macOS 26.6.2, Apple Silicon arm64, Python 3.12.11, Headroom source from this PR, Codex CLI 0.160.0, Sub2API OpenAI-compatible provider, model `gpt-6-luna`, existing API key in the user's Codex `auth.json`. - Setup: start an isolated Headroom proxy on an unused loopback port with `ProxyConfig(optimize=False, cache_enabled=False, rate_limit_enabled=False, openai_api_url=<existing upstream>)`. Generate a temporary provider config with `_ensure_codex_provider(config_path, port)` while credentials resolve from the user's Codex home. Add `model = "gpt-6-luna"` and `model_reasoning_effort = "low"` at the document root. Run the child with its own temporary `CODEX_HOME` and no `OPENAI_API_KEY` or `OPENAI_BASE_URL` environment variable; its home contains the generated config, not a copied credential file. - Exact child command: ```sh CODEX_HOME=<temporary-home> codex exec --ephemeral --skip-git-repo-check --sandbox read-only --json -C <temporary-workspace> 'Reply with exactly OK. Do not call tools.' ``` Captured after-fix output on 2026-10-07: ```text Headroom test proxy: ready (optimization disabled) Codex: 0.160.0; model: gpt-6-luna; auth: generated command, no API-key environment variable codex exec exit: 0 agent messages: ['OK'] Responses bearer matches auth.json: True ``` The last line comes from an ASGI wrapper that compares the HTTP/WebSocket Responses request bearer with the file-backed key without logging either value. The command completed against the real upstream; no mocked model response was used. The temporary provider config/helper and test proxy were removed after validation. - Observed limitation: this upstream rejects WebSocket negotiation with HTTP 426, so Headroom's existing HTTP streaming fallback completed the request. This run proves file-backed command authentication and successful proxy routing, not a successful upstream WebSocket session. - Not tested in this follow-up: a new Windows/Linux runtime run, full test suite, native-extension rebuild, or optimization behavior. The focused Python suite reused the existing local native extension; no Rust code changed. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: N/A. - Stable/default behavior changed: Yes, only when Codex has a file-backed API key and Headroom writes a provider block. - Kill switch / disable path: Remove the Headroom-managed provider block or run the existing Codex unwrap command. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert the generated provider block; auth.json is unchanged. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of the code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did not edit CHANGELOG.md ## Additional Notes Helpers contain no credential value and read the user's existing credential file at runtime, so key rotation does not require reinstalling. Helper names now include a hash of the owning config's canonical path. Legacy fixed-name helpers are retained conservatively because other project configurations may still use them. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3355" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
371 lines
13 KiB
Python
371 lines
13 KiB
Python
from __future__ import annotations
|
|
|
|
import json
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
if sys.version_info >= (3, 11):
|
|
import tomllib
|
|
else: # pragma: no cover - exercised only on Python 3.10
|
|
import tomli as tomllib # type: ignore[no-redef]
|
|
|
|
from headroom.providers.codex.install import (
|
|
CodexAuthConfigError,
|
|
build_codex_auth_config,
|
|
build_provider_section,
|
|
cleanup_codex_auth_helper,
|
|
codex_auth_helper_is_referenced,
|
|
codex_auth_helper_path,
|
|
codex_uses_api_key_auth,
|
|
codex_uses_chatgpt_auth,
|
|
)
|
|
|
|
|
|
def test_codex_provider_section_omits_requires_openai_auth_by_default() -> None:
|
|
"""#406: the flag must default off (API-key users), and only on for OAuth.
|
|
|
|
Setting requires_openai_auth on a custom [model_providers.headroom] block
|
|
forces codex to demand OpenAI OAuth login for every headroom-routed request,
|
|
which breaks API-key users; so callers opt in explicitly for ChatGPT users.
|
|
"""
|
|
section = build_provider_section(port=8787, name="OpenAI via Headroom proxy")
|
|
|
|
assert 'name = "OpenAI via Headroom proxy"' in section
|
|
assert 'base_url = "http://127.0.0.1:8787/v1"' in section
|
|
assert "requires_openai_auth" not in section, (
|
|
f"requires_openai_auth must be absent by default; got:\n{section}"
|
|
)
|
|
assert "supports_websockets = true" in section
|
|
assert 'env_key = "OPENAI_API_KEY"' not in section
|
|
|
|
|
|
def test_codex_provider_section_emits_requires_openai_auth_when_flagged() -> None:
|
|
section = build_provider_section(
|
|
port=8787, name="OpenAI via Headroom proxy", requires_openai_auth=True
|
|
)
|
|
|
|
assert "requires_openai_auth = true" in section
|
|
|
|
|
|
def test_codex_uses_chatgpt_auth_true_for_chatgpt_mode(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"auth_mode": "chatgpt"}', encoding="utf-8")
|
|
|
|
assert codex_uses_chatgpt_auth(auth) is True
|
|
|
|
|
|
def test_codex_uses_chatgpt_auth_true_for_account_id_without_mode(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"tokens": {"account_id": "acct_1"}}', encoding="utf-8")
|
|
|
|
assert codex_uses_chatgpt_auth(auth) is True
|
|
|
|
|
|
def test_codex_uses_chatgpt_auth_false_for_api_key(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"auth_mode": "apikey", "OPENAI_API_KEY": "sk-x"}', encoding="utf-8")
|
|
|
|
assert codex_uses_chatgpt_auth(auth) is False
|
|
|
|
|
|
def test_codex_uses_api_key_auth_detects_file_backed_key(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-x"}', encoding="utf-8")
|
|
|
|
assert codex_uses_api_key_auth(auth) is True
|
|
|
|
|
|
def test_codex_uses_api_key_auth_rejects_missing_or_blank_key(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
for document in ("{}", '{"OPENAI_API_KEY": " "}', "not json"):
|
|
auth.write_text(document, encoding="utf-8")
|
|
assert codex_uses_api_key_auth(auth) is False
|
|
|
|
|
|
def test_build_codex_auth_config_generates_helper_without_copying_key(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
|
|
config = build_codex_auth_config(auth)
|
|
helper = codex_auth_helper_path(auth)
|
|
|
|
assert "auth = { command =" in config
|
|
assert tomllib.loads(config)["auth"]["args"] == [str(helper.resolve())]
|
|
assert "sk-test-only" not in config
|
|
assert helper.read_text(encoding="utf-8").count("OPENAI_API_KEY") == 1
|
|
if sys.platform != "win32":
|
|
assert helper.stat().st_mode & 0o777 == 0o600
|
|
result = subprocess.run(
|
|
[sys.executable, str(helper)], capture_output=True, text=True, check=True
|
|
)
|
|
assert result.stdout == "sk-test-only"
|
|
|
|
|
|
def test_codex_auth_helper_reference_parses_toml_escaped_windows_path() -> None:
|
|
helper = r"C:\Users\example\.codex\.headroom-codex-auth.py"
|
|
config = (
|
|
"[model_providers.headroom]\n"
|
|
f'auth = {{ command = "python", args = [{json.dumps(helper)}] }}\n'
|
|
)
|
|
|
|
assert codex_auth_helper_is_referenced(config, helper) is True
|
|
|
|
|
|
def test_build_codex_auth_config_does_not_overwrite_existing_helper(
|
|
tmp_path: Path,
|
|
) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
helper = codex_auth_helper_path(auth)
|
|
helper.write_text("user content", encoding="utf-8")
|
|
|
|
with pytest.raises(CodexAuthConfigError, match="conflicting file or symlink"):
|
|
build_codex_auth_config(auth)
|
|
assert helper.read_text(encoding="utf-8") == "user content"
|
|
|
|
|
|
def test_build_codex_auth_config_rejects_helper_symlink(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
target = tmp_path / "user-file.txt"
|
|
target.write_text("user content", encoding="utf-8")
|
|
helper = codex_auth_helper_path(auth)
|
|
try:
|
|
helper.symlink_to(target)
|
|
except OSError:
|
|
pytest.skip("symlinks are unavailable")
|
|
|
|
with pytest.raises(CodexAuthConfigError, match="conflicting file or symlink"):
|
|
build_codex_auth_config(auth)
|
|
assert target.read_text(encoding="utf-8") == "user content"
|
|
assert helper.is_symlink()
|
|
|
|
|
|
def test_build_codex_auth_config_handles_invalid_existing_helper(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
helper = codex_auth_helper_path(auth)
|
|
helper.write_bytes(b"\xff\xfe")
|
|
|
|
with pytest.raises(CodexAuthConfigError, match="conflicting file or symlink"):
|
|
build_codex_auth_config(auth)
|
|
assert helper.read_bytes() == b"\xff\xfe"
|
|
|
|
|
|
def test_cleanup_codex_auth_helper_only_removes_headroom_file(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
build_codex_auth_config(auth)
|
|
helper = codex_auth_helper_path(auth)
|
|
|
|
cleanup_codex_auth_helper(auth)
|
|
|
|
assert not helper.exists()
|
|
|
|
|
|
def test_build_codex_auth_config_skips_chatgpt_auth(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"auth_mode": "chatgpt"}', encoding="utf-8")
|
|
|
|
assert build_codex_auth_config(auth) == ""
|
|
|
|
|
|
def test_auth_command_round_trips_astral_paths(tmp_path: Path, monkeypatch) -> None:
|
|
home = tmp_path / "😀"
|
|
home.mkdir()
|
|
auth = home / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
executable = str(home / "python-🐍")
|
|
monkeypatch.setattr(sys, "executable", executable)
|
|
|
|
command = tomllib.loads(build_codex_auth_config(auth))["auth"]
|
|
|
|
assert command["command"] == executable
|
|
assert Path(command["args"][0]).parent == home.resolve()
|
|
|
|
|
|
def test_auth_helpers_are_scoped_to_the_provider_config(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
first = tmp_path / "config.toml"
|
|
second = tmp_path / "project" / ".codex" / "config.toml"
|
|
first_command = tomllib.loads(build_codex_auth_config(auth, config_path=first))["auth"]
|
|
second_command = tomllib.loads(build_codex_auth_config(auth, config_path=second))["auth"]
|
|
first_helper = Path(first_command["args"][0])
|
|
second_helper = Path(second_command["args"][0])
|
|
|
|
assert first_helper != second_helper
|
|
cleanup_codex_auth_helper(auth, config_path=first)
|
|
|
|
assert not first_helper.exists()
|
|
result = subprocess.run(
|
|
[sys.executable, str(second_helper)], capture_output=True, text=True, check=True
|
|
)
|
|
assert result.stdout == "sk-test-only"
|
|
|
|
|
|
def test_auth_helper_creation_failure_is_explicit(tmp_path: Path, monkeypatch) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
|
|
def denied(*args, **kwargs):
|
|
raise PermissionError("read-only directory")
|
|
|
|
monkeypatch.setattr("headroom.providers.codex.install.os.open", denied)
|
|
with pytest.raises(CodexAuthConfigError, match="Check directory permissions"):
|
|
build_codex_auth_config(auth)
|
|
assert not codex_auth_helper_path(auth).exists()
|
|
|
|
|
|
@pytest.mark.parametrize("content", ["invalid TOML", "[model_providers.other]\n{auth}"])
|
|
def test_cleanup_preserves_helper_for_retained_or_unparseable_config(
|
|
tmp_path: Path, content: str
|
|
) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text('{"OPENAI_API_KEY": "sk-test-only"}', encoding="utf-8")
|
|
fragment = build_codex_auth_config(auth)
|
|
(tmp_path / "config.toml").write_text(content.replace("{auth}", fragment), encoding="utf-8")
|
|
|
|
cleanup_codex_auth_helper(auth)
|
|
|
|
assert codex_auth_helper_path(auth).exists()
|
|
|
|
|
|
def test_codex_uses_chatgpt_auth_false_for_missing_or_malformed(tmp_path: Path) -> None:
|
|
assert codex_uses_chatgpt_auth(tmp_path / "absent.json") is False
|
|
bad = tmp_path / "auth.json"
|
|
bad.write_text("not json", encoding="utf-8")
|
|
assert codex_uses_chatgpt_auth(bad) is False
|
|
|
|
|
|
def test_codex_uses_chatgpt_auth_false_for_non_dict_json(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text("[]", encoding="utf-8")
|
|
|
|
assert codex_uses_chatgpt_auth(auth) is False
|
|
|
|
|
|
def test_codex_uses_chatgpt_auth_false_for_empty_object(tmp_path: Path) -> None:
|
|
auth = tmp_path / "auth.json"
|
|
auth.write_text("{}", encoding="utf-8")
|
|
|
|
assert codex_uses_chatgpt_auth(auth) is False
|
|
|
|
|
|
def test_codex_provider_section_supports_custom_markers() -> None:
|
|
section = build_provider_section(
|
|
port=9100,
|
|
name="Headroom init proxy",
|
|
marker_start="# --- start ---",
|
|
marker_end="# --- end ---",
|
|
)
|
|
|
|
assert section.startswith("# --- start ---\n")
|
|
assert section.endswith("# --- end ---\n")
|
|
assert 'base_url = "http://127.0.0.1:9100/v1"' in section
|
|
assert 'env_key = "OPENAI_API_KEY"' not in section
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ChatGPT-auth detection from the id_token claims (#3206)
|
|
#
|
|
# Newer Codex releases can write an auth.json with neither `auth_mode` nor a
|
|
# top-level `tokens.account_id`; the account identity lives only in the
|
|
# id_token claims. Those configs read as API-key mode, so requires_openai_auth
|
|
# is omitted, Codex attaches no Authorization header, and every request 401s
|
|
# with "Missing bearer" -- silently, with doctor reporting green.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _unsigned_jwt(claims: dict[str, object]) -> str:
|
|
import base64
|
|
import json as _json
|
|
|
|
def seg(raw: bytes) -> str:
|
|
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
|
|
|
header = seg(b'{"alg":"none"}')
|
|
payload = seg(_json.dumps(claims).encode("utf-8"))
|
|
return ".".join((header, payload, "sig"))
|
|
|
|
|
|
_CHATGPT_CLAIMS: dict[str, object] = {
|
|
"https://api.openai.com/auth": {
|
|
"chatgpt_account_id": "1a155430-5551-47f4-9c7b-aeab7983f24a",
|
|
"chatgpt_plan_type": "pro",
|
|
}
|
|
}
|
|
|
|
|
|
def _write_auth(tmp_path, document: dict[str, object]): # noqa: ANN001, ANN202
|
|
import json as _json
|
|
|
|
path = tmp_path / "auth.json"
|
|
path.write_text(_json.dumps(document), encoding="utf-8")
|
|
return path
|
|
|
|
|
|
def test_chatgpt_auth_detected_from_id_token_claims_alone(tmp_path) -> None:
|
|
"""The #3206 shape: no auth_mode, no tokens.account_id, only the JWT."""
|
|
path = _write_auth(tmp_path, {"tokens": {"id_token": _unsigned_jwt(_CHATGPT_CLAIMS)}})
|
|
|
|
assert codex_uses_chatgpt_auth(path) is True
|
|
|
|
|
|
def test_explicit_api_key_mode_still_wins_over_a_chatgpt_id_token(tmp_path) -> None:
|
|
"""Guards the #406 regression: API-key users must not get forced OAuth."""
|
|
path = _write_auth(
|
|
tmp_path,
|
|
{"auth_mode": "apikey", "tokens": {"id_token": _unsigned_jwt(_CHATGPT_CLAIMS)}},
|
|
)
|
|
|
|
assert codex_uses_chatgpt_auth(path) is False
|
|
|
|
|
|
def test_api_key_config_without_tokens_is_not_chatgpt(tmp_path) -> None:
|
|
path = _write_auth(tmp_path, {"OPENAI_API_KEY": "sk-test"})
|
|
|
|
assert codex_uses_chatgpt_auth(path) is False
|
|
|
|
|
|
def test_id_token_without_the_chatgpt_claim_is_not_chatgpt(tmp_path) -> None:
|
|
path = _write_auth(tmp_path, {"tokens": {"id_token": _unsigned_jwt({"sub": "user"})}})
|
|
|
|
assert codex_uses_chatgpt_auth(path) is False
|
|
|
|
|
|
def test_malformed_id_token_is_not_chatgpt(tmp_path) -> None:
|
|
for bogus in ("not-a-jwt", "a.b", "a.!!!not-base64!!!.c", ""):
|
|
path = _write_auth(tmp_path, {"tokens": {"id_token": bogus}})
|
|
assert codex_uses_chatgpt_auth(path) is False, bogus
|
|
|
|
|
|
def test_blank_chatgpt_account_id_is_not_chatgpt(tmp_path) -> None:
|
|
claims = {"https://api.openai.com/auth": {"chatgpt_account_id": " "}}
|
|
path = _write_auth(tmp_path, {"tokens": {"id_token": _unsigned_jwt(claims)}})
|
|
|
|
assert codex_uses_chatgpt_auth(path) is False
|
|
|
|
|
|
def test_legacy_account_id_still_detected(tmp_path) -> None:
|
|
path = _write_auth(tmp_path, {"tokens": {"account_id": "acct-123"}})
|
|
|
|
assert codex_uses_chatgpt_auth(path) is True
|
|
|
|
|
|
def test_provider_block_emits_requires_openai_auth_for_the_new_shape(tmp_path) -> None:
|
|
"""End of the chain: the JWT-only shape must produce the key Codex needs."""
|
|
path = _write_auth(tmp_path, {"tokens": {"id_token": _unsigned_jwt(_CHATGPT_CLAIMS)}})
|
|
|
|
block = build_provider_section(
|
|
port=8787,
|
|
name="Headroom",
|
|
include_markers=False,
|
|
requires_openai_auth=codex_uses_chatgpt_auth(path),
|
|
)
|
|
|
|
assert "requires_openai_auth = true" in block
|