Commit Graph
3 Commits
Author SHA1 Message Date
Sabee Ur Rehman KhanandJerrettDavis 58b1454282 fix(settings): report live proxy configuration (#3177)
## Description

The settings schema re-reads process environment values after the CLI
has already resolved the live proxy configuration. Savings-profile
defaults seeded later in startup can therefore make `/settings/schema`
report values that are not actually active. For example, `--lossless`
can produce `ProxyConfig.lossless=True` while the schema reports the
seeded `HEADROOM_LOSSLESS=0`.

This change makes the live `ProxyConfig` the highest-precedence settings
source, preserves provenance for profile-generated environment defaults,
and adds the missing `HEADROOM_MODE` registry field.

Closes #3090

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `HEADROOM_MODE` with canonical `token` and `cache` choices.
- Made live `ProxyConfig` values take precedence in `/settings/schema`.
- Added `runtime_override` metadata for live values that differ from
lower tiers.
- Prevented savings-profile defaults from being labeled as explicit
environment overrides.
- Preserved provenance through multi-worker configuration serialization.
- Added regression tests for precedence, provenance, endpoint reporting,
and worker round-tripping.

## Testing

- [ ] Unit tests pass (`pytest`) — focused settings tests pass; full
suite not run
- [x] Linting passes (`ruff check .`) — changed files checked
- [ ] Type checking passes (`mypy headroom`) — not run
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ focused settings-store test run
36 passed, 1 deselected in 0.09s

$ direct savings-profile provenance check
seed provenance check passed (18 generated keys)

$ ruff check <8 changed files>
All checks passed!

$ ruff format --check <8 changed files>
8 files already formatted

$ python -m compileall -q <8 changed files>
# No output

$ git diff --check
# No output
```

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.12.13, source based on upstream
`5e0ce242` (v0.36.2).
- Exact command / steps: Set profile-seeded `HEADROOM_MODE=cache` and
`HEADROOM_LOSSLESS=0`, then generated the schema with live values
`mode=token` and `lossless=True`, excluding those generated keys from
explicit-environment provenance.
- Observed result: The schema reports `mode=token` and `lossless=true`;
both generated variables report `env_override=false`, and both
differences report `runtime_override=true`.
- Not tested: Complete repository and FastAPI endpoint suites. This
environment lacks the compiled `headroom._core` extension and Rust
toolchain, and dependency downloads are unavailable.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Proxy compression behavior is
unchanged; only settings registry/schema reporting changes.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: The dashboard receives the actual live proxy
values.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — N/A; no
user documentation change required
- [ ] My changes generate no new warnings — focused checks pass; full
suite not run
- [x] I have added tests that prove my fix is effective
- [ ] New and existing unit tests pass locally — focused tests pass;
full suite not run
- [x] I did **not** edit `CHANGELOG.md` — generated by release-please

## Screenshots (if applicable)

N/A — settings API behavior only.

## Additional Notes

This is intentionally narrow for #3090. PR #2346 overlaps by adding
`HEADROOM_MODE`, but currently includes unrelated dashboard/runtime
work; this PR isolates the live-config precedence and provenance
correction.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-01 12:25:26 -05:00
AxelRay de9e0523da fix(settings): accept documented HEADROOM_* env names as settings keys (#2833)
## Description

Settings validation only accepted short JSON/API keys, so documented
HEADROOM_* env names were rejected as unknown. Users following the docs
(for example HEADROOM_LOSSLESS) hit SettingsValidationError / PUT
/settings 400 even though those names are already on each registry
field.

This normalizes known env aliases to their short keys before
validate/save, keeps existing short-key behavior, and rejects
conflicting env+key pairs for the same field.

Closes #2812

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added _BY_ENV and _normalize_values() in settings_store to map
documented env names to short keys
- Call normalization at the start of validate() and save() so
clear/retain paths also accept env aliases
- Reject payloads that supply both an env alias and its short key with
different values
- Add unit coverage for accept/clear/conflict/same-value paths and
update registry monkeypatches to rebuild _BY_ENV

## Testing

- [x] Unit tests pass (pytest)
- [x] Linting passes (ruff check .)
- [ ] Type checking passes (mypy headroom)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
python -m pytest tests/test_proxy/test_settings_store.py -q -k "env_alias or validate_accepts or save_rejects or same_env or conflicting or save_accepts or env_alias_clear or anthropic_extra_headers_retain or TestValidation"
23 passed, 11 deselected

ruff check headroom/settings_store.py tests/test_proxy/test_settings_store.py tests/test_proxy_settings_endpoints.py
All checks passed!

ruff format --check headroom/settings_store.py tests/test_proxy/test_settings_store.py tests/test_proxy_settings_endpoints.py
3 files already formatted
```

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.14.5 via contributor venv,
worktree of headroom main at 7940c05e plus commit e4c87edf
- Exact command / steps: pytest tests/test_proxy/test_settings_store.py
focused selection; ruff check and ruff format --check on the three
touched files; settings_store.validate({"HEADROOM_LOSSLESS": True})
returns {"lossless": True}
- Observed result: Env aliases coerce and persist under short keys;
unknown short keys still error; conflicting env+key pairs raise
SettingsValidationError; ruff clean on touched files
- Not tested: Live dashboard PUT /settings through a running proxy (HTTP
suite needs native headroom._core); mypy; full monorepo CI

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did not edit CHANGELOG.md - it is generated by release-please
from my Conventional Commit PR title (a CI guard enforces this)

## Screenshots (if applicable)

N/A

## Additional Notes

- Scoped to settings key normalization only
- Registry drift Click test was not exercised here because this
environment lacks tomlkit for an unrelated import path
2026-08-11 17:22:27 -05:00
nangsontayandJerrettDavis e5b3a634df fix(proxy): dedupe Codex WS request logging for accurate mixed-provider dashboards (#2189)
## Description

Running Claude Code (Anthropic) and Codex (OpenAI) against the **same**
Headroom proxy instance on one port produced incorrect, unstable
dashboard data. The proxy core is provider-isolated and
multi-provider-safe by design; the defect was in the observability
layer. The Codex `/v1/responses` **WebSocket** handler was the only path
in the proxy that wrote to the request logger by hand instead of through
the unified `emit_request_outcome` funnel, and it did so twice per
session close: the per-turn funnel record plus an unconditional
cumulative session-summary `RequestLog`. This PR removes the duplicate
summary log so Codex WS emits exactly one request log per turn, matching
the HTTP provider paths.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Dropped the duplicate cumulative session-summary `RequestLog` in the
Codex WS handler while preserving the per-turn `emit_request_outcome`
path.
- Preserved gated `request_messages` and `turn_id` on residual outcomes
so dashboard telemetry keeps the useful attribution without
double-counting tokens.
- Ensured explicit `--anyllm-provider` wins over a leaked
`HEADROOM_ANYLLM_PROVIDER` environment variable.
- Registered retry delay settings that had drifted out of the settings
registry.
- Hardened tests against developer-shell `HEADROOM_*` /
`ANTHROPIC_CUSTOM_HEADERS` leakage and stabilized several focused
proxy/wrap test fixtures.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/pytest tests/ -q -p no:cacheprovider
8529 passed, 537 skipped, 5831 warnings in 276.25s (0:04:36)

$ .venv/bin/ruff check <touched files>
All checks passed!
```

## Real Behavior Proof

- Environment: macOS (Darwin 25.4.0), Python 3.13.14, pytest 9.0.3, ruff
via project venv, branch `fix/multi-provider-runtime`.
- Exact command / steps: Ran the full test suite without pytest cache
provider and Ruff on all touched files; used `git stash` to confirm the
stale fake-config failures pre-existed this change.
- Observed result: Full suite passed with no failures; Ruff passed;
Codex WS now routes end-of-session logging through
`emit_request_outcome`, emitting one request log per turn with the same
accounting model as Anthropic HTTP turns.
- Not tested: Live simultaneous Claude + Codex dashboard run. `mypy
headroom` was not run to completion; a scoped run reported one
pre-existing `settings_store.py:470` coercion error outside this diff.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A - server-side observability fix; no UI markup changed.

## Additional Notes

- The proxy's multi-provider routing, header/auth isolation, and
per-model cache keying are already correct and unchanged here; only the
WS observability write path was double-counting.
- Architectural assessment:
`plans/reports/research-260714-0004-multi-provider-upstream-compression-report.md`;
root-cause + resolution trail:
`plans/reports/debug-assessment-260714-0011-dashboard-instability-mixed-claude-codex-report.md`.
- No live simultaneous Claude + Codex dashboard run was performed;
validation is from test coverage and code review of the WS logging path.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 18:18:34 +00:00