Commit Graph
3 Commits
Author SHA1 Message Date
e15e342d9b fix(proxy): resolve the forwarded client from the right and keep settings writes loopback-only (#4847)
## Description

Two fixes for proxies run with `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`
(opt-in, empty by default):

1. **Client address.** The proxy took the *leftmost* `X-Forwarded-For`
element as the client. A gateway that appends to the header keeps
whatever the client itself sent on the left, so the client could pick
its own address (for example `127.0.0.1`). The client is now the
right-most hop that is not itself inside a trusted-gateway CIDR. If
every hop is trusted it falls back to the leftmost, as before.
2. **Settings writes.** A peer inside a trusted-gateway CIDR counted as
loopback in `_request_is_loopback`, so `POST /settings` and `POST
/settings/apply` (which restarts the proxy) were open to anything routed
through the gateway. That rule exists so a containerised dashboard can
show its read-only views. Writes now need a real loopback peer, or a
client inside `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` (the
existing #2466 path, with IP-literal Host and same-origin checks).

**Behaviour change:** someone who runs Headroom in a Docker bridge
container, sets only `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`, and saves
settings from the host browser via `localhost` will now get 404 on save.
Reads still work. To keep saving, add the gateway address to
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` (e.g. `172.17.0.1/32`)
and open the dashboard at `http://127.0.0.1:<port>`. That path is
checked below. Without the env var nothing changes.

Fix adapted from a proposal by HetCreep.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/proxy/forwarded_policy.py`: new `header_client()` walks
`X-Forwarded-For` from the right and skips trusted-gateway hops.
`resolve_forwarded_headers` uses it.
- `headroom/proxy/server.py`: `_request_is_loopback` /
`_request_can_view_dashboard_metadata` take `allow_gateway_peer`
(default unchanged). `POST /settings` and `POST /settings/apply` use a
new write guard with `allow_gateway_peer=False`, and so does the
same-origin guard for those routes.
- Tests: the leftmost-XFF test now expects the right-to-left resolution.
Added tests for a client-written XFF, the all-trusted fallback, settings
reads still served to a gateway peer, writes refused to a gateway peer
(both routes), and writes still allowed on loopback.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# new tests against origin/main (c21b0813)
$ pytest tests/test_forwarded_headers.py tests/test_proxy_loopback_gating.py
FAILED tests/test_forwarded_headers.py::test_client_written_x_forwarded_for_is_not_the_client
FAILED tests/test_proxy_loopback_gating.py::test_settings_write_refused_for_trusted_gateway_peer[/settings]
FAILED tests/test_proxy_loopback_gating.py::test_settings_write_refused_for_trusted_gateway_peer[/settings/apply]
================== 3 failed, 189 passed, 5 warnings in 46.16s ==================

# this branch
$ pytest tests/test_forwarded_headers.py tests/test_proxy_loopback_gating.py tests/test_proxy_hardening.py tests/test_header_isolation.py tests/*settings* tests/*dashboard*
4 failed, 456 passed, 1 skipped
# the 4 failures are tests/test_dashboard/test_live_feed.py (Playwright, needs a live server
# on :8787: net::ERR_CONNECTION_REFUSED). They fail the same way on origin/main.

$ ruff check / ruff format --check (0.17.3) on changed files: All checks passed!
$ mypy headroom/proxy/forwarded_policy.py headroom/proxy/server.py --ignore-missing-imports
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, `create_app(ProxyConfig(...))` driven
by Starlette TestClient with a set peer address.
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS=172.17.0.0/16`.
- Exact command / steps: resolve the client for peer `172.17.0.9` with
`X-Forwarded-For: 127.0.0.1, 203.0.113.9`. Then GET/POST `/settings` as
peer `172.17.0.9` with `Host: localhost:8787`, and POST as peer
`127.0.0.1`.
- Observed result:
```text
                         origin/main   this branch
client ip                127.0.0.1     203.0.113.9
gateway peer GET  /settings   200          200
gateway peer POST /settings   200          404
loopback     POST /settings   200          200
```
Opt-in path: with
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS=172.17.0.1/32`, peer
`172.17.0.1`, `Host: 127.0.0.1:8787`, same-origin `Origin`: `POST
/settings` → 200.
- Not tested: a real Docker bridge deployment behind a real reverse
proxy.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: see Description
- Kill switch / disable path: none needed
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

`_auth_rejection_context` still logs the leftmost hop. That is log-only,
documented as unverified, and not used for any decision, so it is left
as is.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4847"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:45:42 +05:30
JervisandJerrettDavis 0537cbfde4 feat(dashboard): persist lifetime proxy metrics (#2198)
## Description

Persist bounded, aggregate-only Lifetime dashboard metrics across proxy
restarts and expose them through a new `/stats-lifetime` endpoint. The
change keeps session/runtime stats separate from durable lifetime stats,
gates sensitive dashboard metadata for loopback or explicitly trusted
dashboard clients, and updates the dashboard Lifetime view to consume
the new endpoint.

Closes #2137

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- Added bounded persistent lifetime metrics state and wired proxy metric
events into it.
- Added `/stats-lifetime` with sensitive project/persistence details
gated behind dashboard metadata access checks.
- Extended loopback/dashboard metadata access policy for trusted
dashboard client CIDRs without widening admin/debug endpoints.
- Reorganized dashboard session/lifetime presentation around runtime
counters versus durable aggregates.
- Added focused tests for persistent aggregation, persistence, endpoint
registration, loopback gating, trusted dashboard CIDRs, and recent
request ordering.
- Fixed current Ruff/mypy issues in the lifetime metrics normalization
code.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run --extra proxy --with pytest --with pytest-asyncio pytest tests/test_persistent_metrics.py tests/test_persistent_metrics_integration.py tests/test_persistent_metrics_persistence.py tests/test_proxy_loopback_gating.py tests/test_proxy_stats_recent_requests.py -q
53 passed, 1 warning

uvx ruff==0.15.17 check headroom/proxy/forwarded_headers.py headroom/proxy/loopback_guard.py headroom/proxy/persistent_metrics.py headroom/proxy/savings_tracker.py headroom/proxy/server.py tests/test_forwarded_headers.py tests/test_persistent_metrics.py tests/test_persistent_metrics_integration.py tests/test_persistent_metrics_persistence.py tests/test_proxy_loopback_gating.py tests/test_proxy_project_savings.py tests/test_proxy_stats_recent_requests.py
All checks passed!

uv run --extra proxy --with mypy mypy headroom/proxy/persistent_metrics.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows review worktree, Python 3.13.3 via uv.
- Exact command / steps: Ran the focused persistent metrics,
persistence, loopback gating, and recent request tests; ran CI-matching
Ruff on touched files; ran mypy on the new persistent metrics module.
- Observed result: `/stats-lifetime` is registered, non-loopback callers
receive only non-sensitive aggregate data, loopback/trusted dashboard
clients receive the full lifetime payload, admin/debug endpoints remain
loopback-only, and persistent metrics normalize malformed stored state
without type/lint errors.
- Not tested: Full repository pytest suite, full dashboard browser
screenshot pass, or live long-running proxy traffic.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

No changelog entry is required for this dashboard/internal metrics
iteration. The endpoint intentionally exposes only aggregate lifetime
data to ordinary network callers and strips project/persistence details
unless the caller passes the dashboard metadata access policy.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 18:18:13 +00:00
chopratejas 665c0823d9 fix(proxy): F4 — trust X-Forwarded-* only behind allow-listed gateway
The proxy previously had no spoofing defence on `X-Forwarded-*`
headers. Any client could forge `X-Forwarded-For`, `-Proto`, or
`-Host` and rotate rate-limit buckets / poison logs / mis-attribute
auth. F4 adds an explicit gateway-allowlist gate.

* New `headroom/proxy/forwarded_headers.py` module:
  - `resolve_client_ip(request)` — IP for log/auth/rate-limit.
  - `trusted_forwarded_headers(request)` — sanitised `{for,proto,host}`
    dict (empty strings when the gate fails).
  - Configured by `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`
    (comma-separated CIDRs). Empty/unset = strict-secure default.
  - Malformed CIDR raises `ValueError` (loud, no silent skip).
  - Every spoof rejection emits `forwarded_headers_rejected` log
    event with `peer_ip` + the three header values.
  - Result cached on `request.state.{client_ip,forwarded}` so the
    helpers run once per request.

* Anthropic handler now keys the rate-limit bucket on the gated IP.

* 36 tests cover: default-strict / allow-listed peer / non-allow-listed
  peer (with log assertion) / IPv6 / IPv4-mapped IPv6 / multi-CIDR /
  whitespace tolerance / malformed CIDR loud-fail / empty headers /
  caching / FastAPI integration.
2026-05-06 14:37:09 -07:00