Commit Graph
3 Commits
Author SHA1 Message Date
994a89f169 fix(proxy): bound request-log message payloads to the feed window; trim glibc too (#3717)
## Description

A user running an overnight Claude Code session through the proxy on
Linux reported it consuming 100 GB of a 128 GB host (about 3,500
requests at ~92k input tokens each, `--log-messages` on). Two stacked
causes, both fixed here:

1. `RequestLogger` keeps message payloads on all 10,000 entries. With
`--log-messages`, every `RequestLog` in the `MAX_LOG_ENTRIES` deque
holds `request_messages`, `compressed_messages` and `response_content`:
two parsed copies of the whole conversation, ~1.9x their JSON size as
Python objects. The only reader of those payloads is
`/transformations/feed`, which is hard-capped at 100 entries, so the
other 9,900 are retained for nothing and grow linearly for the life of
the process.
2. The periodic allocator trim (#2820, #2879) defaults on only for
macOS. glibc retains freed pages the same way, and per-thread arenas on
a many-core host multiply the high-water mark.

Not changed: `MAX_LOG_ENTRIES` itself (`_build_stats_payload` reads the
light fields of all 10,000), and `get_recent_with_messages(n)` for `n >
100`, which now returns bodies only for the newest 100, matching what
the feed could ever serve.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/proxy/request_logger.py`: `MESSAGE_WINDOW = 100`; on every
append, null the three payload fields on the entry that just left the
window. Light fields stay on every entry, so `get_recent` and `/stats`
are unchanged.
- `headroom/proxy/models.py`, `headroom/proxy/server.py`:
`periodic_malloc_trim_enabled` defaults on for `darwin` and `linux` (was
`darwin` only). Platforms without a trim call already disable the task
themselves; `HEADROOM_MALLOC_TRIM=0` still opts out.
- `tests/test_proxy/test_request_logger.py`: window tests (payloads
nulled beyond the newest 100, light fields intact and ordered, feed path
fully populated, short logs untouched).
- `tests/test_malloc_tuning.py`: default-scope test asserts darwin and
linux on, win32 off.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --frozen --extra dev pytest tests/test_proxy/test_request_logger.py tests/test_malloc_tuning.py -q
22 passed in 1.10s
$ uvx ruff check headroom/proxy/request_logger.py headroom/proxy/models.py headroom/proxy/server.py tests/test_proxy/test_request_logger.py tests/test_malloc_tuning.py
All checks passed!
$ uvx ruff format --check <same files>
5 files already formatted
$ uv run --frozen --extra dev mypy headroom/proxy/request_logger.py headroom/proxy/models.py
Success: no issues found
```

## Real Behavior Proof

- Environment: macOS 15.6 (Darwin 24.6.0), Python 3.12, this branch at
ba5f99d7, `uv run --frozen`.
- Exact command / steps: Logged 2,000 `RequestLog` entries through a
real `RequestLogger(log_full_messages=True)`, each carrying a
300-message, ~128 KB conversation in both `request_messages` and
`compressed_messages`, under `tracemalloc`; once with the fix
(`MESSAGE_WINDOW = 100`) and once with the window set to 10,000 to
reproduce the pre-fix retention.
- Observed result: fix: 41 MB live, 100 entries with bodies; pre-fix:
802 MB live, 2,000 entries with bodies. Same 2,000 entries remain in the
deque either way. Extrapolated to the reporter's 10,000-entry cap and
~1.5 MB per entry, the pre-fix retention alone is ~15 GB before
allocator bloat.
- Not tested: end-to-end RSS on a Linux host with the trim default
flipped (no Linux machine available; the trim path itself is covered by
the existing `test_malloc_tuning.py` suite and was measured on macOS in
#2820).

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, two defaults. (1) Request-log
entries older than the newest 100 no longer carry message payloads; only
`/transformations/feed` reads them and it is capped at 100, so no served
response changes. (2) The periodic allocator trim is now on by default
on glibc Linux, as it already was on macOS.
- Kill switch / disable path: `HEADROOM_MALLOC_TRIM=0` disables the
trim. The payload window has no switch; it changes no output.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert the PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from the Conventional Commit PR title (a CI guard
enforces this)

## Screenshots (if applicable)

N/A

## Additional Notes

Documentation: no user-facing docs describe `MAX_LOG_ENTRIES` or the
trim default, so nothing to update. Two commits, one per cause, in case
the maintainers want the Linux trim default separately.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-24 10:51:02 -05:00
Tejas ChopraandTejas Chopra 96c25f5181 fix(cli): stop the macOS malloc re-exec replacing an embedder's process (#3064)
## Description

**`main` cannot currently run its own test suite on macOS.** `pytest
tests/` dies at roughly 2% with exit code 2 — no traceback, no summary,
no failing test named. The pytest process is simply gone.

Two independent defects, both landed today, both invisible to CI.

### 1. The macOS malloc re-exec replaces the calling process

`headroom proxy` re-execs itself once on Darwin to apply two libmalloc
knobs that libmalloc only reads before `main()` (#2820, PR #2879):

```python
os.execv(sys.executable, [sys.executable, "-m", "headroom.cli", *sys.argv[1:]])
```

That reconstruction is only faithful when the process really *is* the
Headroom CLI. Ten-plus test files invoke the `proxy` command in-process
through Click's `CliRunner`. There, `os.execv` replaces **pytest** with
a Headroom process holding pytest's argv. Run with `-s`, the mechanism
is visible:

```
tests/test_agent_savings.py Usage: python -m headroom.cli [OPTIONS] COMMAND [ARGS]...
Error: No such command 'tests/test_agent_savings.py::test_proxy_cli_reads_agent_90_profile_env'.
```

Everything after the first such test — roughly 98% of the suite — never
runs. The same hazard applies to any application embedding the CLI.

**The documented kill switch does not help.** `tests/conftest.py:41`
scrubs every `HEADROOM_*` variable for hermeticity, so
`HEADROOM_MALLOC_TUNING` is deleted before the guard reads it. Only the
private `_HEADROOM_MALLOC_TUNED` survives, because it starts with an
underscore.

**CI could not have caught this.** The tuning is Darwin-only, and while
the repo *does* have macOS jobs (`macos-native-wrapper`, `wrap-native
(macos-latest)`), neither runs the Python test suite — the `test` shards
are `ubuntu-latest` only. So `sys.platform != "darwin"` returns first
everywhere pytest actually runs. #2879 merged with 37 green checks.

### 2. A semantic merge conflict between two green PRs

#3051 added `bind_scope(tags, request.scope)` at `gemini.py:325` and
updated the three Gemini fakes it knew about. #3035 branched earlier and
added a fourth `_FakeRequest` without `.scope`. Each was green against
its own base; together they fail:

```
AttributeError: '_FakeRequest' object has no attribute 'scope'
```

Git merged both cleanly. Only running the suite on merged `main`
surfaces it.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Changes Made

- Added `_process_is_headroom_cli_entrypoint()`: the re-exec now
verifies its own precondition — `argv[0]` must be the `headroom` console
script or `headroom/cli/__main__.py`.
- The embedded path returns **before** stamping
`_HEADROOM_MALLOC_TUNED`, so a genuine CLI child inheriting the
environment can still apply the tuning.
- Gave the Gemini `_FakeRequest` the `.scope` every real Starlette
`Request` carries.
- `test_reexec_skips_when_operator_already_set_vars` now sets a
realistic `argv[0]`, matching its sibling exec test.
- New `tests/test_cli_proxy_malloc_reexec_guard.py` asserting the
guard's logic on **every** platform, since no CI runner is macOS.

## Testing

- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes (`uv run mypy headroom`) — not run
- [x] New tests added for new functionality

### Test Output

Before, on `main`:

```text
$ .venv/bin/python -m pytest tests/ -q
collected 11622 items / 8 skipped
... tests/test_agent_savings.py ............................
$ echo $?
2
```

No summary line — the run does not end, it is replaced.

After, on this branch:

```text
$ .venv/bin/python -m pytest tests/ -q
3 failed, 11055 passed, 581 skipped, 6034 warnings in 303.69s (0:05:03)
```

All three remaining failures reproduce at `f9807fd6`, before today's
merges, and are unrelated:

| test | cause |
|---|---|
|
`test_graceful_shutdown::test_run_server_installs_cancelled_error_filter`
| full-suite ordering; passes in isolation (11 passed) |
|
`test_learn/test_integration::TestCodexIntegration::test_full_pipeline`
| pre-existing |
| `test_release_workflows::test_no_native_tls_in_wheel_build_tree` |
requires `cargo`, absent on this host |

## Real Behavior Proof

- Environment: macOS 15 (darwin 25.4.0), Python 3.12.13, arm64, real
checkout of `main` at `ef7e07e0`.
- Exact command / steps: bisected the crash to a single test, then to a
single commit — `be5b26d8` (parent) exits 0, `6d87825f` (#2879) exits 2.
Confirmed causation by temporarily replacing the `os.execv` line with
`return`, which makes the test pass. Recovered the mechanism by running
the crashing test with `-s`, which prints the Headroom CLI rejecting
pytest's own argv.
- Observed result: on `main` the suite cannot reach a summary; on this
branch it completes with 11,055 passing. The two-file reproduction
(`test_agent_savings.py` + `test_anthropic_beta_session_sticky.py`) goes
from exit 2 to 62 passed.
- Not tested: a real `headroom proxy` launch on macOS confirming
libmalloc still receives the knobs after re-exec. The guard is covered
by unit tests asserting `execv` is still called with `["-m",
"headroom.cli", "proxy", "--port", "8787"]` for a console-script
`argv[0]`, but I have not watched `vmmap` on a live proxy. **A macOS
maintainer should confirm #2820's RSS fix still works end to end before
this ships.**

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no for a real CLI launch; the re-exec
no longer fires when the CLI is invoked in-process, which was never
intended to work.
- Kill switch / disable path: `HEADROOM_MALLOC_TUNING=0` still disables
the tuning outright.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit — but that restores a `main` whose
test suite cannot run on macOS.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

**This is my fault and worth recording.** I merged both #2879 and #3035
earlier today on the rule "approved + green CI". Both were genuinely
approved and genuinely green. Neither was rebased onto current `main`
first, and CI has no macOS runner, so green meant less than it appeared
to.

Two process gaps this exposes, neither of which this PR fixes:

1. **The Python test suite never runs on macOS.** The repo has macOS
jobs (`macos-native-wrapper`, `wrap-native (macos-latest)`), but the
`test` shards are `ubuntu-latest` only, so Darwin-only code paths — the
allocator tuning is one, `wrap` has others — are unreachable by pytest
in CI. Even a reduced macOS shard would have caught this.
2. **Nothing requires a PR to be current with `main` before merging.**
Both defects here are cross-PR interactions that no per-PR check can
see. Enabling "require branches to be up to date before merging" on
`main` would have forced a rebase and surfaced the Gemini fake.

I would suggest an issue for each rather than folding them in here.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
2026-08-16 17:56:10 -07:00
Abhay Singh 6d87825f62 fix(proxy): tune macOS libmalloc and trim allocator pages so long-lived RSS stays bounded (#2879)
## Summary

Fixes #2820.

Prevents long-lived macOS proxies from retaining every largest transient
request-body allocation in libmalloc. The reporter’s production A/B
isolated the allocator behavior and verified the two pre-main libmalloc
knobs; this PR applies them through a one-time Darwin-only re-exec and
adds periodic per-worker pressure relief.

- `MallocAggressiveMadvise=1` returns freed pages eagerly.
- `MallocLargeCache=0` disables the large-allocation death-row cache.
- Operator-set allocator variables are preserved;
`HEADROOM_MALLOC_TUNING=0` is the kill switch.
- Periodic trim defaults on only for macOS, runs off the event loop,
performs no forced Python GC, validates its interval, and is
retained/cancelled through the app lifecycle.
- Non-Darwin behavior remains unchanged unless explicitly enabled.
- Semantically rebased onto current `main`, retaining startup dependency
validation, MCP SDK v1 compatibility, and all newer proxy behavior.

## Verification

- 147 proxy CLI/config/malloc/MCP-contract tests pass; 1 platform skip.
- Ruff check and formatting clean; `git diff --check` clean.
- The reporter’s macOS A/B reduced dirty empty malloc regions to zero
and lowered steady/startup RSS; the control flow and shutdown lifecycle
are covered locally.

## Safety

The re-exec is Darwin-only, PID-preserving, loop-guarded, and opt-out.
The trim task is per worker because allocator state is per process, and
shutdown cancels it explicitly.
2026-08-16 15:04:54 -07:00