Files
headroom/tests/test_cli/test_wrap_proxy_detach.py
Raúl 3d3c629a9f fix(proxy): write per-port proxy logs so concurrent instances stop truncating each other (#3204)
## Description

Every proxy instance wrote the same two fixed files:
`~/.headroom/logs/proxy.log` (runtime log, via the `RotatingFileHandler`
in `_setup_file_logging`) and `~/.headroom/logs/proxy-stdio.log`
(subprocess stdout/stderr from `_start_proxy`). The name is a constant
(`paths.py` `_PROXY_LOG_FILE = "proxy.log"`), so N concurrent proxies on
different ports (wrapped copilot, droid, claude; plus any persistent
`install agent` deployment) run N independent rotating handlers over the
*same* file and truncate/rename each other's history.

This fix keys both files by port (`proxy-<port>.log`,
`proxy-stdio-<port>.log`) so each proxy rotates only its own file.
`headroom perf` aggregates every per-port file and still reads a lone
legacy `proxy.log` when that is all that exists (backward compat); the
`proxy-stdio-*` captures (raw stdout/stderr, not PERF records) are
excluded from the parser.

This matters more now that wrap-spawned proxies self-exit after an idle
grace (orphan-proxy watchdog, #3202): the process is gone, so its
per-port log is the only forensic evidence and must survive an unrelated
proxy's rotation.

Closes #3203

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/paths.py`: `proxy_log_path(port=None)` and new
`proxy_stdio_log_path(port=None)` are the single source of truth for the
filenames (`proxy-<port>.log` / `proxy-stdio-<port>.log` with a port,
legacy names without). The `_PROXY_LOG_FILE` constant stays only as the
readers' fallback name.
- `headroom/proxy/helpers.py`: `_setup_file_logging(port=None)` targets
`proxy_log_path(port)`. The duplicate-handler guard runs BEFORE the
`RotatingFileHandler` is constructed (construction opens the file), so a
second `create_app` on a different port in one process cannot leave an
empty stray `proxy-<port>.log` or leak an fd; a handler pointing at a
different port is replaced and closed so per-port keying holds.
- `headroom/proxy/server.py`: `create_app` resolves `config` first, then
calls `_setup_file_logging(config.port)` (the proxy always knows its own
port).
- `headroom/cli/wrap.py`: `_get_log_path(port=None)` /
`_get_proxy_stdio_log_path(port=None)` are per-port; `_start_proxy`
passes its `port`. The stdio filename comes from
`paths.proxy_stdio_log_path` (single source) and its directory from
`_get_log_path`.
- `headroom/perf/analyzer.py`: `parse_log_files` selects PERF logs by a
positive filename match (`_PERF_LOG_FILE_RE` =
`proxy(-<digits>)?.log(.N)?`) rather than a `proxy-stdio` blacklist, so
`proxy-stdio-*.log` and any other `proxy-<word>.log` are never ingested;
ordering is deterministic (mtime, then path).
- `headroom/cli/perf.py`, `wiki/cli.md`: doc/help text updated for
per-port + legacy.
- `tests/test_perf_per_port_logs.py` (new): per-port path helpers; a
second `_setup_file_logging` on a new port reroutes cleanly with no
cross-file leakage (regression for the guard-ordering bug); and
`parse_log_files` aggregates per-port + legacy while excluding stdio and
other non-PERF `proxy-*.log`.
- `tests/test_cli_proxy_env.py`, `tests/test_agent_savings.py`,
`tests/test_cli/test_wrap_{codex,claude_vertex_proxy_env,proxy_detach}.py`:
the `_get_log_path` monkeypatches take the new optional `port`, and the
two stdio-filename assertions/fixtures in `test_cli_proxy_env.py` use
the per-port name.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`) — plus `ruff format --check`
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_perf_per_port_logs.py tests/test_cli_proxy_env.py tests/test_agent_savings.py -q -p no:randomly
115 passed, 1 skipped in 4.44s

$ .venv/bin/python -m pytest tests/test_cli_perf_format.py tests/test_learn/test_analyzer.py -q
107 passed in 4.16s

$ .venv/bin/python -m pytest tests/test_cli/test_wrap_proxy_detach.py tests/test_cli/test_wrap_codex.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py -q
120 passed in 171.19s

$ .venv/bin/ruff check <touched> && .venv/bin/ruff format --check <touched>
All checks passed!

$ .venv/bin/mypy headroom/paths.py headroom/cli/wrap.py headroom/proxy/helpers.py headroom/proxy/server.py headroom/perf/analyzer.py
Success: no issues found in 5 source files
```

Rust gate (`cargo clippy/test/fmt`) not run: Python-only change, no
crate touched.

## Real Behavior Proof

- Environment: macOS 26.5 arm64, Python 3.13, local editable source
build.
- Exact command / steps: `headroom proxy --port 8890` and `headroom
proxy --port 8891` started concurrently; inspected `~/.headroom/logs/`;
then `headroom perf --hours 1 --format json`.
- Observed result: each proxy wrote its own file (`proxy-8890.log`,
`proxy-8891.log`, independent), the pre-existing shared `proxy.log` was
not touched, and `headroom perf` reported `log_files_read: 4`,
aggregating per-port + legacy with no error.
`test_setup_file_logging_reroutes_on_port_change` additionally drives
the production helper twice in one process and asserts each port's
records land only in its own file.
- Not tested: Windows path semantics (filename construction is
platform-independent `pathlib`); a legacy `proxy.log` written by an OLD
binary running beside a new per-port one end to end (readers handle it,
but not exercised live).

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A (no flag; readers accept both layouts).
- Stable/default behavior changed: log filenames change from `proxy.log`
to `proxy-<port>.log`; `headroom perf` reads both, so history is not
orphaned.
- Kill switch / disable path: N/A (no flag).
- Unsafe override required: no.
- Qualification impact: none — readers stay backward compatible with a
lone legacy `proxy.log`.
- Rollback path: revert; a downgraded binary writes legacy `proxy.log`
again and readers still aggregate any per-port files left behind.

Disk growth: each per-port family keeps the prior bound (`maxBytes=10 MB
× backupCount=5` = 60 MB), but there is now one family per distinct port
used instead of one global family. Ports are usually stable (default
8787; wrap auto-selects a nearby free port), so in practice a handful of
families; the theoretical worst case is bounded by the port scan
(`_find_available_port`, ≤100) → ≤ ~6 GB. No automatic pruning; stale
`proxy-<port>.log*` families can be deleted safely (perf reads whatever
remains). A retention policy is intentionally out of scope.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — log-filename change; no UI.

## Additional Notes

Follow-up (separate SDK PR, not here): the TypeScript SDK helper
`sdk/typescript/src/paths.ts::proxyLogPath()` still returns legacy
`proxy.log` with no port argument and is pinned by `paths.test.ts`. It
mirrors `paths.py`, so it should gain an optional port argument in an
SDK PR (kept separate to stay one-surface, and because that package's
vitest suite is the right gate for it).
2026-09-09 15:54:34 -05:00

147 lines
5.9 KiB
Python

"""Detachment of the shared proxy subprocess in ``headroom wrap``.
``_start_proxy`` launches the proxy that every wrapped agent on a port shares.
It must outlive an *ungraceful* close of the agent that happened to start it
(closing the terminal window, taskkill, a crash) -- otherwise the OS tree-kills
the proxy and breaks the other live clients, bypassing the marker-based
reference counting in ``_make_cleanup``.
On Windows that means detaching from the launcher's console and Job object via
creation flags; on POSIX ``start_new_session`` already detaches via setsid().
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
import pytest
from headroom.cli import wrap as wrap_cli
# Windows-only creation flag; not always exported by the host's ``subprocess``.
_CREATE_BREAKAWAY_FROM_JOB = 0x01000000
class _FakeProc:
"""Stand-in for a live proxy process (``poll() is None``)."""
returncode = 0
def poll(self) -> None:
return None
def _capture_popen_kwargs(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> dict[str, Any]:
"""Invoke ``_start_proxy`` with all I/O stubbed; return the Popen kwargs."""
captured: dict[str, Any] = {}
def _fake_popen(cmd: Any, **kwargs: Any) -> _FakeProc:
captured.clear()
captured.update(kwargs)
return _FakeProc()
monkeypatch.setattr(wrap_cli.subprocess, "Popen", _fake_popen)
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_cli.time, "sleep", lambda _seconds: None)
monkeypatch.setattr(wrap_cli, "_get_log_path", lambda port=None: tmp_path / "proxy.log")
monkeypatch.setattr(wrap_cli, "_resolve_wrap_proxy_timeout_seconds", lambda: 1)
wrap_cli._start_proxy(8787)
return captured
def test_start_proxy_detaches_on_windows(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
# Force the Windows branch regardless of the host OS, and supply the
# Windows-only ``subprocess`` constants the host lacks on POSIX.
monkeypatch.setattr(wrap_cli.sys, "platform", "win32")
monkeypatch.setattr(wrap_cli.subprocess, "CREATE_NO_WINDOW", 0x08000000, raising=False)
monkeypatch.setattr(wrap_cli.subprocess, "CREATE_NEW_PROCESS_GROUP", 0x200, raising=False)
flags = _capture_popen_kwargs(monkeypatch, tmp_path)["creationflags"]
assert flags & 0x08000000 # CREATE_NO_WINDOW: own hidden console
assert flags & 0x200 # CREATE_NEW_PROCESS_GROUP: ignore the parent's Ctrl-C
assert flags & _CREATE_BREAKAWAY_FROM_JOB # survive Job kill-on-close
def test_start_proxy_keeps_creationflags_zero_off_windows(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
monkeypatch.setattr(wrap_cli.sys, "platform", "linux")
kwargs = _capture_popen_kwargs(monkeypatch, tmp_path)
# POSIX detaches via setsid(); no Windows creation flags are applied.
assert kwargs["creationflags"] == 0
assert isinstance(kwargs["start_new_session"], bool)
def test_start_proxy_retries_without_breakaway_when_job_forbids_it(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
# Force the Windows branch and supply its constants, as above.
monkeypatch.setattr(wrap_cli.sys, "platform", "win32")
monkeypatch.setattr(wrap_cli.subprocess, "CREATE_NO_WINDOW", 0x08000000, raising=False)
monkeypatch.setattr(wrap_cli.subprocess, "CREATE_NEW_PROCESS_GROUP", 0x200, raising=False)
# First spawn raises OSError (the launcher's Job forbids breakaway); the
# second must succeed without CREATE_BREAKAWAY_FROM_JOB.
seen: list[int] = []
def _flaky_popen(cmd: Any, **kwargs: Any) -> _FakeProc:
seen.append(kwargs["creationflags"])
if len(seen) == 1:
raise OSError("a process in the job cannot break away")
return _FakeProc()
monkeypatch.setattr(wrap_cli.subprocess, "Popen", _flaky_popen)
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_cli.time, "sleep", lambda _seconds: None)
monkeypatch.setattr(wrap_cli, "_get_log_path", lambda port=None: tmp_path / "proxy.log")
monkeypatch.setattr(wrap_cli, "_resolve_wrap_proxy_timeout_seconds", lambda: 1)
wrap_cli._start_proxy(8787)
assert len(seen) == 2 # the call was retried exactly once
assert seen[0] & _CREATE_BREAKAWAY_FROM_JOB # first attempt requests breakaway
assert not seen[1] & _CREATE_BREAKAWAY_FROM_JOB # retry drops it
assert seen[1] & 0x08000000 # but still gets its own hidden console
assert seen[1] & 0x200 # and still gets its own process group
class _TrackingFile:
"""Records whether the parent closed its copy of the stdio log handle."""
def __init__(self) -> None:
self.closed = False
def close(self) -> None:
self.closed = True
def test_start_proxy_closes_log_when_both_spawns_fail(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
# Both spawn attempts raise (e.g. the Job forbids breakaway AND the retry
# still fails). The parent must not leak the stdio log file handle.
monkeypatch.setattr(wrap_cli.sys, "platform", "win32")
monkeypatch.setattr(wrap_cli.subprocess, "CREATE_NO_WINDOW", 0x08000000, raising=False)
monkeypatch.setattr(wrap_cli.subprocess, "CREATE_NEW_PROCESS_GROUP", 0x200, raising=False)
log_file = _TrackingFile()
monkeypatch.setattr(wrap_cli, "open", lambda *a, **k: log_file, raising=False)
def _always_fails(cmd: Any, **kwargs: Any) -> _FakeProc:
raise OSError("spawn failed on both attempts")
monkeypatch.setattr(wrap_cli.subprocess, "Popen", _always_fails)
monkeypatch.setattr(wrap_cli.time, "sleep", lambda _seconds: None)
monkeypatch.setattr(wrap_cli, "_get_log_path", lambda port=None: tmp_path / "proxy.log")
monkeypatch.setattr(wrap_cli, "_resolve_wrap_proxy_timeout_seconds", lambda: 1)
with pytest.raises(OSError):
wrap_cli._start_proxy(8787)
assert log_file.closed # finally closed the handle despite the failure