Commit Graph
27 Commits
Author SHA1 Message Date
Vishal TyagiandJerrettDavis 6b903cbc81 fix(install): stabilize Windows task lifecycle and eliminate console window popup (#4627)
## Description

Fixes Windows persistent installation lifecycle and console flash
issues:

1. **`start_supervisor()` failure and fallback immediate startup**: When
S4U / BootTrigger task registration is rejected without administrator
credentials (Issue #3418), Headroom falls back to session-scoped health
recovery and intentionally omits the `<profile>-startup` task.
`start_supervisor()` now tolerates absent tasks during `schtasks`
execution and immediately launches the fallback runtime
(`start_detached_agent`) when the startup task is absent, avoiding the
45-second startup timeout.
2. **Watchdog tracking under Task Scheduler supervisor lifecycle**:
`_render_windows_vbs_launcher()` executes PowerShell synchronously via
`WshShell.Run(..., 0, True)`. This keeps `wscript.exe` running through
the full health check lifecycle, allowing Task Scheduler's `IgnoreNew`
policy to prevent overlapping executions and enabling `schtasks /End`
during `stop_supervisor()` to cleanly terminate in-flight recovery
before process exit.
3. **`stop_runtime()` teardown identity race**: Immediately after
`TerminateProcess` on Windows, PEB and command-line memory unmap,
causing `_process_identity(pid, manifest)` to return `None`. The polling
loop now checks `if _process_identity(...) is False:` to tolerate
transient `None` during process kernel teardown.
4. **Flashing console/terminal window on health check**: Task Scheduler
running under `InteractiveToken` executes `.vbs` actions via
`wscript.exe //B //Nologo`, preventing interactive console/terminal
window popups.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- **`headroom/install/supervisors.py`**:
- In `start_supervisor()`, tolerated absent tasks during `schtasks`
enablement and execution; when `<service_name>-startup` is absent,
immediately launches `start_detached_agent(manifest.profile)` if the
runtime is not yet ready, while continuing to surface genuine permission
errors.
- In `_render_windows_vbs_launcher()`, set `WaitOnReturn=True` in
`WshShell.Run(..., 0, True)` to keep `wscript.exe` alive for the
duration of the PowerShell check so Task Scheduler accurately tracks
task activity and supports `schtasks /End`.
- Updated `_windows_task_xml()` so `.vbs` actions execute through
`wscript.exe //B //Nologo`.
- Updated `install_supervisor()` fallback path to register the silent
VBS launcher under `InteractiveToken`.
- **`headroom/install/runtime.py`**:
- Changed `_process_identity(pid, manifest) is not True` to
`_process_identity(pid, manifest) is False` in the `stop_runtime()`
polling loop.
  - Removed duplicate `if not pid_alive(pid): if not pid_alive(pid):`.
- **`tests/test_install/test_supervisors.py`**:
- Added coverage in `test_windows_start_tolerates_absent_tasks`
asserting immediate detached agent startup when the startup task is
absent.
- Added `test_windows_start_skips_detached_agent_when_already_ready` and
`test_windows_start_surfaces_access_denied`.
- Added `test_windows_task_xml_vbs_uses_wscript` and updated
`test_render_windows_vbs_launcher` to assert synchronous `, 0, True`
execution.
- **`tests/test_install/test_runtime.py`**:
- Added
`test_stop_runtime_tolerates_transient_unknown_identity_during_exit`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --extra proxy pytest tests/test_install/test_supervisors.py tests/test_install/test_runtime.py -q
131 passed, 2 skipped in 17.89s

$ uvx ruff check headroom/install/supervisors.py headroom/install/runtime.py tests/test_install/test_supervisors.py tests/test_install/test_runtime.py
All checks passed!

$ uvx ruff format --check headroom/install/supervisors.py headroom/install/runtime.py tests/test_install/test_supervisors.py tests/test_install/test_runtime.py
4 files already formatted

$ uvx --from mypy==1.20.2 mypy headroom --ignore-missing-imports
Success: no issues found in headroom/install/supervisors.py and headroom/install/runtime.py
```

## Real Behavior Proof

- Environment: Windows 11 x86_64, Python 3.13.17, headroom 0.41.0
- Exact command / steps: Executed 'headroom install apply' on a standard
user account to trigger session-scoped supervisor fallback. Verified
Task Scheduler entry and executed 'schtasks /Run /TN
headroom-default-health'. Verified immediate daemon startup and teardown
via 'headroom install status' and 'headroom install stop'.
- Observed result: Task Scheduler registered headroom-default-health
with wscript.exe //B //Nologo ensure-headroom.vbs. Background health
runs completely silently with zero console window popups. When the
startup task is absent, start_supervisor immediately launches the
detached agent, and stop_supervisor ends the in-flight task process
cleanly.
- Not tested: Non-Windows platforms (Linux systemd and macOS launchd
paths are untouched and guarded behind platform checks).

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: On Windows, session-scoped scheduled
tasks run through wscript.exe rather than cmd.exe to prevent terminal
window popups, and missing startup tasks trigger immediate runtime
launch.
- Kill switch / disable path: Not applicable (deployment script
generation).
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this pull request.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` ? it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 23:08:23 -06:00
7b6cb77905 fix(install): keep the HuggingFace cache on a writable mount in Docker presets (#4207) (#4209)
## Description

With the `persistent-docker` preset, Kompress never loads. The container
runs as the host user (`--user 501:20`) with `HOME=/tmp/headroom-home`,
but the image never creates that directory. Docker creates it as
`root:root 0755` to hold the bind mounts, so the proxy cannot create
`~/.cache` and the HuggingFace download of the Kompress model fails with
`Permission denied (os error 13)`. The log never shows that error: it
reports a missing dependency instead (`Kompress requires onnxruntime or
torch`), pointing the operator at reinstalling extras that are already
present.

Fix: point `HF_HOME` and `XDG_CACHE_HOME` at the bind-mounted,
host-owned `.headroom` workspace so the cache is writable regardless of
who Docker made the container's `$HOME`; and keep the underlying
`OSError` in both `kompress_compressor.py` log sites instead of masking
it.

Closes #4207

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/runtime.py`: `build_runtime_command()` now emits
`HF_HOME` and `XDG_CACHE_HOME` as real `--env` flags pointing under the
bind-mounted `.headroom` workspace (`<container_home>/.headroom/cache`),
and applies them to the persistent-container args path, not only the
dev/foreground path.
- `headroom/install/runtime.py`: added `_append_cache_env()` and wired
it through both command builders.
- `scripts/install.sh` and `docker/docker-compose.native.yml`: set
`HF_HOME`/`XDG_CACHE_HOME` under the writable `.headroom` mount for the
same preset.
- `headroom/transforms/kompress_compressor.py`: `_create_onnx_session()`
appends the underlying cause to its `No loadable ONNX artifact` message;
`_load_kompress()` re-raises the real ONNX failure (chained) instead of
reporting a missing dependency when onnxruntime is present but loading
failed.

## Testing

- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_install/ tests/test_kompress_failsafe.py -q
278 passed, 11 skipped, 1 failed

The single failure is tests/test_install/test_runtime.py::test_build_runtime_command_lets_the_host_dashboard_read_its_data
which fails with ImportError: FastAPI required in this environment. It fails identically on unmodified main here,
so it is a local environment gap, not this change.

New regression tests, all passing:
- test_build_runtime_command_keeps_hf_cache_on_the_writable_mount
- test_build_runtime_command_persistent_path_keeps_hf_cache_on_the_writable_mount
- test_no_loadable_artifact_message_carries_the_underlying_error
- test_onnx_failure_while_onnxruntime_available_reports_the_real_error
- test_missing_dependency_still_reports_install_hint

$ bash -n scripts/install.sh          # clean
$ python -c "import yaml,sys;yaml.safe_load(open('docker/docker-compose.native.yml'))"   # clean
```

## Real Behavior Proof

- Environment: Linux, Python venv at `.venv`, repo at this branch. No
Docker daemon available on this machine.
- Exact command / steps: `pytest tests/test_install/
tests/test_kompress_failsafe.py -q`; asserted the generated `docker run`
argv from both `build_runtime_command()` and
`build_persistent_container_args()` contains `--env HF_HOME` / `--env
XDG_CACHE_HOME` pointing under the `.headroom` mount.
- Observed result: all new tests pass; both argv builders now emit the
cache env pointing at the writable host-owned mount.
- Not tested: the end-to-end container run from the issue's reproduction
steps — no Docker daemon here. The fix is verified at the level of the
generated `docker run` argv and the unit tests above.

## Runtime Rollout Safety

- Rollout-managed feature(s): Kompress proxy model loading under the
`persistent-docker` preset.
- Minimum rollout channel: all channels that use the persistent-docker
preset; the change only affects env vars for cache directories.
- Stable/default behavior changed: no. Default (non-persistent) paths
are unchanged; persistent paths gain two `--env` flags pointing at an
existing writable mount.
- Kill switch / disable path: unset/override `HF_HOME`/`XDG_CACHE_HOME`,
or run without the persistent-docker preset.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this commit; the preset returns to the previous
(broken) cache behavior.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Docs untouched: no user-facing docs describe
`HF_HOME`/`XDG_CACHE_HOME` for this preset, so no documentation change
was needed (hence the unchecked box).
- No lint/type-check run: `ruff`/`mypy` are not installed in this
environment.
- No Docker daemon available, so the container-level e2e reproduction
was not executed — flagged under "Not tested" above.

---------

Co-authored-by: Shivam Maurya <shivam@programmingwithmaurya.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: JerrettDavis <jerrett@headroomlabs.ai>
2026-10-24 08:38:28 -07:00
Tejas ChopraandClaude Opus 5.5 c191beef59 refactor: remove write-only and test-only code in subscription and install (#4123)
## Description

This PR removes private code in `subscription/` and `install/` that
nothing reads.

- `subscription/tracker.py`: `_PERSIST_FILE_ENV`, `_DEFAULT_PERSIST_DIR`
and `_DEFAULT_PERSIST_FILE` are left over from before the path moved to
`paths.subscription_state_path()`. `_full_tokens` was a counter that was
written on every request and never read. It also kept an 8-character
OAuth token prefix in memory.
- `install/runtime.py`: `_process_matches_runtime` was a test-only
wrapper for `_process_identity(...) is True`. Product code calls
`_process_identity` directly.

Evidence: `rg -n '<name>' headroom tests crates plugins sdk examples
benchmarks scripts e2e docs wiki .github pyproject.toml` and the same
search over the consumer repos find only the definitions, plus the tests
changed here. No `getattr` string spelling exists for any of them.

Refs #4136

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `headroom/subscription/tracker.py`: remove the three persist constants
and `_full_tokens`.
- `headroom/install/runtime.py`: remove `_process_matches_runtime`.
- `tests/test_install/test_runtime.py`: the six asserts now check
`_process_identity(...) is True` or `is False` directly.
- `tests/test_subscription_tracker.py`: drop the one assert on
`_full_tokens`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ pytest -q -p no:cacheprovider tests/test_subscription_tracker.py tests/test_install/test_runtime.py tests/test_network_diff_capture.py tests/test_subscription_window_render.py tests/test_subscription_poll_credential_policy.py tests/test_stateless_no_disk.py tests/test_cli_learn.py
# on origin/main (a9815bbb):
176 passed, 1 warning in 9.92s
# on this branch:
176 passed, 1 warning in 9.52s
$ uvx ruff@0.16.8 check <4 touched files>
All checks passed!
$ mypy headroom/subscription/tracker.py headroom/install/runtime.py --ignore-missing-imports
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo `.venv`, this branch's worktree.
Temp `HOME`, `HEADROOM_WORKSPACE_DIR` and `CODEX_HOME`; nothing touched
the real home.
- Exact command / steps:
1. Shared smoke harness: a real proxy from this checkout against a local
fake upstream, with one Anthropic and one OpenAI request carrying a
compressible tool result (`smoke.sh <worktree>`).
2. Lifecycle path that the removed wrapper used to front: saved a
`persistent-task` manifest whose `runner.pid` holds the PID of an
unrelated live process (`sleep 300`), then ran the real CLI `headroom
install status --profile rt`. `runtime_status` goes through
`_process_identity`, which rejects the PID.
3. Tracker path: `SubscriptionTracker().notify_active("Bearer
oauth-token-123", from_local_operator=True)`.
- Observed result:

```text
branch: cleanup/cli-integrations-dead-subscription-install-capture @ 06377018
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines ($SMOKE/ws/logs/proxy-50694.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: $SMOKE)
```

```text
# runner.pid holds the PID of an unrelated live process (sleep 300)
$ headroom install status --profile rt
Profile:    rt
Preset:     persistent-task
Runtime:    python
Supervisor: none
Scope:      user
Port:       18999
Status:     stopped
Healthy:    no
exit=0
sleep still alive (status never signalled it)

$ python -c "...notify_active('Bearer oauth-token-123', from_local_operator=True)..."
active: True | token prefix counter present: False
```

- Not tested: a live persistent install under launchd/systemd; this PR
deletes only private names, and `_process_identity` itself is unchanged.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A; dead-code removal.
- Stable/default behavior changed: No.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

Left for the maintainer: public dataclass field.
`CapturedExchange.request_body_preview` in
`headroom/capture/network_diff.py` is filled for every captured exchange
(by `_preview_body`, capped at `MAX_BODY_PREVIEW_CHARS`) and never read;
`CaptureDiff.to_dict` does not serialize exchanges. `CapturedExchange`
is in `headroom.capture.__all__`, so the field and the code that fills
it stay unchanged here.

`SubscriptionTracker.latest_snapshot` is used only by tests, but it
stays: it is a public property, and removing it was optional.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:01:22 -07:00
derrik fandJD Davis 64a661d5b4 fix(docker): adds mount for the litellm config dir (#2454)
litellm expects access token in its config dir for github copilot

## Description

<!-- Briefly explain the change and why it is needed. -->
I could not authenticate github copilot via litellm when using
persistent install via docker without mounting litellm's config dir

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- adds mount for litellm's config dir to the proxy container

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
ruff check .
All checks passed!

mypy headroom/install/runtime.py
Success: no issues found in 1 source file

pytest tests/test_install/test_runtime.py
====================================================================================================== test session starts =======================================================================================================
platform darwin -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0 -- /Users/derrik.fleming/repos/headroom/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/derrik.fleming/repos/headroom
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.9.3, asyncio-1.3.0, respx-0.23.1, cov-7.0.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 25 items

tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_includes_deployment_env PASSED                                                                                                                   [  4%]
tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_includes_gpu_passthrough PASSED                                                                                                                  [  8%]
tests/test_install/test_runtime.py::test_build_runtime_command_docker_manifest_env_beats_host_passthrough PASSED                                                                                                           [ 12%]
tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_matches_wrapper_parity PASSED                                                                                                                    [ 16%]
tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_does_not_duplicate_entrypoint PASSED                                                                                                             [ 20%]
tests/test_install/test_runtime.py::test_resolve_headroom_command_prefers_headroom_binary PASSED                                                                                                                           [ 24%]
tests/test_install/test_runtime.py::test_resolve_headroom_command_falls_back_to_python_module PASSED                                                                                                                       [ 28%]
tests/test_install/test_runtime.py::test_runtime_env_and_mount_source PASSED                                                                                                                                               [ 32%]
tests/test_install/test_runtime.py::test_build_runtime_command_python_and_docker_user PASSED                                                                                                                               [ 36%]
tests/test_install/test_runtime.py::test_read_pid_handles_invalid_content PASSED                                                                                                                                           [ 40%]
tests/test_install/test_runtime.py::test_write_read_and_clear_pid PASSED                                                                                                                                                   [ 44%]
tests/test_install/test_runtime.py::test_runtime_start_lock_is_nonblocking PASSED                                                                                                                                          [ 48%]
tests/test_install/test_runtime.py::test_runtime_start_lock_blocks_another_process PASSED                                                                                                                                  [ 52%]
tests/test_install/test_runtime.py::test_run_foreground_and_detached_helpers PASSED                                                                                                                                        [ 56%]
tests/test_install/test_runtime.py::test_start_detached_agent_closes_parent_log_fd PASSED                                                                                                                                  [ 60%]
tests/test_install/test_runtime.py::test_start_detached_agent_closes_log_fd_when_popen_raises PASSED                                                                                                                       [ 64%]
tests/test_install/test_runtime.py::test_start_stop_wait_and_runtime_status_branches PASSED                                                                                                                                [ 68%]
tests/test_install/test_runtime.py::test_stop_runtime_for_docker_stops_and_removes_container PASSED                                                                                                                        [ 72%]
tests/test_install/test_runtime.py::test_runtime_status_reads_container_and_pid_state PASSED                                                                                                                               [ 76%]
tests/test_install/test_runtime.py::test_runtime_status_reports_live_pid_without_terminating PASSED                                                                                                                        [ 80%]
tests/test_install/test_runtime.py::test_runtime_status_survives_winerror87_systemerror PASSED                                                                                                                             [ 84%]
tests/test_install/test_runtime.py::TestRestartCurrentDeployment::test_foreground_when_no_deployment_env PASSED                                                                                                            [ 88%]
tests/test_install/test_runtime.py::TestRestartCurrentDeployment::test_docker_returns_host_command_without_restarting PASSED                                                                                               [ 92%]
tests/test_install/test_runtime.py::TestRestartCurrentDeployment::test_task_mode_detected_and_not_restarted PASSED                                                                                                         [ 96%]
tests/test_install/test_runtime.py::TestRestartCurrentDeployment::test_service_spawns_detached_restart PASSED                                                                                                              [100%]

======================================================================================================= 25 passed in 0.35s =======================================================================================================
```

## Real Behavior Proof

- Environment: Docker / MacOS
- Exact command / steps:  My steps included:
- `headroom install apply --image headroom:local --preset
persistent-docker --scope provider --providers manual --backend
github_copilot`
    - send prompt through supported proxy to supported model
- Observed result: litellm cannot authenticate to provider
- Not tested: I didn't test all models from provider, the problem was
apparent. I did not test other providers through litellm.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

Maintainer note: existing runtime command tests cover the Docker
command-building path touched here; this PR does not add new test files
or test cases.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2454"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-08 11:04:25 -05:00
Rod BoevandJD Davis 44db755ee6 fix(install): replace the launchd wrapper with the proxy listener (#3224)
## Description

The persistent service path can leave a wrapper, listener, native
supervisor, or Docker container with conflicting ownership. Rebuild
runtime ownership so launchd directly owns Python, Docker owns Docker
across service/task presets, status and stop require deployment
identity, readiness cannot be health-only, and failed cleanup preserves
rollback state. Closes #3208

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (feature that would cause existing behavior to
change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Make the launchd-owned Python service exec the listener in place.
- Classify Docker ownership from both runtime and preset, with no
competing native supervisor.
- Route Docker service/task start, restart, recovery, and teardown
through the Docker owner.
- Verify exact process command, port, deployment profile, and runtime
before status or stop.
- Add POSIX, macOS, and Windows process identity fallbacks without
requiring psutil.
- Require identity on readiness and fail closed when cleanup cannot
prove the new owner stopped.
- Persist an inactive previous-deployment snapshot, keep active state
durable before owner start, and make environment/provider mutation
activation transactional.
- Surface cleanup, persistence, supervisor, Docker, and
recovery-snapshot deletion failures with actionable recovery guidance.

## Testing

- [x] Unit tests pass (`uv run --locked pytest
tests/test_provider_package_init.py::test_install_registry_apply_provider_scope_mutations_skips_missing_and_none
tests/test_install/test_providers.py tests/test_cli/test_install_cli.py
tests/test_install/test_runtime.py
tests/test_install/test_supervisors.py tests/test_install/test_paths.py
tests/test_install/test_state.py -q`)
- [x] Linting passes (`uv run --locked ruff check` on changed install,
provider, supervisor, and focused test files)
- [x] Type checking passes (`uv run --locked mypy headroom
--ignore-missing-imports`)
- [x] New tests added for the regression
- [ ] Manual testing performed through live process/supervisor
environments

### Test Output

```text
uv run --locked pytest tests/test_provider_package_init.py::test_install_registry_apply_provider_scope_mutations_skips_missing_and_none tests/test_install/test_providers.py tests/test_cli/test_install_cli.py tests/test_install/test_runtime.py tests/test_install/test_supervisors.py tests/test_install/test_paths.py tests/test_install/test_state.py -q
============================= 197 passed in 1.07s =============================
uv run --locked ruff check .
All checks passed!
uv run --locked ruff format --check .
1534 files already formatted
uv run --locked mypy headroom --ignore-missing-imports
Success: no issues found in 527 source files
git diff --check
passed; the only worktree change is the pre-existing unstaged uv.lock line-ending change
```

## Real Behavior Proof

- Environment: Windows focused process and supervisor tests, with
platform-specific launchd and Docker owner paths represented by fakes.
- Exact command / steps: run the focused lifecycle and state suite
listed above, targeted Ruff and format checks, and the full-head mypy
check.
- Observed result: each persistent mode has one owner; mismatched,
malformed, or unavailable identity cannot authorize cleanup; active
state is durable before owner start; failed cleanup retains recovery
state; mutation rollback preserves unreverted records.
- Not tested: owner-level POSIX exec, a live macOS launchd job, and a
Docker daemon

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This changes the default persistent
install and recovery ownership paths.
- Minimum rollout channel: Stable, after platform-specific CI and
maintainer review.
- Stable/default behavior changed: Yes. Persistent Python service
ownership and Docker supervisor routing change.
- Kill switch / disable path: `headroom uninstall` removes the managed
deployment; users can reinstall with another supported preset or
runtime.
- Unsafe override required: No.
- Qualification impact: launchd, Docker, and Windows identity paths
require their platform-owned environments; local proof uses focused
tests and process/supervisor fakes.
- Rollback path: Revert the rollback-ordering commits in this PR, then
reinstall the prior deployment if needed. Uncertain cleanup retains the
active new manifest and inactive previous snapshot until the new owner
is removed and recovery is completed.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

The changelog is generated by the release pipeline. Live launchd and
Docker-owner behavior remains platform-owned proof; local tests do not
claim those owner environments. Final implementation head: `4004e4a5`.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3224"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:34:42 -05:00
ee731d7ea8 fix(proxy): refuse token-less non-loopback binds, gate operator routes (#3852)
## Why

Four related gaps in the proxy's network gate (findings 01-F1, 01-F2,
01-F5, 01-F9):

- **Open bind.** A non-loopback bind with no `HEADROOM_PROXY_TOKEN` only
logged a warning, then served the `/v1/*` relay, with the operator's
upstream credentials, to every peer that could reach the port.
- **Missing peer address.** `is_loopback_host(None)` returned `True`. A
Unix-domain socket, an ASGI adapter that leaves `scope["client"]` unset,
or a hand-built request therefore turned off the token gate, the
`/admin` and `/debug` guards and the WebSocket gate.
- **Operator data routes.** `/stats-history` (including CSV export),
`/quota`, `/subscription-window` and `/metrics` were served to any
network caller.
- **WebSocket loopback exemption** checked only the peer IP, not the
`Host` header, so it had no DNS-rebinding defence.

## What changes

- New `headroom/proxy/bind_policy.py`. A non-loopback bind (`0.0.0.0`,
`::`, a LAN address or a hostname) with no token is refused at three
places: `create_app` (embedders and the multi-worker factory),
`run_server` (one exit with code 2 before uvicorn forks) and the
`headroom proxy` CLI (a clear error before the banner, exit code 1). An
absent host means uvicorn's default `127.0.0.1`.
- `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` acknowledges an open bind when
the runtime already restricts reachability, such as a container
published on `127.0.0.1`. The `proxy_open_bind` warning is still logged
on that path, and the banner says the bind is open.
- `is_loopback_host(None)` now returns `False` (fail closed), matching
`identity.py`.
- `/stats-history`, `/quota` and `/subscription-window` answer only: a
non-loopback caller that passed the token gate, a loopback caller (peer
and `Host` header both loopback), a trusted dashboard client, or, under
an acknowledged token-less bind, the container's own default-gateway TCP
peer with a loopback `Host` (the host browser reaching a
`127.0.0.1`-published container). Everyone else gets 404, as with
`/admin`. Settings writes keep their stricter rule with no token
short-cut.
- `/metrics` answers a non-loopback caller that passed the token gate, a
loopback caller, a connecting peer in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` (whatever it forwards), or a
resolved client in the dashboard CIDRs. For CIDR peers a hostname `Host`
is fine, so Prometheus scrapers work; a cross-origin browser request is
still refused.
- The token short-cut applies only to non-loopback requests. The gate
exempts loopback peers without checking `Host`, so a loopback request
has not proven it holds the token and must pass the loopback `Host`
check. This blocks a DNS-rebound browser on the operator's machine.
- The `/dashboard` shell stays ungated. It is a static template, like
the already public `/dashboard/static` assets. Gating it would break the
dashboard for containers published on host loopback, where the peer is
the bridge gateway and `Host` is `localhost`. The data routes behind it
are gated.
- The WebSocket loopback exemption requires a loopback peer and a
loopback `Host` header. A loopback peer with a foreign `Host` can still
connect with the token.
- Launchers that always publish on `127.0.0.1` set the acknowledgement
themselves: `headroom install` (docker preset), `scripts/install.sh` and
`scripts/install.ps1` (through one helper that adds the acknowledgement
only together with the `127.0.0.1` publication), and
`docker/docker-compose.native.yml`.
- Docs: `proxy.mdx` documents the new variable and which routes the
token also gates; `docker-install` (docs and wiki) and the
`docker-compose.yml` comment show the acknowledgement in the
loopback-published example. `e2e/docker-bind-security.sh` sets it and
checks that the same launch without it refuses to start.

## Behaviour change (breaking)

- `headroom proxy --host 0.0.0.0`, or `HEADROOM_HOST` set to any
non-loopback address, without `HEADROOM_PROXY_TOKEN` now refuses to
start. The error names both remedies. Set a token, bind loopback, or set
`HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` when the port is already
confined. Anyone running the image by hand with `--host 0.0.0.0` and no
token must add one of these.
- Without a token, `/stats-history`, `/quota`, `/subscription-window`
and `/metrics` return 404 to non-loopback callers outside a trusted
CIDR. Prometheus scrapers on another host must be in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` or send the token.
- Code that relied on a request with no peer address being treated as
loopback now gets the non-loopback path.
- The default loopback run is unchanged.

## Tests

- `tests/test_proxy_bind_policy.py`: the refuse/allow matrix
(non-loopback, loopback, unset host, token, acknowledgement values), the
warning on the acknowledged path, `create_app`, the `run_server` exit
before uvicorn, and the CLI (refusal, `HEADROOM_HOST` from env,
acknowledged start, offline banner, start with a token).
- `tests/test_proxy_loopback_gating.py`: each data route against
non-loopback, loopback, trusted-CIDR and token callers; the dashboard
shell stays reachable; the host of an acknowledged, loopback-published
container reads the data routes while another bridge peer, a forwarded
gateway address and a foreign `Host` do not; CSV export is not served to
network callers; `/metrics` across loopback, both CIDR lists,
out-of-range peers, cross-origin browsers and token callers, including a
trusted gateway relaying a scraper outside its own range. A regression
test shows that a token on the proxy does not exempt a loopback peer
from the `Host` check; it fails without the fix.
- `tests/test_proxy_hardening.py`: the HTTP gate with no peer address,
WebSocket `Host` cases, `require_loopback` and `/debug` with no peer.
- Installer tests: `headroom install` adds the acknowledgement, and the
shell and PowerShell wrappers add it only together with the loopback
publication. Launcher-level tests start the proxy with the environment
and `--host` that `build_runtime_command()` and
`docker-compose.native.yml` produce and check that the host browser
(bridge-gateway peer) gets `/stats-history` and another container does
not.
- Tests that bound `0.0.0.0` for unrelated reasons now set a token or
the acknowledgement. Tests that read operator routes use an explicit
loopback peer.

On `main`, 14 of the gating and hardening tests fail and the bind-policy
module does not exist. Manual runs: token-less `--host 0.0.0.0` exits
with the refusal from both the CLI and `run_server`; with the
acknowledgement, the data routes are 404 to a LAN peer and to a loopback
peer with a foreign `Host`; with a token, a LAN peer gets 401 without it
and 200 with it; the default loopback bind starts as before.

Not tested: a real Unix-domain-socket deployment and a real Prometheus
scrape (both covered by tests only).

## Merge order

Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 →
#3863. This PR goes first: it is the gate the others assume, and it is
the release-note event.

## Not in this PR

- Enforcing the token for loopback callers when one is configured.

Revised 2026-10-02 after review: `/metrics` matches gateway CIDRs
against the TCP peer, and the operator data routes serve the host of a
loopback-published container (bridge-gateway peer under the launchers'
bind acknowledgement) so the default Docker dashboard works.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3852"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:27:35 -05:00
Rod BoevandJerrettDavis b0c19a25f0 fix(security): reject unauthenticated public proxy binds (#3316)
## Description

Headroom's published image and maintained Docker launchers could expose
the
proxy on every host interface without a proxy token. This change moves
the
security boundary to the deployment artifacts: published images default
to a
container-loopback listener, supported local launchers publish only on
host
loopback, and deliberate public deployments retain explicit token
protection.
The application factory remains transport-neutral.

Closes #3283

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set both published image stages to a container-loopback default while
keeping explicit host overrides available.
- Changed native Compose and the maintained Bash and PowerShell
launchers to use an internal `0.0.0.0` listener with host-loopback
publication.
- Preserved the top-level Compose token requirement and the existing
HTTP and WebSocket credential behavior for deliberate public binds.
- Scoped differential-capture token injection to the internal
`headroom-proxy` destination and kept sensitive headers redacted in
captured records.
- Added focused launch-contract, native-core, capture, and Docker
publication checks, plus a base/head harness that tolerates a base
already containing the fix.
- Updated Docker, installer, capture, and mirrored documentation,
including an unauthenticated local loopback quickstart and an
authenticated deliberate-public example.

## Testing

- [x] Focused unit and integration tests pass
- [x] Linting and formatting checks pass
- [ ] Type checking passes
- [x] New tests added for the changed deployment and capture behavior
- [ ] Manual testing performed

### Test Output

```text
uv run python -m pytest tests/test_proxy_hardening.py tests/test_install/test_native_installers.py tests/test_docker_compose_persistence.py tests/test_network_diff_capture.py tests/test_rust_core_smoke.py -q
76 passed, 1 skipped, 1 warning in 23.45s

uv run ruff check tests/test_install/test_native_installers.py tests/test_install/test_runtime.py
All checks passed!

bash -n e2e/docker-bind-security.sh
passed

bash e2e/docker-bind-security.sh --base-image headroom-t3283-base --head-image headroom-t3283-head --fixture tests/fixtures/docker-bind-exposure.json --base-ref origin/main
base_ref=origin/main ... runtime_slim_config=loopback rust_core=loaded local_completion=200 attacker_loopback=000 ws_status=403
docker bind security: fixture forwarding, loopback isolation, built-image health, slim-image loopback config, and token controls passed
```

## Real Behavior Proof

- Environment: Windows, Docker Desktop 29.6.2, current `origin/main`
base, head image, and a real Docker bridge peer.
- Exact command / steps: run the focused security/runtime tests, Compose
configuration checks, then `bash e2e/docker-bind-security.sh` against
the current base and head image definitions.
- Observed result: `76 passed, 1 skipped`; the built-image harness
passed base/head health, local completion, non-loopback isolation,
authenticated public HTTP/WebSocket controls, loopback runtime-slim
configuration, and direct Rust-core loading.
- Not tested: runtime-slim proxy service health on this host, because
its unrelated ONNXRuntime import path exits with code 139; the full
hosted matrix and a future published GHCR image remain CI-owned.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Unsafe override required: no
- Minimum rollout channel: next release containing the Docker and
launcher changes
- Stable/default behavior changed: published images and maintained local
Docker launchers now use loopback-safe publication defaults
- Kill switch / disable path: revert the deployment-artifact changes
before release; deliberate public deployments remain explicitly
operator-controlled
- Qualification impact: Docker users relying on all-interface default
publication must opt into public publication and configure the existing
proxy token
- Deliberate public deployment: use an explicit public host publication,
set `HEADROOM_PROXY_TOKEN`, and send the matching bearer or
`X-Headroom-Proxy-Token` credential
- Local deployment: use the maintained loopback publication and no
mandatory token
- Rollback path: revert the deployment-artifact and documentation
changes together

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove the behavior is effective
- [x] Focused tests pass locally with these changes
- [x] I did **not** edit `CHANGELOG.md`; release-please generates
release notes from the conventional PR title

## Additional Notes

The issue fixture remains provenance for the unauthenticated
publication. Native
`headroom proxy` defaults and the application-level authentication
owners are
unchanged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-24 23:58:46 -05:00
Radhakrishnan Pachyappan 11c7320bbc fix(install): wait for the old process to stop before restarting (#3670)
## Description

`headroom install restart` reports success but leaves the deployment
stopped. Stopping the supervisor returns before the old proxy finishes
its graceful shutdown, so `/readyz` still answers. `_start_deployment`
treats a ready endpoint as "already running" and returns without
starting anything. The old process then exits and nothing replaces it.
`install start` works afterwards because by then nothing is answering.

Restart now waits (bounded, 15s) for the health endpoint to go down
before starting, and fails instead of reporting success if it never
does.

Closes #3658

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `_wait_until_stopped` in `headroom/cli/install.py` and call it
between `_stop_deployment` and `_start_deployment` in `install restart`.
- Add two regression tests: old process lingers after stop (restart must
wait, then start), and old process never stops (restart must fail
without starting or reporting success).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
python -m pytest tests/test_cli/test_install_cli.py -q
42 passed

python -m ruff check headroom/cli/install.py tests/test_cli/test_install_cli.py -> All checks passed
python -m ruff format --check (same files) -> already formatted

Without the source change, test_install_restart_waits_for_old_process_before_starting fails (1 failed).
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.14.3. The issue was reported on
macOS (launchd), which I could not run.
- Exact command / steps: the unit tests above simulate the old process
still answering `/readyz` after stop.
- Observed result: restart waits for the endpoint to go down, then
starts; it errors if the endpoint never goes down.
- Not tested: a real launchd / systemd restart; mypy; the wider test
suite beyond `test_install_cli.py`; the docker and task presets. The
root cause is inferred from the code path and the issue's symptoms, not
reproduced on macOS.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: `install restart` now waits up to 15s
for the old process to stop, and errors if it does not.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Docs: not applicable (no user-facing docs describe restart timing). Full
unit suite not run locally; only `test_install_cli.py`.
2026-09-24 11:29:03 -05:00
William EchoandWilliam Echo 21accecc98 fix(install): preserve host file ownership with rootless Podman (#3592)
## Description

Rootless Podman versions that honor an image's `USER root` despite
`--userns=keep-id` create bind-mounted files as a subordinate host UID.
A newly created mode-600 state file then becomes unreadable to the host
CLI. Reproduced with Podman 5.4.2 and host UID 1007.

Keep `--userns=keep-id` and also select the invoking user's UID/GID with
`--user`. This is not the `--user`-without-keep-id mapping that caused
#2804. The override is limited to Linux. Docker retains its existing
user arguments, macOS Podman keeps its existing keep-id-only behavior,
Windows remains unchanged, and the configured image is preserved.

Fixes #3569 (the file-ownership defect). The wrapper-script
autodetection side note is not changed; its existing environment
override remains necessary.

## Type of Change

- [x] Bug fix
- [x] Documentation update

## Changes Made

- Apply the existing host UID/GID selection to Linux Podman as well as
Docker, after selecting Podman's keep-id namespace.
- Replace the old keep-id-only assertion with a 12-case
platform/UID/GID/image matrix, including macOS and Windows regression
coverage.
- Document the Podman environment override and the limits of recovery
for files already owned by another UID.

## Testing

- [x] New tests fail before the fix and pass after it (4 cases).
- [x] Final focused run (installation subsystem plus the PATH-sensitive
MCP case): 162 passed, 6 skipped.
- [x] `ruff check .` and `ruff format --check .`: passed.
- [x] Repository pre-commit checks, including mypy: passed.
- [x] `make ci-precheck-rust` with Rust 1.95.0: passed.
- [x] Manual real-behavior testing performed.

Full Python run: 12,148 passed, 595 skipped, one failure in
`test_ordinary_install_does_not_adopt_serena` because the invocation did
not put the venv CLI on PATH. The original checkout fails identically;
adding `.venv/bin` to PATH makes it pass. The final focused run above
includes this recheck and the platform guard added after the full run.
This is a full run plus scoped revalidation, not a claim of one entirely
green full-suite invocation.

`git push` completed with the installed pre-push hook: the full `make
ci-precheck` passed, including a Rust 1.95.0 extension rebuild, the
required Python regression set, and commitlint. No hooks were bypassed.

### Test Output

```text
# Full Python run (PATH-sensitive failure explained above):
1 failed, 12148 passed, 595 skipped, 6143 warnings in 931.68s
# Final installation subsystem and corrected-PATH MCP recheck:
162 passed, 6 skipped in 0.52s
# Installed pre-push hook's Python regression set:
175 passed, 4 skipped, 1 warning in 4.09s
✅ ci-precheck PASSED — safe to push.
```

## Real Behavior Proof

- Environment: Debian 13.4, Python 3.13.5, rootless Podman 5.4.2,
ordinary user UID/GID 1007. This ran inside a disposable privileged
Docker Desktop Linux container to permit nested user namespaces, with no
host data directories mounted. Podman used VFS, disabled cgroups and no
network. `podman info` reported rootless=true. No provider/model was
used.

- Exact command / steps: Import the Debian rootfs with `USER root`, run
the standalone command below, repeat with `--user 1007:1007`, then
execute the real `build_runtime_command` output twice with root and
nonroot fixture images.

The initial standalone reproduction was:

```sh
podman --storage-driver=vfs run --rm --network=none --cgroups=disabled \
  --userns=keep-id -v /home/tester/proof:/proof localhost/root-fixture \
  sh -c 'id; umask 077; echo written > /proof/state; cat /proc/self/uid_map'
stat -c '%u:%g %a %n' /home/tester/proof/state
cat /home/tester/proof/state
```

- Observed result: Before the fix, process UID 0, file owner
`100000:100000`, mode 600, host read failed with Permission denied.
Repeating with `--user 1007:1007` produced a host-readable file owned by
`1007:1007`.

The end-to-end follow-up called the real
`build_runtime_command(DeploymentManifest(...))` and executed its
returned command through a docker→podman symlink. The fixture image's
entrypoint created a private file/directory and appended to a
pre-existing private file. The original source failed the ownership
assertion. After the patch, two consecutive runs produced:

```text
uid=1007(tester) gid=1007(tester) groups=1007(tester)
0 probe-state 1007 1007 0o600
0 existing 1007 1007 0o600
0 probe-cache 1007 1007 0o700
uid=1007(tester) gid=1007(tester) groups=1007(tester)
1 probe-state 1007 1007 0o600
1 existing 1007 1007 0o600
1 probe-cache 1007 1007 0o700
PASS new files, private directories, append and restart
```

The same end-to-end probe passed with a custom image declaring `USER
1000:1000`, while the host remained UID 1007. Test files left with
incorrect ownership by the red run were removed before the green run;
this patch does not repair existing ownership.

- Not tested: the production Headroom image/proxy startup, Arch/SELinux,
rootful Podman, newer Podman releases, or Windows runtime behavior.
Python unit/lint checks ran on macOS with Python 3.12.14. Current Podman
documentation describes different default USER behavior; the explicit
process user is needed for the reproduced 5.4.2 behavior.

## Runtime Rollout Safety

- Rollout-managed feature(s): None; this is an installer
command-generation bug fix.
- Minimum rollout channel: No feature rollout channel applies; use a
build containing this patch.
- Stable/default behavior changed: Linux Podman receives an explicit
invoking UID/GID alongside keep-id. Image names and configuration
schemas are unchanged.
- Kill switch / disable path: Stop the persistent deployment with
`headroom install stop --profile <profile>`; there is no new feature
flag.
- Unsafe override required: None for this fix. The existing
HEADROOM_CONTAINER_RUNTIME=podman override is still needed when a
wrapper script cannot be detected.
- Qualification impact: Real rootless Podman 5.4.2 file-ownership proof
and platform regressions passed; other runtime environments listed above
remain untested.
- Rollback path: Revert this patch and recreate/restart the deployment.
No data migration is involved; the original ownership problem can recur.
Files already owned by a subordinate UID require separate recovery.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] Style guidelines followed; hard-to-understand mapping behavior
documented
- [x] Corresponding documentation updated
- [x] Regression tests added
- [x] No `CHANGELOG.md` edits

---------

Co-authored-by: William Echo <175406538+qinghuanandejiangshi@users.noreply.github.com>
2026-09-17 15:47:00 -07:00
Abhay Singh 3488f8d4b5 fix(install): use --userns=keep-id under Podman so bind-mount writes don't fail (#2846)
## Description

`build_runtime_command` unconditionally adds `--user <uid>:<gid>` on
non-Windows hosts:

```python
# headroom/install/runtime.py
if not _is_windows():
    getuid = getattr(os, "getuid", None)
    getgid = getattr(os, "getgid", None)
    if callable(getuid) and callable(getgid):
        command.extend(["--user", f"{getuid()}:{getgid()}"])
```

That is correct for Docker, where container UIDs equal host UIDs, but
wrong for rootless Podman, where the host user is already mapped to
container UID 0 and the `/etc/subuid` range is mapped to container UIDs
1 and above. Passing `--user $(id -u):$(id -g)` therefore selects a
container UID backed by a subordinate host UID that owns nothing. The
bind-mounted `~/.headroom` appears inside the container as `root:root`
and is unwritable, so every write fails:

```text
PermissionError: [Errno 13] Permission denied: '/tmp/headroom-home/.headroom/memories'
event=proxy_inbound_request_aborted path=/v1/messages reason=PermissionError
```

The proxy still starts and reports healthy, so the failure only surfaces
once a request touches a write path. As the reporter confirmed,
`--userns=keep-id` (or omitting `--user`) fixes it.

The fix detects Podman and uses `--userns=keep-id` instead of `--user`,
which maps the host user to the same UID inside the container and keeps
the bind mounts writable. Docker still gets `--user`, unchanged.
Detection is subprocess-free: it resolves the `docker` binary and checks
its real name for the common `docker -> podman` symlink shim (e.g. NixOS
`/run/current-system/sw/bin/docker -> podman`), with an explicit
`HEADROOM_CONTAINER_RUNTIME` (`podman` / `docker`) override for setups
the symlink heuristic cannot see, such as a wrapper script.

Fixes #2804

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/runtime.py`: added `_container_runtime_is_podman()`
(env override, then a `docker`-binary realpath basename check, no
subprocess). In `build_runtime_command`, when Podman is detected the
command uses `--userns=keep-id` instead of `--user <uid>:<gid>`.
- `tests/test_install/test_runtime.py`: pinned the existing docker test
to the Docker path via `HEADROOM_CONTAINER_RUNTIME=docker` and asserted
`--userns=keep-id` is absent there; added
`test_build_runtime_command_podman_uses_keep_id_not_user` asserting the
Podman path drops `--user` and adds `--userns=keep-id`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# Fail-before (source fix stashed, new test kept):
tests/test_install/test_runtime.py::test_build_runtime_command_podman_uses_keep_id_not_user FAILED
  assert "--userns=keep-id" in command
  AssertionError: assert '--userns=keep-id' in ['docker', 'run', '--rm', ...]

# Pass-after (fix applied):
tests/test_install/test_runtime.py  26 passed

# Broader install suite (excluding the pre-existing env-specific PowerShell installer test):
tests/test_install/  142 passed, 1 skipped

# uvx ruff@0.15.17 check  -> All checks passed!
# uvx mypy@1.20.2 headroom/install/runtime.py -> Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.17 and mypy 1.20.2 via uvx.
- Exact command / steps: confirmed `build_runtime_command` adds `--user`
unconditionally on non-Windows, then drove both runtimes
deterministically via the `HEADROOM_CONTAINER_RUNTIME` override.
Fail-before with `git stash push headroom/install/runtime.py` and
`python -m pytest tests/test_install/test_runtime.py -k
podman_uses_keep_id` (the command still carries `--user`, no keep-id),
pass-after with `git stash pop` and rerunning the file (26 passed).
- Observed result: with Podman detected the docker command now contains
`--userns=keep-id` and no `--user`/`1000:1001`, matching the
`--userns=keep-id` invocation the reporter verified writes successfully;
with Docker it is unchanged (`--user 1000:1001`, no keep-id).
- Not tested: a live rootless-Podman deployment writing to a bind mount
(no Podman in this environment). The command construction is verified
directly, and `--userns=keep-id` is the documented, reporter-confirmed
switch for the rootless-Podman ID-mapping.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Detection is intentionally subprocess-free and conservative: it only
diverges from today's behavior when the `docker` binary literally
resolves to a `podman`-named target, or when
`HEADROOM_CONTAINER_RUNTIME` is set. Real Docker installs are untouched.
The override also gives a clean escape hatch in both directions if a
given host's symlink layout hides the runtime. This is the `--user` half
of the persistent-docker + Podman issues; the separate host-memory-path
problem (#2803) is addressed in its own PR.
2026-08-08 01:30:48 -05:00
ParideboyandClaude Sonnet 5 045f3dfe6f fix(install): use CREATE_NO_WINDOW instead of DETACHED_PROCESS on Windows (#2527)
## Description
On Windows, the detached agent process spawned by `install hook ensure`
(and the `install restart` self-spawn) pops up a visible black console
window repeatedly, because `DETACHED_PROCESS` makes `CREATE_NO_WINDOW` a
no-op per the Win32 process-creation-flags docs. #2521

## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)

## Changes Made
- `headroom/install/runtime.py`: `start_detached_agent()` now uses
`CREATE_NO_WINDOW` instead of `DETACHED_PROCESS`, combined with
`CREATE_NEW_PROCESS_GROUP` (unchanged detach/isolation semantics, window
hidden).
- `headroom/install/runtime.py`: `_spawn_detached_restart()` now also
sets `CREATE_NO_WINDOW` on Windows (previously had no `creationflags` at
all on that platform).
- `tests/test_install/test_runtime.py`: updated the Windows branch of
`test_start_detached_agent_and_run_foreground` to assert the actual
`creationflags` value passed to `Popen`, instead of just monkeypatching
an unused `DETACHED_PROCESS` attribute.

## Testing
- [x] Added/updated tests
- [x] Ran full local test suite

```
$ python -m pytest tests/test_install -q
======================= 137 passed, 1 skipped in 48.68s =======================

$ ruff check headroom/install/runtime.py tests/test_install/test_runtime.py
All checks passed!

$ ruff format --check headroom/install/runtime.py tests/test_install/test_runtime.py
2 files already formatted

$ mypy headroom --ignore-missing-imports
Success: no issues found in 506 source files
```

## Real Behavior Proof
- Environment: Windows 11, Python 3.13.11, headroom repo local checkout
- Exact command / steps: `python -m pytest
tests/test_install/test_runtime.py -q`, plus manual read of `subprocess`
Windows creation-flag semantics (`DETACHED_PROCESS` + child console
allocation vs `CREATE_NO_WINDOW`)
- Observed result: all 25 tests in `test_runtime.py` pass, including the
updated assertion that `creationflags == CREATE_NO_WINDOW |
CREATE_NEW_PROCESS_GROUP` on the Windows code path
- Not tested: did not reproduce the original visible-console-popup repro
end-to-end via live Claude Code hook invocation (no environment with the
full hook-triggered respawn loop set up in this session); relying on the
Win32 docs and the reporter's own local verification of the same flag
swap

## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-25 19:24:53 -07:00
JD Davis 560ffae103 feat(deploy): Add turnkey deploy command (#1404)
## Description

Adds `headroom deploy` as the turnkey, zero-config local deployment
entrypoint. The command chooses the most capable deployment path it can
verify on the current host, configures detected tools through the
existing persistent-install machinery, starts the proxy, and preserves
the existing rollback behavior if an update fails.

The selection order favors performance first: NVIDIA Docker GPU
passthrough when `nvidia-smi` and Docker's NVIDIA runtime are available,
then plain Docker, then native scheduled recovery, then a detached
Python runtime fallback.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- Added the top-level `headroom deploy` command and reused the existing
install manifest/apply/start/rollback path.
- Added conservative runtime selection for GPU Docker, plain Docker,
native schedulers, and detached Python fallback.
- Added Docker runtime support for manifest-driven `--gpus all`
passthrough.
- Added tests for Docker selection, GPU Docker selection, detached
fallback, GPU command rendering, and subprocess wrapper compliance.
- Updated README and persistent-install docs to present the turnkey
deployment flow and performance-first GPU behavior.
- Allowed documented `opencode` targets through `headroom install apply
--target`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Formatting passes (`ruff format --check`)
- [x] Type checking passes in local pre-commit and CI
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run --with pytest --with pytest-asyncio python -m pytest tests/test_cli/test_subprocess_utf8_encoding.py tests/test_cli/test_install_cli.py tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_includes_gpu_passthrough tests/test_install/test_planner.py -q
47 passed in 1.57s

uvx --from ruff==0.15.17 ruff check headroom/cli/install.py headroom/install/runtime.py tests/test_cli/test_install_cli.py tests/test_install/test_runtime.py --output-format concise
All checks passed!

uvx --from ruff==0.15.17 ruff format --check headroom/cli/install.py headroom/install/runtime.py tests/test_cli/test_install_cli.py tests/test_install/test_runtime.py
4 files already formatted
```

GitHub checks are green on the current head.

## Real Behavior Proof

- Environment: Windows local worktree `C:\git\headroom`, Python 3.13 via
`uv`; GitHub Actions Ubuntu/macOS/Windows runners for full PR CI.
- Exact command / steps: Ran the focused deploy/install tests above,
checked the touched Python files with the CI-pinned Ruff version, and
confirmed the current PR head is mergeable with green GitHub checks.
- Observed result: The deploy command, runtime selection, Docker GPU
command rendering, install CLI behavior, and subprocess encoding
coverage all pass locally; the branch is no longer conflicted.
- Not tested: Actual RTX 4090 hardware passthrough on a physical NVIDIA
workstation; the PR tests conservative detection and Docker command
rendering without requiring GPU hardware in CI.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A - CLI/runtime behavior only.

## Additional Notes

CHANGELOG update is not included because this is an unreleased feature
PR and the repository's release tooling owns release notes from
conventional commits.
2026-07-15 18:37:20 +00:00
nangsontayandJerrettDavis e5b3a634df fix(proxy): dedupe Codex WS request logging for accurate mixed-provider dashboards (#2189)
## Description

Running Claude Code (Anthropic) and Codex (OpenAI) against the **same**
Headroom proxy instance on one port produced incorrect, unstable
dashboard data. The proxy core is provider-isolated and
multi-provider-safe by design; the defect was in the observability
layer. The Codex `/v1/responses` **WebSocket** handler was the only path
in the proxy that wrote to the request logger by hand instead of through
the unified `emit_request_outcome` funnel, and it did so twice per
session close: the per-turn funnel record plus an unconditional
cumulative session-summary `RequestLog`. This PR removes the duplicate
summary log so Codex WS emits exactly one request log per turn, matching
the HTTP provider paths.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Dropped the duplicate cumulative session-summary `RequestLog` in the
Codex WS handler while preserving the per-turn `emit_request_outcome`
path.
- Preserved gated `request_messages` and `turn_id` on residual outcomes
so dashboard telemetry keeps the useful attribution without
double-counting tokens.
- Ensured explicit `--anyllm-provider` wins over a leaked
`HEADROOM_ANYLLM_PROVIDER` environment variable.
- Registered retry delay settings that had drifted out of the settings
registry.
- Hardened tests against developer-shell `HEADROOM_*` /
`ANTHROPIC_CUSTOM_HEADERS` leakage and stabilized several focused
proxy/wrap test fixtures.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/pytest tests/ -q -p no:cacheprovider
8529 passed, 537 skipped, 5831 warnings in 276.25s (0:04:36)

$ .venv/bin/ruff check <touched files>
All checks passed!
```

## Real Behavior Proof

- Environment: macOS (Darwin 25.4.0), Python 3.13.14, pytest 9.0.3, ruff
via project venv, branch `fix/multi-provider-runtime`.
- Exact command / steps: Ran the full test suite without pytest cache
provider and Ruff on all touched files; used `git stash` to confirm the
stale fake-config failures pre-existed this change.
- Observed result: Full suite passed with no failures; Ruff passed;
Codex WS now routes end-of-session logging through
`emit_request_outcome`, emitting one request log per turn with the same
accounting model as Anthropic HTTP turns.
- Not tested: Live simultaneous Claude + Codex dashboard run. `mypy
headroom` was not run to completion; a scoped run reported one
pre-existing `settings_store.py:470` coercion error outside this diff.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A - server-side observability fix; no UI markup changed.

## Additional Notes

- The proxy's multi-provider routing, header/auth isolation, and
per-model cache keying are already correct and unchanged here; only the
WS observability write path was double-counting.
- Architectural assessment:
`plans/reports/research-260714-0004-multi-provider-upstream-compression-report.md`;
root-cause + resolution trail:
`plans/reports/debug-assessment-260714-0011-dashboard-instability-mixed-claude-codex-report.md`.
- No live simultaneous Claude + Codex dashboard run was performed;
validation is from test coverage and code review of the WS logging path.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 18:18:34 +00:00
c3db8e47f8 fix(install): default docker image to headroomlabs-ai GHCR registry (#1867) (#2039)
## Description

The GitHub repository was transferred from `chopratejas/headroom` to
`headroomlabs-ai/headroom`. GitHub 301-redirects transferred repos for
web and git operations, but **GitHub Container Registry (GHCR) does
not** — the old package `ghcr.io/chopratejas/headroom` is now orphaned
and frozen (its `latest` tag stopped advancing at `0.27.0`), while CI
publishes new images to `ghcr.io/headroomlabs-ai/headroom` (the workflow
derives the path from `${{ github.repository }}`).

Headroom's install tooling still defaulted to the dead path, so
`headroom install apply --preset persistent-docker`, `headroom init`,
and the standalone install scripts all pulled a stale `0.27.0` image
instead of the current release.

This changes every docker-image **default** to
`ghcr.io/headroomlabs-ai/headroom:latest`.

Closes #1867

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/models.py` — `InstallManifest.image` default.
- `headroom/cli/install.py` — `--image` Click option default.
- `headroom/cli/init.py` — two `InstallManifest(...)` image args.
- `scripts/install.sh` / `scripts/install.ps1` — `IMAGE_DEFAULT` /
`$ImageDefault` plus the `--image` help-text default.
- `docker/docker-compose.native.yml` — image default in both services.
- `tests/test_install/test_planner.py` (4) and
`tests/test_install/test_runtime.py` (5) — updated the assertions that
pinned the old image (including `assert "<image>" in command`), so they
now verify the corrected registry threads through the planner and docker
runtime command.

Scope note: `github.com/chopratejas/...` links and the plugin
marketplace slug are intentionally **not** changed — GitHub redirects
those, so they still work. Only the genuinely-dead GHCR image references
are touched. No new dependency, no new abstraction.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_install/ -q
99 passed, 1 skipped in 48.34s

$ uv run ruff check headroom/install/models.py headroom/cli/install.py headroom/cli/init.py
All checks passed!

$ grep -rn "ghcr.io/chopratejas/headroom" headroom/ scripts/ docker/ tests/
# (no source matches — every default now points at headroomlabs-ai)
```

The install-test assertions pinned the old image, so they fail against
the old defaults and pass after the fix — they are the regression guard.

## Real Behavior Proof

- **Environment:** Windows 11, Python 3.13.5, headroom installed from
this branch (editable, via `uv`).
- **Exact command / steps:** The dead-registry claim is verifiable at
the registry level, independent of a release:
  ```text
docker pull ghcr.io/chopratejas/headroom:latest # old default → 0.27.0
(frozen / orphaned)
docker pull ghcr.io/headroomlabs-ai/headroom:latest # new default →
current release
  ```
And the install pipeline now emits the correct image (covered by
`tests/test_install/test_runtime.py`, which asserts the resolved docker
command contains `ghcr.io/headroomlabs-ai/headroom:latest`).
- **Observed result:** `grep` confirms no `ghcr.io/chopratejas/headroom`
default remains in code, scripts, compose, or tests;
`tests/test_install/` is green (99 passed) with the corrected image
asserted end-to-end through planner → runtime command.
- **Not tested:** A live `docker pull` of both tags on this specific
machine (no local Docker daemon guaranteed) — the registry difference is
reproducible by anyone running the two `docker pull` commands above; and
an end-to-end `headroom install apply --preset persistent-docker`
against a real Docker host.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- Documentation checklist item is N/A — no user-facing docs surface
beyond the CHANGELOG entry.
- `mypy` left unchecked — not run as part of this verification; the
change is a string-default swap with no type-level surface.
- The `--image` help text and `docker-compose.native.yml` were included
so every user-facing default is consistent; only the dead GHCR image
string was changed.

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-13 14:01:28 -04:00
Abhay SinghandTejas Chopra b097ef3e25 fix(install): don't let host env override the manifest in persistent-docker (#2090)
## Description

In persistent-docker deployments a stale host env var can silently
override the value the deployment manifest pinned for the container.

`build_runtime_command` builds the `docker run` argv in two passes:

1. It emits the manifest's pinned env as `--env NAME=VALUE` (from
`base_env` plus the deployment env).
2. It then walks `os.environ` and, for every name matching a
`PASSTHROUGH_ENV_PREFIXES` prefix, appends a bare `--env NAME` so the
host value is forwarded into the container.

A manifest-pinned name and a host-exported name can collide when they
share a passthrough prefix. `HEADROOM_BACKEND` is the clearest case: the
manifest pins `--env HEADROOM_BACKEND=anthropic` in pass 1, and pass 2
also matches the `HEADROOM_` prefix and appends a bare `--env
HEADROOM_BACKEND`. Docker resolves duplicate `--env` flags last-wins,
and the bare passthrough comes last, so a stale host export
`HEADROOM_BACKEND=anyllm` wins and the container runs a different
backend than its deployment config says.

`start_persistent_docker` runs the resulting command through
`subprocess.run` with the parent process environment, so whatever the
operator happened to have exported leaks in and overrides the manifest.

The fix skips the bare passthrough for any name the manifest already
pins, so the pinned value stands while unrelated host secrets (API keys
and so on) are still passed through as before.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/runtime.py`: skip the bare `--env NAME` passthrough
when `NAME` is already pinned by the manifest (`and name not in
runtime_env`).
- `tests/test_install/test_runtime.py`: add
`test_build_runtime_command_docker_manifest_env_beats_host_passthrough`,
which exports a conflicting `HEADROOM_BACKEND` and asserts the command
keeps the manifest value and emits no bare passthrough for it.
- `CHANGELOG.md`: Bug Fixes entry.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ uvx ruff@0.15.17 format headroom/install/runtime.py tests/test_install/test_runtime.py
2 files left unchanged
$ uvx ruff@0.15.17 check headroom/install/runtime.py tests/test_install/test_runtime.py
All checks passed!
$ python -m py_compile headroom/install/runtime.py tests/test_install/test_runtime.py
OK
```

## Real Behavior Proof

- Environment: local checkout, Python 3.11, `uvx ruff@0.15.17`.
- Exact command / steps: ran a standalone script that reproduces the
two-pass argv build and models Docker's duplicate `--env` last-wins
resolution, with the manifest pinning `HEADROOM_BACKEND=anthropic` and
the host exporting `HEADROOM_BACKEND=anyllm`.
- Observed result: the old build resolves the effective
`HEADROOM_BACKEND` to the host value `anyllm` (bare passthrough wins);
the new build keeps the manifest value `anthropic` and emits no bare
`HEADROOM_BACKEND` token, while a non-pinned passthrough
(`ANTHROPIC_API_KEY`) is still forwarded.
- Not tested: I did not run the full `pytest` suite locally because it
pulls in the ML stack; the new regression test is left for CI.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML dependencies, which I can't run in
this environment; the change is a pure function over
`build_runtime_command`, verified by the standalone proof above and
covered by the new regression test for CI.

Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-13 10:24:41 -04:00
JD Davis 1d2b76e72e fix: harden persistent install startup (#1851)
## Description

Hardens persistent install startup and proxy compression behavior for
issue #1843. Repeated `headroom install start` / scheduled ensure calls
no longer spawn duplicate runtimes by default, and `/v1/compress` now
fails open on compression timeout instead of returning a 503. The PR
also adds a machine-readable platform feature matrix and app-level
stabilization tests for health, compression functionality, timeout
behavior, and matrix evidence.

Refs #1843

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Wrapped direct persistent deployment starts with the existing
profile-local runtime start lock.
- Made `headroom install start` idempotent when the deployment is
already healthy.
- Added wedged-runtime handling: if a PID is running but `/readyz` does
not recover inside the grace window, stop it before starting again.
- Kept `install agent ensure` inside the already-held lock while
delegating to the shared start helper.
- Changed `/v1/compress` timeout behavior from `503 compression_timeout`
to fail-open `200` with original messages, `compression_skipped: true`,
and `skip_reason: compression_timeout`.
- Added `tests/test_platform_stabilization_functional.py` covering real
FastAPI health/compression routes, successful compression metrics,
timeout fail-open speed, and a real JSON tool payload that reduces
tokens.
- Added `docs/platform-feature-matrix.json` and
`docs/platform-stabilization.md` for Linux/macOS/Windows hardening
coverage and known gaps.
- Strengthened matrix tests so cited local test/workflow paths must
exist.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Clean source tree without copied Rust extension: functional module is skipped locally, as CI copies _core from the built wheel.
> python -m pytest tests/test_install tests/test_cli/test_install_cli.py tests/test_platform_feature_matrix.py tests/test_platform_stabilization_functional.py -q
collected 120 items / 1 skipped
119 passed, 2 skipped in 17.08s

> python -m ruff check headroom/proxy/handlers/openai.py tests/test_platform_stabilization_functional.py tests/test_platform_feature_matrix.py
All checks passed!

# Local Windows compiled-core proof:
> python -m maturin build --profile ci --out dist-local
Built wheel for abi3 Python >= 3.10 to dist-local\headroom_ai-0.29.0-cp310-abi3-win_amd64.whl

# Copied _core.pyd from the wheel into headroom/ for local route execution, then:
> python -m pytest tests/test_platform_stabilization_functional.py -q
collected 4 items
4 passed in 6.71s

> python -m pytest tests/test_install tests/test_cli/test_install_cli.py tests/test_platform_feature_matrix.py -q
collected 120 items
119 passed, 1 skipped in 17.16s

Commit hooks:
Sync plugin versions.....................................................Passed
check for merge conflicts................................................Passed
ruff.....................................................................Passed
ruff-format..............................................................Passed
mypy.....................................................................Passed
```

## Real Behavior Proof

- Environment: Windows 11, PowerShell, Python 3.13.13, worktree
`C:\git\headroom-stabilization` on branch
`jd/cross-platform-stabilization`.
- Exact command / steps: built the Windows wheel with `maturin`,
extracted `_core.pyd`, ran the new FastAPI route tests and
install/matrix tests listed above, then removed generated artifacts
before committing.
- Observed result: direct start paths now no-op when healthy, skip
spawning when the start lock is contended, and stop a wedged runtime
before restart. `/v1/compress` now returns original messages quickly on
timeout instead of a 503. The real JSON tool-payload smoke test returns
`tokens_before > tokens_after`, `tokens_saved > 0`, `compression_ratio <
1.0`, and non-empty transforms through the public route.
- Not tested: full native Windows persistent process e2e remains blocked
by the upstream CRT/wheel issue already documented in workflows and in
the matrix. No real OS service was installed locally; service manager
behavior is covered by argument-level unit tests.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

CHANGELOG is not updated because this is an unreleased
hardening/test/documentation pass. The platform matrix intentionally
records partial/blocked Windows/macOS e2e gaps instead of claiming full
coverage where the repo cannot currently run it.
2026-07-10 00:40:34 -04:00
Abhay SinghandJerrettDavis 816cb85fa8 fix(install): close parent log fd in start_detached_agent (#1576)
## Description

`start_detached_agent()` opens the agent log file and hands it to
`subprocess.Popen` as `stdout`/`stderr`, then returns the process
**without
closing the parent's copy of the file descriptor**. The child inherits
the fd
and writes to it, but the parent keeps its own copy open forever.

The result: every `headroom install start` leaks one file descriptor in
the
parent, and the leaked handle pins the log file open so it can't be
rotated.
On a tight `ulimit` or inside a container, repeated starts can walk
straight
into the fd limit.

```python
# headroom/install/runtime.py — before
log_file = open(log_file_path, "a", encoding="utf-8", errors="replace")
kwargs = {"stdout": log_file, "stderr": log_file, ...}
return subprocess.Popen(command, **kwargs)   # parent's log_file never closed
```

The fix closes the parent's copy in a `try/finally` right after `Popen`
returns:

```python
try:
    proc = subprocess.Popen(command, **kwargs)
finally:
    # The child has inherited the log file descriptor, so the parent's
    # copy is dead weight. Closing it (even when Popen raises) avoids
    # leaking one fd per `headroom install start` and lets the log file
    # be rotated.
    log_file.close()
return proc
```

The `finally` is deliberate: it also covers the case where `Popen`
itself
raises (bad executable, fork failure), which would otherwise leak the
just-opened handle. This matches the `with open(...)` pattern already
used by
`run_foreground()` a few lines above.

Closes #1554

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/install/runtime.py`: close the parent's log file descriptor
in a `try/finally` after `subprocess.Popen` in `start_detached_agent()`,
so it is released on the normal path and when `Popen` raises.
- `tests/test_install/test_runtime.py`: add two regression tests — one
for a normal start, one for `Popen` raising — asserting the parent's log
handle is closed afterwards.
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

Before the fix (`runtime.py` reverted to its parent commit, new tests
kept) —
the assertion inspects the *actual* log file handle and finds it still
open:

```text
E   AssertionError: assert False is True
E    +  where False = <_io.TextIOWrapper name='...\deploy\demo\runner.log' mode='a' encoding='utf-8'>.closed
FAILED tests/test_install/test_runtime.py::test_start_detached_agent_closes_parent_log_fd
FAILED tests/test_install/test_runtime.py::test_start_detached_agent_closes_log_fd_when_popen_raises
============================== 2 failed in 0.43s ==============================
```

After the fix:

```text
tests\test_install\test_runtime.py ................
====================== 16 passed, 1 deselected in 0.35s =======================
```

(The one deselected test,
`test_runtime_start_lock_blocks_another_process`, is a
pre-existing failure on my Windows box — it fails identically on a clean
checkout of `main` and is unrelated to this change.)

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, headroom built from this
branch (`uv sync --extra dev`).
- Exact command / steps: ran the two regression tests, which drive the
real `start_detached_agent` code path — real `open()`, real
`Popen(stdout=...)`, real (or missing) `close()`. Only
`subprocess.Popen` is stubbed so the test never launches an actual
detached agent; the fd-lifecycle bug lives entirely in how the parent
handles its own handle, and that runs for real.
- Observed result: the log file handle the parent passed to `Popen` is
`.closed == False` before the fix and `.closed == True` after — for both
the normal path and the `Popen`-raises path (output above).
- Not tested: I intentionally did not spin up many real detached agents
to watch the OS fd table grow — on Windows that means flashing console
windows and isn't a clean signal anyway. The handle-state assertion on
the real file object is the deterministic equivalent. Did not run the
full `mypy headroom` pass (one-line lifecycle change, no new types).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Single logical change, no new dependencies, default behavior otherwise
unchanged.
- Rebased/merged latest `main` to clear a `CHANGELOG.md` conflict.
- @chopratejas this is a sibling of #1555 (the `wrap.py` readiness-loop
handle leak you've got filed). I scoped this PR to #1554 only; happy to
follow up on #1555 separately if you'd like.

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-01 23:07:01 -05:00
ParideboyandClaude Opus 4.8 6b227b9c90 fix(install): use Windows-safe PID liveness probe in runtime_status (#1544) (#1560)
## Description

`headroom install status` crashed with `OSError: [WinError 87] The
parameter is incorrect` on Windows and, worse, tore down the live proxy
it was only meant to inspect. `runtime_status()` probed liveness with a
bare `os.kill(pid, 0)` guarded only by `except OSError`. Against a
detached Windows agent (`DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP`),
that call raises WinError 87, which CPython surfaces as a `SystemError`
— not an `OSError` — so it escaped the handler, crashed status, and left
the deployment dead (PID file removed, port 8787 freed). This mirrors
the `os.kill`/`SystemError` fix PR #1315 applied to `cli/wrap.py`.

Closes #1544

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added a shared Windows-safe `headroom._subprocess.pid_alive()` helper:
rejects non-positive PIDs, prefers `psutil.pid_exists()`, and treats
`SystemError` (WinError 87) as "not alive".
- `install/runtime.py` `runtime_status()` now delegates to `pid_alive()`
instead of an unguarded `os.kill(pid, 0)`.
- `install/runtime.py` `stop_runtime()` now also catches `SystemError`
to avoid the same crash class on shutdown.
- `cli/wrap.py` `_pid_alive()` now delegates to the shared helper, so
the marker-cleanup path and the install/runtime status path share one
liveness probe (the shared helper the issue asked for).
- Added regression tests for the helper and `runtime_status`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ ruff check .
All checks passed!

$ ruff format --check headroom/_subprocess.py headroom/install/runtime.py headroom/cli/wrap.py tests/test_install/test_runtime.py tests/test_pid_alive.py
5 files already formatted

$ mypy headroom --ignore-missing-imports
(exit 0)

$ pytest tests/test_pid_alive.py tests/test_install/test_runtime.py tests/test_cli/test_wrap_helpers.py tests/test_cli/test_wrap_persistent.py \
    --deselect "tests/test_install/test_runtime.py::test_runtime_start_lock_blocks_another_process"
89 passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13, ruff 0.15.17 / mypy 1.20.2 /
pytest 9.1.0, psutil 7.2.2, branch `fix/1544-windows-pid-liveness`.
- Exact command / steps: ran the four checks above; the new
`tests/test_pid_alive.py` injects a `SystemError` (simulated WinError
87) and a stubbed `psutil` to drive both code paths, and
`test_runtime_status_*` exercise `runtime_status()` end to end with a
PID file present.
- Observed result: `runtime_status` returns `"running"` for a live PID
without sending any signal (asserted), returns `"stopped"` instead of
crashing when the probe raises `SystemError`, and the helper only ever
passes signal `0`. All 89 targeted tests pass; ruff/format/mypy clean.
- Not tested: the full `headroom install apply --preset persistent-task`
detached-agent reproduction against a live proxy was not run end to end;
it is instead covered by the deterministic `SystemError`/WinError-87
injection regression tests.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- One pre-existing test,
`test_runtime_start_lock_blocks_another_process`, fails on my local
Windows checkout **before** these changes too (it asserts cross-process
file-lock blocking and depends on `HOME` semantics that differ on
Windows). It is unrelated to this fix and is deselected above; it passes
on the Linux CI runners.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 17:13:11 -05:00
Ben YounesandClaude Opus 4.8 feedead077 fix(install): stop duplicating ENTRYPOINT in persistent-docker runtime command (#833) (#1348)
## Description

`headroom install apply --preset persistent-docker` pulls the image,
starts the container, then fails after ~45s with "Deployment 'default'
did not become ready after start." The rollback removes the container
and manifest, leaving nothing running and no logs.

Root cause: the published image already bakes the proxy invocation into
its ENTRYPOINT (`Dockerfile`: `ENTRYPOINT ["headroom", "proxy"]`), but
`build_runtime_command()` in `headroom/install/runtime.py` re-added
`headroom proxy` after the image name. Docker concatenates ENTRYPOINT +
args, so the container ran `headroom proxy headroom proxy --host 0.0.0.0
...` and Click aborted with `Got unexpected extra arguments (headroom
proxy)`.

The runtime command now appends only the proxy flags after the image
name, substituting the all-interface container bind host for the host
pair carried in `proxy_args`.

Closes #833

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/runtime.py`: drop the duplicated `headroom proxy`
from the docker `build_runtime_command` output; append only `--host
<bind> *proxy_args[2:]`. Extracted `CONTAINER_BIND_HOST` and
`_PROXY_ARGS_HOST_PAIR_LEN` named constants.
- `tests/test_install/test_runtime.py`: new regression test asserting
the args appended after the image name never re-add the `headroom proxy`
ENTRYPOINT.
- `CHANGELOG.md`: Unreleased → Bug Fixes entry.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_install/ -q
91 passed, 1 skipped in 5.48s

$ uv run ruff check headroom/install/runtime.py tests/test_install/test_runtime.py
All checks passed!

$ uv run mypy headroom/install/runtime.py
Success: no issues found in 1 source file
```

#### RED → GREEN proof

RED — new test with the prod fix reverted (test kept):

```text
E   AssertionError: container args re-add the ENTRYPOINT — got
    ['headroom', 'proxy', '--host', '0.0.0.0', '--port', '8787', '--backend', 'anthropic']
FAILED tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_does_not_duplicate_entrypoint
1 failed in 0.17s
```

GREEN — with the fix applied:

```text
tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_does_not_duplicate_entrypoint
1 passed in 0.11s
```

## Real Behavior Proof

- Environment: Linux, Python 3.13, headroom @ this branch.
- Exact command / steps: reproduce the exact concatenation Docker
performs (ENTRYPOINT `headroom proxy` + the buggy CMD `headroom proxy
--host 0.0.0.0 --port 8787`):

  ```text
  $ headroom proxy headroom proxy --host 0.0.0.0 --port 8787
  Usage: headroom proxy [OPTIONS]
  Try 'headroom proxy --help' for help.
  Error: Got unexpected extra arguments (headroom proxy)
  ```

This is the exact error from the issue. After the fix,
`build_runtime_command` appends only the flags after the image name:

  ```text
args after image: ['--host', '0.0.0.0', '--port', '8787', '--backend',
'anthropic']
  ```

so the container runs `headroom proxy --host 0.0.0.0 --port 8787
--backend anthropic` (ENTRYPOINT + flags) and Click accepts it.
- Observed result: pre-fix Click aborts with the unexpected-arguments
error (container crash-loops); post-fix the command line is valid.
- Not tested: pulling and running the real `ghcr.io` image end-to-end
(requires the published image + Docker host); the failure is fully
determined by the generated argv, which is covered above and by the unit
test.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

Scope is limited to the docker runtime command construction. The Python
(`runtime_kind=python`) path was already correct and is unchanged.
Screenshots N/A (CLI-only change).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:13:38 -05:00
Hc 9252d852c5 fix(init): guard persistent task startup (#616)
## Description

Prevent `headroom init` hooks from spawning duplicate persistent-task
runners while a proxy is still starting.

Fixes #615

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Problem

`_ensure_profile_running()` checked readiness for only one second and
then launched `start_detached_agent()` whenever the proxy was not ready
yet. When Claude/Codex hooks fired close together, each hook could race
through that path and spawn another detached persistent-task runner.

## Changes Made

- Add a profile-local, nonblocking runtime start lock around init hook
startup.
- Re-check readiness after acquiring the lock so late-arriving hooks do
not start a duplicate runner.
- If a runtime is already alive, wait up to 15 seconds for readiness
before stopping and restarting it.
- Add regression tests for lock contention, slow startup, and
cross-process lock behavior.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

## Test Output

```
UV_SKIP_WHEEL_FILENAME_CHECK=1 uv run --frozen --extra dev --extra proxy pytest tests/test_cli/test_init_cli.py tests/test_cli/test_install_cli.py tests/test_cli/test_wrap_persistent.py tests/test_install/test_runtime.py
# 89 passed in 0.61s

UV_SKIP_WHEEL_FILENAME_CHECK=1 uv run --frozen --extra dev --extra proxy ruff check .
# All checks passed!

UV_SKIP_WHEEL_FILENAME_CHECK=1 uv run --frozen --extra dev --extra proxy ruff format --check .
# 775 files already formatted

UV_SKIP_WHEEL_FILENAME_CHECK=1 uv run --frozen --extra dev --extra proxy mypy headroom --ignore-missing-imports
# Success: no issues found in 346 source files
```

Manual sandbox check:

```
# before this change: 3 ensure calls spawned 3 detached starts
# after this change: 3 ensure calls spawned 1 detached start while the runtime was still starting
```

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

Docs and CHANGELOG were left unchanged because this is a small runtime
bug fix with no user-facing CLI/API change.
2026-06-10 20:34:43 -05:00
GarmandClaude Opus 4.7 efd2ac1ca4 chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.

Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.

Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:33:30 +02:00
JerrettDavisandCopilot d4574f4ae2 test: avoid global platform leaks in install tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 08:15:41 -05:00
JerrettDavisandCopilot 38bf3e639c test: expand coverage across helper slices
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 07:39:52 -05:00
JerrettDavisandClaude Opus 4.7 4a87753713 feat(docker): forward HEADROOM_WORKSPACE_DIR and HEADROOM_CONFIG_DIR into containers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 19:19:25 -05:00
JerrettDavisandCopilot bd242fc62d test: expand persistent install coverage
Add focused regression coverage for install, runtime, provider, state, health, supervisor, and persistent wrap flows so the new persistent deployment surfaces are exercised more thoroughly in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 18:24:15 -05:00
JerrettDavisandCopilot 865bef2216 fix: harden persistent install wrappers and review gaps
Align Docker-native wrapper help and runtime behavior with the Python install contract, including persistent deployment metadata, baked install-image defaults, and explicit unsupported wrap targets.

Harden the Python persistent-install path with profile validation, safer provider-scope handling, Windows environment restoration, runtime parity improvements, and rollback-safe apply/update behavior.

Update README, Docker install docs, CI, and focused regressions to cover the Windows BOM failure, wrapper parity, compose coverage, and Docker-native wrap behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 18:03:21 -05:00
JerrettDavisandCopilot 21896a095c feat: add persistent install lifecycle management
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 13:47:05 -05:00