Commit Graph
4 Commits
Author SHA1 Message Date
Radhakrishnan PachyappanandJerrettDavis 125a703a65 fix(security): restrict the Windows runtime log via NTFS ACL, not just warn (#4135)
## Description

Windows never got the owner-only guarantee POSIX mode bits give the
proxy log, its rotated backups, and the JSONL request log — `os.chmod`
there only toggles the read-only attribute and leaves the real access
control (the ACL) untouched. The prior fix (#3728) correctly scoped that
gap and warned about it, but the warning text read as cryptic to at
least one user (reported here and on Discord), and closing the gap was
explicitly left as a follow-up pending a Windows CI runner to verify
against.

This implements that follow-up.

Closes #4058

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- `headroom/_fileperms_windows.py` (new): the NTFS ACL implementation
via `ctypes` (no `pywin32` dependency) — a DACL with exactly one entry,
the calling user with full control, built fresh rather than merged with
the inherited entries that made the file readable in the first place,
applied with `PROTECTED_DACL_SECURITY_INFORMATION` so those inherited
entries are dropped.
- `headroom/fileperms.py`:
`restrict_fd_to_owner`/`restrict_path_to_owner` go through the new
module on Windows instead of giving up immediately; added
`verify_owner_only(path)`, which re-reads the actual DACL rather than
trusting a static platform flag — the ACL call can fail for reasons
unrelated to this code (a non-NTFS volume, a missing privilege).
- `headroom/proxy/helpers.py`: the startup warning now fires only when
`verify_owner_only` reports failure (not unconditionally on Windows),
and the wording says plainly that other accounts may be able to read the
file instead of the more technical original phrasing.
- `tests/test_cache_breakpoint_diagnostics.py`: new Windows-specific
tests exercising the ACL path directly (mirroring the existing POSIX
mode-bit tests), plus updates to the two tests whose assumption
("Windows protection is impossible") this PR deliberately changes.
- `docs/content/docs/proxy.mdx`: updated the "Who can read the logs"
section to describe the ACL instead of "Headroom does not restrict the
log file."
- `.github/workflows/ci.yml`: added a `windows-file-permissions` job
(same lightweight pattern as the existing
`windows-tool-result-interceptors` job) so this is exercised on a real
Windows runner going forward, addressing the maintainer's own "I'd want
a Windows runner in CI first" from #3728's review thread.

Deliberately **not** extended to the sqlite-backed memory/CCR stores
(`ensure_private_file`/`connect_private_sqlite`) or `private_dir` —
those remain unenforced on Windows, same as before this change, to keep
this PR scoped to what #4058 is actually about.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality

### Test Output

```text
$ pytest tests/test_cache_breakpoint_diagnostics.py tests/test_private_state_files.py tests/test_proxy_handler_helpers.py -q
99 passed, 49 skipped in 13.50s

$ ruff check headroom/fileperms.py headroom/_fileperms_windows.py headroom/proxy/helpers.py tests/test_cache_breakpoint_diagnostics.py
All checks passed!

$ ruff format --check <same files>
4 files already formatted

$ mypy headroom/fileperms.py headroom/_fileperms_windows.py --ignore-missing-imports --platform linux
Found 1 error in 1 file (checked 2 source files)
# that 1 error is in headroom/providers/model_metadata.py, pre-existing and
# unrelated -- confirmed via `git stash` against main before these changes.

$ actionlint .github/workflows/ci.yml
# clean, no output
```

## Real Behavior Proof

- Environment: actual Windows 11 machine (not simulated/mocked) — this
is the one part of #3728 the original author explicitly could not do ("I
cannot execute or test on this machine")
- Exact command / steps: created a file, called the new
`restrict_handle`/`restrict_path`, then independently verified the
result with `icacls <path>` (a separate tool, not my own code checking
its own work)
- Observed result:
  ```
  C:\Users\...\Temp\acl_proto_test.txt LAPTOP-...\<user>:(F)
  Successfully processed 1 files; Failed processing 0 files
  ```
Exactly one ACE, granting the current user full control — no `Everyone`,
no `Users`, no leftover inherited entries. Also ran the full
`test_cache_breakpoint_diagnostics.py` suite on this same machine,
including the new Windows-specific tests
(`test_runtime_log_is_owner_only_via_acl_with_previews_off`,
`test_rotated_backups_are_owner_only_via_acl`) which were previously
impossible to exercise without a Windows box.
- Not tested: a non-NTFS volume or a locked-down-privilege environment
(the failure paths `verify_owner_only` is meant to catch) — those fall
back to the warning, covered by the existing simulated test, but I don't
have a non-NTFS Windows volume handy to exercise for real.

## Runtime Rollout Safety

- Rollout-managed feature(s): N/A
- Minimum rollout channel: N/A
- Stable/default behavior changed: Only on Windows — the runtime log is
now actually restricted instead of warned-about; POSIX behavior is
unchanged.
- Kill switch / disable path: None needed — this only adds protection
where there was none; a failure falls back to the pre-existing
warning-only behavior automatically via `verify_owner_only`.
- Unsafe override required: No
- Qualification impact: None
- Rollback path: Revert; Windows returns to warn-only, same as before
this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Scope boundary, stated explicitly rather than left implicit: this does
not touch `ensure_private_file`/`connect_private_sqlite`/`private_dir`,
which stay unenforced on Windows as before. Extending ACL protection
there would be a reasonable follow-up but is a separate, larger piece of
surface (sqlite main file + `-wal`/`-shm`/`-journal` sidecars) than what
#4058 asked about.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4135"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:39:05 -07:00
Tejas ChopraandClaude Opus 5.5 72302a3998 fix(proxy): say why proxy.log is missing and why a knob or extension was ignored (#4088)
## Description

Several failures an operator needs to see are swallowed silently or
logged only at DEBUG. This PR makes each of them visible with one
warning. It does not change defaults or control flow.

1. **Missing runtime log.** `_setup_file_logging` caught `OSError` with
`pass`. On a read-only workspace, a wrong owner, or a file where the
logs directory belongs, `proxy.log` simply never appeared and nothing
said why. It now logs `Headroom runtime log disabled: cannot write the
log file: <OSError>. Point HEADROOM_WORKSPACE_DIR at a writable
directory to restore proxy.log.` The OSError text names the path. The
symlink case just above it already warned this way.
2. **Six import-time numeric knobs.** Each one fell back to its default
on an unparseable value, without a word:
   - `HEADROOM_COMPRESSION_TIMEOUT_SECONDS`
   - `HEADROOM_COLD_START_FAST_PASS_TIMEOUT_SECONDS`
   - `HEADROOM_EAGER_PRELOAD_TIMEOUT_SECONDS`
   - `HEADROOM_COMPRESSION_CACHE_MAX_SESSIONS`
   - `HEADROOM_COMPRESSION_CACHE_TTL_SECONDS`
   - `HEADROOM_COMPRESSION_CACHE_MAX_ENTRIES`

Each now logs `Ignoring invalid NAME='value'; using DEFAULT` once, at
import. That matches the idiom in `hard_watchdog.py`.
3. **Extension discovery failure.** When entry-point enumeration fails,
every proxy extension is dropped, Enterprise plugins included. That was
logged at DEBUG. It is now a warning: `proxy extensions: entry-point
enumeration failed (<describe_exception>), so no proxy extensions will
be installed`. `describe_exception` gives the exception types and code
locations, never the message, and no traceback is logged at any level.
It runs at startup only: once in `install_all`, and once more for the
CLI banner.
4. **Pricing lookups in `cost.py`.** `_get_list_price`,
`_get_output_price`, `_get_cache_prices` and the cache-rate lookup in
the savings summary returned `None` on any exception. A pricing bug
therefore showed up as a $0 cost with no trace. All four, plus the
existing `estimate_cost` path, go through one helper,
`_warn_pricing_once(lookup, what, model, exc)`. Example line: `Failed to
get pricing for model 'gpt-4o': RuntimeError at
headroom/proxy/cost.py:... in estimate_cost`. It logs the lookup name,
`safe_id(model)` and `describe_exception(exc)` (types and code
locations, never the message), at WARNING only. No level carries the raw
exception or a traceback, because a pricing backend error can quote
provider responses or credentials. Dedupe is by `(lookup, model)`
through the shared `WarnOnce(1024, "unpriceable models")`. Each pair
warns at most once an hour (the `WarnOnce` window). Past 1024 models,
one overflow notice is logged, and a key is not used up while WARNING is
disabled.
5. **Malformed `HEADROOM_OPENAI_TOOL_SEARCH_MODELS` regex.** The
override was ignored on every request with no log. It now warns once per
bad pattern, and still falls back to the version gate as before. A valid
pattern re-arms the warning, so bad A, then valid B, then bad A again
warns twice.

None of the new lines logs header values, request content or
credentials. They log only knob names, their raw values (operator-set
env), model names, regex patterns, exception type names, the `re.error`
text, and `OSError` text naming a local path. No exception message or
traceback from pricing or extension discovery reaches any log level.

Refs #4136

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`:
- add `_warn_invalid_env_number(name, default)` and call it from the six
import-time `except ValueError` branches;
  - log the `OSError` in `_setup_file_logging`;
- add a once-per-pattern warning for a malformed
`HEADROOM_OPENAI_TOOL_SEARCH_MODELS`.
- `headroom/proxy/extensions.py`: entry-point enumeration failure logs
at warning with `describe_exception(exc)` instead of at debug.
- `headroom/proxy/cost.py`:
- the four silent pricing `except Exception` branches call
`_warn_pricing_once`;
- `_warn_pricing_once(lookup, what, model, exc)` logs `safe_id(model)`
plus `describe_exception(exc)` at WARNING, with nothing at DEBUG. It
dedupes through `log_safety.WarnOnce`.
- `tests/test_cost_pricing_warning_dedup.py`: the tests reset with a
fresh `WarnOnce`, and match the quoted model id (`for model
'glm-5.2':`).
- Regression tests. Each one attaches a handler to the module logger
directly, because caplog misses `headroom.*` records after proxy
startup. Each positive test fails on main and is paired with a negative
case that passes on both.
  - `tests/test_cache_breakpoint_diagnostics.py`:
- `test_runtime_log_says_why_it_is_missing` makes the logs directory a
regular file. It asserts one warning that names the path and
`HEADROOM_WORKSPACE_DIR`.
    - Negative: `test_runtime_log_stays_quiet_when_writable`.
  - `tests/test_openai_tool_search_deferral.py`:
- `test_malformed_override_warns_once_not_per_request` calls the gate 5
times with a bad pattern. It asserts exactly one warning and that the
version-gate result is unchanged.
    - Negative: a valid override is silent (same test).
- `test_valid_override_rearms_the_malformed_warning`: bad, valid, bad
gives two warnings.
  - `tests/test_proxy/test_compression_timeout_config.py`:
- `test_bad_env_says_it_was_ignored` reloads `helpers` with
`HEADROOM_COMPRESSION_TIMEOUT_SECONDS=not-a-number` and asserts the
exact warning.
- Negative: `test_valid_env_is_silent`. The existing fixture reloads
`helpers` afterwards.
  - `tests/test_proxy_extensions.py`:
- `test_discover_warns_when_entry_point_enumeration_fails` makes
`importlib.metadata.entry_points` raise. It asserts one WARNING that
says no extensions will be installed and names `RuntimeError` but not
the exception text.
    - Negative: `test_discover_is_quiet_when_nothing_is_installed`.
  - `tests/test_cost_cache_prices.py`:
- `test_cache_rate_failure_warns_once_per_model` makes `resolve_rates`
raise. It asserts two `_get_cache_prices` calls give one warning naming
the model and `RuntimeError`, without the exception text.
    - Negative: a resolvable model is silent (same test).
- `test_pricing_error_never_logs_its_text_at_any_level` runs the real
`CostTracker.estimate_cost('gpt-4o', 100, 10)` with a pricing error
whose text holds an Authorization credential canary, with
`headroom.proxy` at DEBUG. It formats every captured record with
`logging.Formatter` (which renders `exc_info`), asserts the canary is
absent and the result is `None`, and asserts the warning is present. On
the previous commit it fails: the formatted DEBUG traceback contains the
canary.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_cost_cache_prices.py -k never_logs   # previous commit 3202058c (test adapted to its old dedupe set)
E     RuntimeError: upstream rejected request (Authorization: Bearer sk-canary-7f3a9c)']
======================= 1 failed, 25 deselected in 3.48s =======================
$ pytest -q -p no:cacheprovider tests/test_pr208_changes.py tests/test_cost_budget_basis.py tests/test_cost_tracker_counterfactual.py tests/test_cost_tracker_totals.py tests/test_tool_search_client_deferral.py tests/test_cache_breakpoint_diagnostics.py tests/test_proxy_cache_ttl_metrics.py tests/test_proxy_byte_faithful_forwarding.py tests/test_memory_tool_mode.py tests/test_codex_ws_compression_scheduler.py tests/test_proxy_savings_history.py tests/test_compression_cache_registry.py tests/test_proxy_extension_ordering.py tests/test_proxy_eager_preload_bind.py tests/test_perf_per_port_logs.py tests/test_cost_cache_prices.py tests/test_openai_tool_search_deferral.py tests/test_proxy_extensions.py tests/test_direct_chat_ccr_resolution.py tests/test_org_scale_limits.py tests/test_savings_ledger.py tests/test_cost_pricing_warning_dedup.py tests/test_proxy/test_compression_timeout_config.py tests/test_log_safety.py   # 64ff3f03
================== 517 passed, 1 skipped in 85.70s (0:01:25) ===================
$ pytest -q (same pricing/extension/regex/log/knob/log_safety files) after merging log-safety-helpers @ 0ce74a63   # 4ff57af4
======================== 139 passed, 1 skipped in 4.23s ========================
$ uvx ruff@0.16.8 check / format --check headroom/proxy/cost.py headroom/proxy/extensions.py tests/test_cost_cache_prices.py tests/test_cost_pricing_warning_dedup.py tests/test_proxy_extensions.py
All checks passed! / 5 files already formatted
$ mypy headroom/proxy/cost.py headroom/proxy/extensions.py --ignore-missing-imports
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, branch checkout at 4ff57af4
(log-safety-helpers merged at 0ce74a63), using the repo .venv. A temp
HOME, workspace and CODEX_HOME were used each time; the real HOME was
not touched.
- Exact command / steps:
1. Ran the shared smoke harness, which starts a real proxy from this
checkout against a local fake upstream and sends one Anthropic and one
OpenAI request.
2. Ran the real `CostTracker.estimate_cost('gpt-4o', 100, 10)` twice,
with every logger at DEBUG and a LiteLLM stand-in whose `cost_per_token`
raises an error quoting `Authorization: Bearer sk-canary-7f3a9c`.
3. The earlier forced-failure run for the knob, log-directory and regex
warnings (d83e7d5c, below) is unchanged by this commit.
- Observed result:
```text
branch: cleanup/proxy-core-a-surface-ignored-config @ 4ff57af4
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>):
(smoke dir: <tmp>)
```
```text
WARNING headroom.proxy: Failed to get pricing for model 'gpt-4o': RuntimeError at <string>:8 in cost_per_token <- headroom/proxy/cost.py:999 in estimate_cost
result: None None
```
- One payload-safe WARNING appears, with the exception type and code
location, and both calls return `None`.
- The canary appears in no output line at any level: `grep -c canary`
over the whole run, LiteLLM's own DEBUG chatter included, gives 0.
  - Earlier forced-failure run:
```text
branch: cleanup/proxy-core-a-surface-ignored-config @ d83e7d5c
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>):
(smoke dir: <tmp>)
```
The harness also printed two pre-existing WARNING lines, which I left
out above: offline-mode air-gap active, and the Kompress model prefetch
refused offline. Neither comes from this PR. Output of the direct
command:
```text
WARNING headroom.proxy: Ignoring invalid HEADROOM_COMPRESSION_TIMEOUT_SECONDS='abc'; using 30.0
WARNING headroom.proxy: Headroom runtime log disabled: cannot write the log file: [Errno 17] File exists: '<tmp>/ws/logs'. Point HEADROOM_WORKSPACE_DIR at a writable directory to restore proxy.log.
WARNING headroom.proxy: Ignoring invalid HEADROOM_OPENAI_TOOL_SEARCH_MODELS regex '[bad' (unterminated character set at position 0); using the gpt-5.4+ version gate
gate: True True
```
- The proxy serves both providers, and the normal path adds no new
warnings.
- Each forced failure gives exactly one warning, even though the regex
gate ran twice. The fallbacks are unchanged: 30.0, and the version gate
is True for gpt-5.4.
- Not tested: A real read-only filesystem, a real broken
`importlib.metadata` and a live pricing-catalog failure (the stand-in
above raises from the same call). Each goes through the same `except`
branch that the unit tests force.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A; log output only.
- Stable/default behavior changed: No. Every fallback value and code
path is the same; only new WARNING lines appear, when something was
already failing.
- Kill switch / disable path: N/A. A pricing warning fires at most once
an hour per (lookup, model), through `WarnOnce`; the regex warning fires
once per bad pattern.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Part of a maintainer-requested codebase cleanup (dead code, duplication,
debuggability), split into small PRs for review. It is part of
maintainer-requested cleanup #4136.

Depends on #4138; its two files show in this diff until it merges. #4137
also edits `_setup_file_logging`, so whichever of the two merges second
rebases.

Docs are unchanged because no knob's meaning changed. The doc impacts
are recorded in the cleanup docs ledger for the follow-up docs PR. The
six import-time knob warnings have no test: testing them would need
`importlib.reload(helpers)`, which would reset the module globals other
tests depend on.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 11:51:06 -07:00
5cb87bc351 fix(cache): default CCR payload previews off; harden the log when they are on (#3728)
## A-6 — tool-result content was written to the runtime log by default
(S2)

### RELEASE NOTE — a default changes, and operators lose log content
they may rely on

`HEADROOM_LOG_PAYLOAD_PREVIEW` **now defaults to off.** Operator-visible
impact:

- Each `event=headroom_retrieve` line in
`~/.headroom/logs/proxy-<port>.log` now carries
`"payload_preview":""` and `"payload_preview_chars":0` instead of up to
4096 characters
of the retrieved payload. `payload_chars` (the byte count) is unchanged,
so anything
measuring retrieval size still works; anything grepping the log for
retrieved *content*
  will find nothing until the operator opts in.
- The opt-in is now **explicit and positive**:
`HEADROOM_LOG_PAYLOAD_PREVIEW=1|true|yes|on`.
Previously *any* value other than `0/false/no/off` — including a typo —
meant "on".
- **On Linux and macOS, every Headroom log file is now created `0600`**
(owner-only) — the
runtime log `~/.headroom/logs/proxy-<port>.log`, each of its rotated
backups
(`proxy-8000.log.1`, …), and the `--log-file` JSONL request log. This is
**unconditional**:
it does not depend on `HEADROOM_LOG_PAYLOAD_PREVIEW`, `--log-messages`,
or any other
switch. **A log shipper, sidecar, or collector that reads these files as
a different user
or group will stop being able to read them.** Run the collector as the
same user, or copy
the file out with a privileged job. On Windows nothing changes — see the
Windows note
  below.
- If the runtime log path already exists as a **symlink**, Headroom now
refuses to write the
log at all (with a warning naming the path) instead of writing through
it. An operator who
deliberately symlinks `~/.headroom/logs/proxy-<port>.log` onto another
volume must replace
  it with a real path or point `HEADROOM_WORKSPACE_DIR` at that volume.

### The bug

`_payload_preview_enabled()` returned `True` for an unset variable, so a
stock proxy wrote
up to 4 KB of the retrieved tool-result payload into the always-on
runtime log on every CCR
retrieval. That payload is verbatim tool output: source code, customer
data, and any
credential the three redaction regexes (`*_KEY|TOKEN|SECRET|...`,
`Bearer`/`Basic`, `sk-…`)
do not happen to match. The file was created at the process umask —
`0644` on a normal
developer machine.

**The claim held**, at `headroom/cache/compression_store.py:59`/`:116`
on current `main`
(line numbers unchanged since `89a58fd1`). Demonstrated by the
fail-before run below, where
the new test found the string `ssn=123-45-6789` sitting in a real log
file, mode `0644`.

### The fix

- `headroom/cache/compression_store.py` — `_payload_preview_enabled()`
defaults to `False`
and now requires an affirmative `1/true/yes/on`. The redaction pass is
untouched; it is
now a second line of defence behind an explicit opt-in rather than the
only one.
- `headroom/fileperms.py` (new) — owner-only file creation, and an
explicit
`OWNER_ONLY_SUPPORTED` flag recording *where the mode bits actually
decide read access*.
- `headroom/proxy/helpers.py` — `_OwnerOnlyRotatingFileHandler`, now
used by
`_setup_file_logging` **unconditionally**. `logging` opens the stream
itself at
construction *and on every rollover* with no mode hook, so `_open` is
replaced outright:
the file is opened `O_CREAT|O_APPEND|O_WRONLY|O_NOFOLLOW` at `0600` and
tightened with
`fchmod` on that same descriptor (so an existing `0644` log is fixed,
and the mode cannot
land on a different inode). `rotate()` re-applies the mode to each
backup, and backups
  left `0644` by an older build are tightened when the handler opens.
- `headroom/proxy/request_logger.py` — the `--log-file` JSONL, which
carries whole request
  and response bodies under `--log-messages`, is created the same way.
- `docs/content/docs/proxy.mdx` — the variable was entirely
undocumented; it now has a row
in **Observability**, plus a **Who can read the logs** subsection
stating the POSIX
  guarantee and the Windows non-guarantee in as many words.

### Fail-before / pass-after

Method: commit the work, `git checkout origin/main --
headroom/cache/compression_store.py
headroom/proxy/helpers.py` keeping the new tests, run, then `git
checkout HEAD -- <those
files>`.

**Before (sources reverted to `origin/main`, new tests kept):**

```
__________ test_runtime_log_holds_no_payload_text_at_default_settings __________
tests/test_cache_breakpoint_diagnostics.py:246: in test_runtime_log_holds_no_payload_text_at_default_settings
    assert secret not in text
E   assert 'BEGIN-CUSTO...USTOMER-DATA' not in '2026-09-23 ...MER-DATA"}\n'
E     'BEGIN-CUSTOMER-DAT...s END-CUSTOMER-DATA' is contained here:
E       preview":"BEGIN-CUSTOMER-DATA ssn=123-45-6789 def sekrit(): pass END-CUSTOMER-DATA"}

_____________ test_runtime_log_is_owner_only_when_preview_enabled ______________
tests/test_cache_breakpoint_diagnostics.py:257: in test_runtime_log_is_owner_only_when_preview_enabled
    assert stat.S_IMODE(log_path.stat().st_mode) == 0o600
E   AssertionError: assert 420 == 384        # 0o644 vs 0o600

____ test_runtime_log_hardening_survives_a_pre_existing_world_readable_log _____
E   AssertionError: assert 420 == 384

FAILED tests/test_cache_breakpoint_diagnostics.py::test_payload_preview_disabled_by_default
FAILED tests/test_cache_breakpoint_diagnostics.py::test_payload_preview_stays_off_for_anything_else[]
FAILED tests/test_cache_breakpoint_diagnostics.py::test_payload_preview_stays_off_for_anything_else[maybe]
FAILED tests/test_cache_breakpoint_diagnostics.py::test_payload_preview_stays_off_for_anything_else[  ]
FAILED tests/test_cache_breakpoint_diagnostics.py::test_runtime_log_holds_no_payload_text_at_default_settings
FAILED tests/test_cache_breakpoint_diagnostics.py::test_runtime_log_is_owner_only_when_preview_enabled
FAILED tests/test_cache_breakpoint_diagnostics.py::test_runtime_log_hardening_survives_a_pre_existing_world_readable_log
========================= 7 failed, 17 passed in 1.59s =========================
```

That first failure is the finding itself: on `main`, customer data lands
in a real log file
under default settings.

**After (sources restored):**

```
tests/test_cache_breakpoint_diagnostics.py ........................      [ 24%]
tests/test_compression_store.py ............................................ [100%]
============================= 100 passed in 9.36s ==============================
```

`tests/test_perf_per_port_logs.py` (10 passed) covers the handler swap —
dedup, per-worker
paths, and sequential port changes still behave.

### Tests

New, in `tests/test_cache_breakpoint_diagnostics.py`:

- `test_payload_preview_disabled_by_default` — unset ⇒ byte counts only.
- `test_payload_preview_opt_in_values` /
`test_payload_preview_stays_off_for_anything_else`
— only an affirmative value enables it; a typo (`maybe`, `""`,
whitespace) does not.
- `test_runtime_log_holds_no_payload_text_at_default_settings` —
end-to-end through the
real `_setup_file_logging` handler: a store+retrieve leaves
`event=headroom_retrieve` in
  the file with the right `payload_chars` and **no** payload text.
- `test_runtime_log_is_owner_only_when_preview_enabled` — `0600` on the
created file.
-
`test_runtime_log_hardening_survives_a_pre_existing_world_readable_log`
— a `0644` log
from a pre-hardening run is chmodded, since `O_CREAT`'s mode does not
apply to it.

Updated: the two tests in `tests/test_compression_store.py` that
exercise the preview and
its redaction now opt in explicitly, and
`test_payload_preview_enabled_by_default` is
inverted to `test_payload_preview_disabled_by_default`.

### Notes

- Independent of #3726 (A-5) — both are based on `origin/main`, and they
touch disjoint
  parts of `compression_store.py`.
- A-5's audit found that the runtime log is one of the files
`--stateless` does *not*
suppress, so before this PR a stateless proxy was getting payload text
on disk too.
- `CHANGELOG.md` not touched (CI guard).


---

## Update after review (@JerrettDavis) — both blockers addressed

### 1. Windows: the guarantee is now scoped in code, not just in prose

You were right that this was a control that silently did nothing on a
supported platform.
On NTFS, read access is decided by the file's ACL; `os.chmod` only
toggles the read-only
attribute, so `chmod(0o600)` returns successfully while `stat.S_IMODE`
still reports
`0666` — which is exactly what your run showed.

I did **not** implement a Windows ACL. Python ships no ACL API; doing it
means ~150 lines
of `ctypes` building a `SECURITY_ATTRIBUTES` DACL, which I cannot
execute or test on this
machine, and I would rather not ship unverifiable security code than
ship an untested one.

So the guarantee is scoped honestly, and made loud rather than silent:

- `headroom/fileperms.OWNER_ONLY_SUPPORTED` is `os.name == "posix"`.
`restrict_fd_to_owner`
/ `restrict_path_to_owner` return `False` off POSIX rather than
reporting a protection
  they did not establish.
- `_setup_file_logging` emits **one warning per process** on such a
platform: *"Headroom
cannot create &lt;path&gt; owner-only on this platform: file modes do
not control read
access here, and Headroom does not set an ACL. The runtime log can
contain request and
response content (`--log-messages`, wire debug,
`HEADROOM_LOG_PAYLOAD_PREVIEW`) — protect
the log directory itself."* Logging still works; it just no longer
implies a protection.
- The docs say it in as many words, under **Who can read the logs**: on
Windows the log
inherits the ACL of its parent directory, and the directory is the
access-control
  boundary.
- The tests assert the real platform-conditional behaviour. The mode
assertions carry
`@_posix_only` with that reason;
`test_owner_only_support_matches_what_the_platform_can_enforce`
  asserts the flag equals `os.name == "posix"` **on every platform**;
`test_no_owner_only_claim_is_made_off_posix` runs *only* on Windows and
asserts the
  absence of the claim (the handler still creates the file;
  `restrict_path_to_owner` returns `False`); and
`test_unsupported_platform_says_so_rather_than_silently_not_protecting`
simulates the
unsupported platform on any host so the warning itself is covered by
Linux CI.

**In one line: on Linux and macOS the log is `0600`, enforced. On
Windows Headroom does not
restrict the log, says so at startup and in the docs, and no test claims
otherwise.**

### 2. The permissions no longer key off `HEADROOM_LOG_PAYLOAD_PREVIEW`

Agreed, and taken as stated: **every** Headroom runtime log is created
owner-only, rollover
files included, regardless of which content switch is on. Previews were
never the only
source of request content in that file — `--log-messages` bodies,
wire-debug dumps and
query logging land there too, each behind its own switch, so gating on
any one of them left
the rest writing a sensitive file at the umask. `_setup_file_logging`
now always installs
`_OwnerOnlyRotatingFileHandler` and no longer imports
`_payload_preview_enabled` at all.
I extended it to the `--log-file` JSONL as well, since under
`--log-messages` that is the
file with the whole bodies in it.

Previews still default off — that part of the PR is unchanged.

This **does** break log collection for an operator reading the file as
another user, which
is the trade-off the original PR body ducked. It is now called out
explicitly in the release
note at the top, with the two workarounds.

### Also fixed while in here (found in an internal pass)

- **Rotated backups.** `rotate()` re-applies `0600` to the destination.
`os.rename` happens
to carry the mode across, but a configured `rotator` need not, so the
invariant is
asserted on the result rather than assumed from how it got there.
Backups left `0644` by
  an older build are tightened when the handler opens.
- **`_open` followed symlinks.** The old code pre-created the file and
then delegated to
`super()._open()`, so both the `chmod` and the write followed a symlink
planted at the log
path — an attacker chose the destination *and* the permissions. `_open`
is now replaced
outright and opens with `O_NOFOLLOW` at `0600`, tightening with `fchmod`
on that same
descriptor (no TOCTOU window, and no way for the mode to land on another
inode). A log
path that is *already* a symlink is refused up front with a warning,
which is what carries
the refusal on platforms without `O_NOFOLLOW`. Deliberate decision:
**fail closed** — a
redirected sensitive log is worse than no log, and logging is already
best-effort here.

### Fail-before / pass-after for this round

Baseline is `ec014d64` (the commit you reviewed), not `origin/main`,
since these are new
findings on top of it. Method: commit, `git checkout ec014d64 --
headroom/proxy/helpers.py
headroom/proxy/request_logger.py headroom/cache/compression_store.py`
keeping the new
tests, run, restore.

```
FAILED test_runtime_log_is_owner_only_with_previews_off                       assert 420 == 384
FAILED test_runtime_log_hardening_survives_a_pre_existing_world_readable_log   assert 420 == 384
FAILED test_rotated_backups_are_owner_only                                     assert 420 == 384
FAILED test_pre_existing_backups_are_tightened_when_the_handler_opens          assert 420 == 384
FAILED test_runtime_log_refuses_a_symlinked_path       AssertionError: logging was wired up through the symlink
FAILED test_jsonl_request_log_is_owner_only                                    assert 420 == 384
FAILED test_unsupported_platform_says_so_rather_than_silently_not_protecting
    AttributeError: module 'headroom.proxy.helpers' has no attribute '_owner_only_warning_emitted'
=================== 7 failed, 25 passed, 1 skipped in 1.23s ====================
```

(`420` is `0o644`, `384` is `0o600`. A `predictable_umask` fixture pins
the umask to `0o022`
so "would have been world-readable" is not down to the runner's
environment.)

After, sources restored:

```
tests/test_cache_breakpoint_diagnostics.py tests/test_compression_store.py
tests/test_perf_per_port_logs.py tests/test_proxy/test_request_logger.py
tests/test_proxy_streaming_request_logger.py
======================= 132 passed, 1 skipped in 10.90s ========================
```

The 1 skip is `test_no_owner_only_claim_is_made_off_posix` — it is the
Windows-only half,
and it is the one I cannot run here.

Ruff 0.16.7 `check` and `format --check` are clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-09-24 13:09:06 +02:00
gglucass def3d76e5a fix(cache): mirror client cache_control positions instead of single-marker consolidation
Preserve client cache-control breakpoint positions.
2026-08-11 18:15:44 -07:00