Files
Tejas ChopraandClaude Opus 5.5 a88e2d5145 fix(security): resolve 23 open CodeQL alerts (#4045)
## Description

CodeQL had 43 open alerts on `headroom` (19 high, 24 medium). We need 0
for SOC 2 (risk R-10: developers introduce vulnerabilities). This PR
fixes the 23 that are real problems in code we ship or run. The 16 in
`tests/` are handled by #4047. 11 were dismissed in GitHub with written
reasons: 7 stale alerts from the retired default-setup scan, and 4 false
positives (#21, #105, #109, #135).

**In plain words:**
1. **Error replies no longer include internal error text.** When a
request body couldn't be read, the proxy sent back Python's raw error
message. It now sends a fixed message that still says what kind of
problem it was.
2. **A benchmark no longer logs a possible API key.** `--model_args` is
shown as `<redacted>` in the log, but `lm_eval` still receives the real
value.
3. **A test no longer prints part of the Gemini API key.**
4. **Five workflows declare read-only permissions**, and CodeQL scans
GitHub Actions workflows again.
5. **The npm release scripts no longer pass arguments through Windows
`cmd.exe`**, which reinterprets `&` and `|` inside arguments, file paths
included.

Closes CodeQL alerts #20, #23, #24, #25, #26, #27, #28, #29, #31, #33,
#193 (stack-trace exposure), #177, #51, #145, #146, #147. The
workflow-permission fixes correspond to the dismissed #2, #7, #67, #85,
#86, #87, #125.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`:
- New `invalid_request_body_message(exc)` returns a fixed string per
error type and logs the type:
    - `…: malformed JSON`
    - `…: not valid UTF-8 (possibly compressed?)`
    - `…: must be a JSON object`
    - `…: too large`
    - plain `Invalid request body` for anything else
- New `RequestBodyNotObject(ValueError)` replaces a bare `ValueError`,
so every existing `except ValueError` still catches it.
- `headroom/proxy/handlers/{anthropic,openai,gemini,batch}.py`:
  - All 10 `f"Invalid request body: {e!s}"` replies now use the helper.
- The Google batch forwarder's 500 reply drops the exception text; it
was already logged on the line above.
- `headroom/evals/comprehensive_benchmark.py`: `--model_args` is
appended after the command is logged, and the log line shows it as
`<redacted>`.
- `tests/test_google_multimodal_e2e.py`: stops printing the first 10
characters of the API key.
-
`.github/workflows/{devcontainers,eval,init-e2e,init-native-e2e,wrap-e2e}.yml`:
top-level `permissions: contents: read`. None of these jobs write to the
repo; caching and artifact upload don't use the repo token.
- `.github/workflows/security.yml`: CodeQL matrix adds `actions`.
- `scripts/{build,verify}_npm_release_assets.mjs`:
- On Windows, npm runs as `node <npm-cli.js> …` instead of `cmd.exe /c
npm.cmd …`.
- `npm-cli.js` is found via `npm_execpath`, then next to node, then
beside each `npm.cmd` on PATH. This handles node and npm in different
directories, per Devin's review.
  - If none is found, it fails listing every path searched.
- New tests:
  - `tests/test_invalid_request_body_message.py`
- `tests/test_evals/test_comprehensive_benchmark_log_redaction.py`, per
Devanshi's review.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`): one error remains, the
same one `main` has, in an untouched file. See notes.
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_release_workflows.py tests/test_invalid_request_body_message.py \
    tests/test_evals/test_comprehensive_benchmark_log_redaction.py
============================== 62 passed in 3.10s ==============================

# The redaction test against the pre-fix comprehensive_benchmark.py (origin/main):
========================= 1 failed, 1 passed in 0.50s ==========================
# With the fix:
============================== 2 passed in 0.57s ===============================

$ pytest -q (proxy/handler/batch/eval/request-body files, run one file at a time)
187 files: 182 OK; the 5 below fail or hang identically on unchanged origin/main:
  tests/test_proxy_uds.py                          11 failed  (macOS 104-byte sun_path limit)
  tests/test_anthropic_buffered_ccr_usage.py        6 failed
  tests/test_proxy_copilot_auth_hooks.py            2 failed
  tests/gateway/test_parity_proxy_vs_gateway.py     2 failed
  tests/test_cli_proxy_improvements.py             hangs >300s

$ ruff check . -q                      -> clean
$ ruff format --check headroom tests   -> clean
$ mypy headroom
headroom/integrations/langchain/chat_model.py:394: error: Function "RunnableBinding" could always be true in boolean context  [truthy-function]
Found 1 error in 1 file (checked 566 source files)   # identical on origin/main

$ node --check scripts/build_npm_release_assets.mjs && node --check scripts/verify_npm_release_assets.mjs  -> ok
```

CI: the `test (3)` failures (`tests/test_onnx_dependency_contract.py`, 5
tests) and `test (4)` failures (`tests/test_xai_cache_economics.py`, 5
tests) also fail on `main`'s latest CI run (run 37806990929), so they're
unrelated to this PR.

## Real Behavior Proof

- Environment: macOS 26 (arm64), Python 3.12.13, Node 24, npm 11.19.0.
Worktree of `origin/main` at 87e05b68 plus this branch.
- Exact command / steps: the three checks below.
1. **Error replies:** `invalid_request_body_message` was exercised with
each exception type, including a `JSONDecodeError` whose document is
`"secret-doc"`. Each case asserts the reply names the category and
contains neither the exception text nor the document.
2. **Benchmark:**
`run_lm_eval(model_args="model=gpt-4o-mini,api_key=sk-test-FAKE-…")` was
called with the subprocess call mocked. The log shows `--model_args
<redacted>` and not the fake key, while the mocked call received
`model=gpt-4o-mini,api_key=sk-test-FAKE-…,base_url=http://localhost:8787/v1`.
3. **Windows npm lookup:** `findNpmCli()` was extracted from each script
and run against a temporary layout reproducing Devin's case: node shim
in `dev/bin`, `npm.cmd` plus `node_modules/npm/bin/npm-cli.js` in
`Program Files/nodejs`, both on PATH.
- Observed result: all three Windows npm lookup cases pass in both
scripts (output below); checks 1 and 2 pass as shown in Test Output.
  ```text
  == scripts/build_npm_release_assets.mjs
  separate node/npm dirs -> found npm-cli.js beside npm.cmd on PATH
  npm_execpath set       -> uses npm_execpath
  missing npm            -> clear error: true
  == scripts/verify_npm_release_assets.mjs
  separate node/npm dirs -> found npm-cli.js beside npm.cmd on PATH
  npm_execpath set       -> uses npm_execpath
  missing npm            -> clear error: true
  ```
- Not tested: the two items below.
- A real Windows run of the release scripts. No Windows machine was
available, and CI runs these scripts on Ubuntu, where the code path is
unchanged (`npm` spawned directly, no shell).
- The new CodeQL `actions` scan. It runs on this PR's CI and on `main`
after merge.

### Maintainer verification at `128f271f` (2026-10-08)

- Environment: actual Windows 11, Python 3.12.13, Node 22.22.0, npm
10.9.4. Contributor logic unchanged; this head contains current `main`.
- Commands: the focused `test_invalid_request_body_message.py`,
`test_comprehensive_benchmark_log_redaction.py`, and
`test_release_workflows.py` suites; the existing Windows npm lookup
probe against both checked-in scripts; and a two-process-boundary
loopback uvicorn/httpx malformed-body probe against this head.
- Observed: **62 focused tests passed**. Both npm scripts invoked the
real npm CLI under Node with `shell:false`, including Node copied into a
separate `G:/tmp/headroom-review/resume49-node shim & literal` directory
while npm remained under `C:/Program Files/nodejs`; `npm_execpath` was
unset, so PATH lookup found the separate npm installation. npm reported
10.9.4 and a cache argument containing spaces and `&` was preserved
literally. **36 actual TCP requests** (four malformed-body categories
across nine endpoints) returned the expected fixed 400 messages, with no
private marker disclosed and no upstream requests. Only outbound
provider requests were replaced with a fail-if-called boundary; HTTP
parsing and proxy handlers were real.
- Hosted qualification at this exact head: **67 passed, 11 skipped**,
including all three CodeQL languages. Evidence is retained in the
maintainer review artifacts. The earlier simulated Windows-layout proof
remains below as historical author evidence.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: yes, the text of 400 replies for
unreadable request bodies. It was `Invalid request body: <python
exception text>` and is now a fixed category message. Status codes,
error `type`/`code` fields and response shape are unchanged. Clients
that parse the free-text message, rather than the status and type, would
see different text.
- Kill switch / disable path: none needed; this is message text only.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (none
needed)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- Review feedback addressed in 921de19b: 
- Devin: Windows npm lookup when node and npm are in different
directories.
  - Devanshi: benchmark redaction test.
- **#21 (`security_block` reply in `anthropic.py`) is left as is.** It
returns the security plugin's deliberate, user-facing reason for
blocking a request. It was dismissed as a false positive with that
reason.
- The `mypy` error is pre-existing: the same `truthy-function` error
appears on `origin/main`, in a file this PR doesn't touch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:40:04 -05:00
..