mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description CodeQL had 43 open alerts on `headroom` (19 high, 24 medium). We need 0 for SOC 2 (risk R-10: developers introduce vulnerabilities). This PR fixes the 23 that are real problems in code we ship or run. The 16 in `tests/` are handled by #4047. 11 were dismissed in GitHub with written reasons: 7 stale alerts from the retired default-setup scan, and 4 false positives (#21, #105, #109, #135). **In plain words:** 1. **Error replies no longer include internal error text.** When a request body couldn't be read, the proxy sent back Python's raw error message. It now sends a fixed message that still says what kind of problem it was. 2. **A benchmark no longer logs a possible API key.** `--model_args` is shown as `<redacted>` in the log, but `lm_eval` still receives the real value. 3. **A test no longer prints part of the Gemini API key.** 4. **Five workflows declare read-only permissions**, and CodeQL scans GitHub Actions workflows again. 5. **The npm release scripts no longer pass arguments through Windows `cmd.exe`**, which reinterprets `&` and `|` inside arguments, file paths included. Closes CodeQL alerts #20, #23, #24, #25, #26, #27, #28, #29, #31, #33, #193 (stack-trace exposure), #177, #51, #145, #146, #147. The workflow-permission fixes correspond to the dismissed #2, #7, #67, #85, #86, #87, #125. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/helpers.py`: - New `invalid_request_body_message(exc)` returns a fixed string per error type and logs the type: - `…: malformed JSON` - `…: not valid UTF-8 (possibly compressed?)` - `…: must be a JSON object` - `…: too large` - plain `Invalid request body` for anything else - New `RequestBodyNotObject(ValueError)` replaces a bare `ValueError`, so every existing `except ValueError` still catches it. - `headroom/proxy/handlers/{anthropic,openai,gemini,batch}.py`: - All 10 `f"Invalid request body: {e!s}"` replies now use the helper. - The Google batch forwarder's 500 reply drops the exception text; it was already logged on the line above. - `headroom/evals/comprehensive_benchmark.py`: `--model_args` is appended after the command is logged, and the log line shows it as `<redacted>`. - `tests/test_google_multimodal_e2e.py`: stops printing the first 10 characters of the API key. - `.github/workflows/{devcontainers,eval,init-e2e,init-native-e2e,wrap-e2e}.yml`: top-level `permissions: contents: read`. None of these jobs write to the repo; caching and artifact upload don't use the repo token. - `.github/workflows/security.yml`: CodeQL matrix adds `actions`. - `scripts/{build,verify}_npm_release_assets.mjs`: - On Windows, npm runs as `node <npm-cli.js> …` instead of `cmd.exe /c npm.cmd …`. - `npm-cli.js` is found via `npm_execpath`, then next to node, then beside each `npm.cmd` on PATH. This handles node and npm in different directories, per Devin's review. - If none is found, it fails listing every path searched. - New tests: - `tests/test_invalid_request_body_message.py` - `tests/test_evals/test_comprehensive_benchmark_log_redaction.py`, per Devanshi's review. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`): one error remains, the same one `main` has, in an untouched file. See notes. - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest -q tests/test_release_workflows.py tests/test_invalid_request_body_message.py \ tests/test_evals/test_comprehensive_benchmark_log_redaction.py ============================== 62 passed in 3.10s ============================== # The redaction test against the pre-fix comprehensive_benchmark.py (origin/main): ========================= 1 failed, 1 passed in 0.50s ========================== # With the fix: ============================== 2 passed in 0.57s =============================== $ pytest -q (proxy/handler/batch/eval/request-body files, run one file at a time) 187 files: 182 OK; the 5 below fail or hang identically on unchanged origin/main: tests/test_proxy_uds.py 11 failed (macOS 104-byte sun_path limit) tests/test_anthropic_buffered_ccr_usage.py 6 failed tests/test_proxy_copilot_auth_hooks.py 2 failed tests/gateway/test_parity_proxy_vs_gateway.py 2 failed tests/test_cli_proxy_improvements.py hangs >300s $ ruff check . -q -> clean $ ruff format --check headroom tests -> clean $ mypy headroom headroom/integrations/langchain/chat_model.py:394: error: Function "RunnableBinding" could always be true in boolean context [truthy-function] Found 1 error in 1 file (checked 566 source files) # identical on origin/main $ node --check scripts/build_npm_release_assets.mjs && node --check scripts/verify_npm_release_assets.mjs -> ok ``` CI: the `test (3)` failures (`tests/test_onnx_dependency_contract.py`, 5 tests) and `test (4)` failures (`tests/test_xai_cache_economics.py`, 5 tests) also fail on `main`'s latest CI run (run 37806990929), so they're unrelated to this PR. ## Real Behavior Proof - Environment: macOS 26 (arm64), Python 3.12.13, Node 24, npm 11.19.0. Worktree of `origin/main` at87e05b68plus this branch. - Exact command / steps: the three checks below. 1. **Error replies:** `invalid_request_body_message` was exercised with each exception type, including a `JSONDecodeError` whose document is `"secret-doc"`. Each case asserts the reply names the category and contains neither the exception text nor the document. 2. **Benchmark:** `run_lm_eval(model_args="model=gpt-4o-mini,api_key=sk-test-FAKE-…")` was called with the subprocess call mocked. The log shows `--model_args <redacted>` and not the fake key, while the mocked call received `model=gpt-4o-mini,api_key=sk-test-FAKE-…,base_url=http://localhost:8787/v1`. 3. **Windows npm lookup:** `findNpmCli()` was extracted from each script and run against a temporary layout reproducing Devin's case: node shim in `dev/bin`, `npm.cmd` plus `node_modules/npm/bin/npm-cli.js` in `Program Files/nodejs`, both on PATH. - Observed result: all three Windows npm lookup cases pass in both scripts (output below); checks 1 and 2 pass as shown in Test Output. ```text == scripts/build_npm_release_assets.mjs separate node/npm dirs -> found npm-cli.js beside npm.cmd on PATH npm_execpath set -> uses npm_execpath missing npm -> clear error: true == scripts/verify_npm_release_assets.mjs separate node/npm dirs -> found npm-cli.js beside npm.cmd on PATH npm_execpath set -> uses npm_execpath missing npm -> clear error: true ``` - Not tested: the two items below. - A real Windows run of the release scripts. No Windows machine was available, and CI runs these scripts on Ubuntu, where the code path is unchanged (`npm` spawned directly, no shell). - The new CodeQL `actions` scan. It runs on this PR's CI and on `main` after merge. ### Maintainer verification at `128f271f` (2026-10-08) - Environment: actual Windows 11, Python 3.12.13, Node 22.22.0, npm 10.9.4. Contributor logic unchanged; this head contains current `main`. - Commands: the focused `test_invalid_request_body_message.py`, `test_comprehensive_benchmark_log_redaction.py`, and `test_release_workflows.py` suites; the existing Windows npm lookup probe against both checked-in scripts; and a two-process-boundary loopback uvicorn/httpx malformed-body probe against this head. - Observed: **62 focused tests passed**. Both npm scripts invoked the real npm CLI under Node with `shell:false`, including Node copied into a separate `G:/tmp/headroom-review/resume49-node shim & literal` directory while npm remained under `C:/Program Files/nodejs`; `npm_execpath` was unset, so PATH lookup found the separate npm installation. npm reported 10.9.4 and a cache argument containing spaces and `&` was preserved literally. **36 actual TCP requests** (four malformed-body categories across nine endpoints) returned the expected fixed 400 messages, with no private marker disclosed and no upstream requests. Only outbound provider requests were replaced with a fail-if-called boundary; HTTP parsing and proxy handlers were real. - Hosted qualification at this exact head: **67 passed, 11 skipped**, including all three CodeQL languages. Evidence is retained in the maintainer review artifacts. The earlier simulated Windows-layout proof remains below as historical author evidence. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: n/a. - Stable/default behavior changed: yes, the text of 400 replies for unreadable request bodies. It was `Invalid request body: <python exception text>` and is now a fixed category message. Status codes, error `type`/`code` fields and response shape are unchanged. Clients that parse the free-text message, rather than the status and type, would see different text. - Kill switch / disable path: none needed; this is message text only. - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (none needed) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Review feedback addressed in921de19b: - Devin: Windows npm lookup when node and npm are in different directories. - Devanshi: benchmark redaction test. - **#21 (`security_block` reply in `anthropic.py`) is left as is.** It returns the security plugin's deliberate, user-facing reason for blocking a request. It was dismissed as a false positive with that reason. - The `mypy` error is pre-existing: the same `truthy-function` error appears on `origin/main`, in a file this PR doesn't touch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
229 lines
7.0 KiB
JavaScript
229 lines
7.0 KiB
JavaScript
#!/usr/bin/env node
|
|
|
|
import {
|
|
existsSync,
|
|
mkdirSync,
|
|
readFileSync,
|
|
readdirSync,
|
|
rmSync,
|
|
writeFileSync,
|
|
} from "node:fs";
|
|
import path from "node:path";
|
|
import { spawnSync } from "node:child_process";
|
|
import { fileURLToPath } from "node:url";
|
|
|
|
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
|
const rootDir = path.resolve(__dirname, "..");
|
|
const sdkDir = path.join(rootDir, "sdk", "typescript");
|
|
const openClawDir = path.join(rootDir, "plugins", "openclaw");
|
|
|
|
const rawArgs = process.argv.slice(2);
|
|
const flags = new Set(rawArgs.filter((arg) => arg.startsWith("--")));
|
|
const positional = rawArgs.filter((arg) => !arg.startsWith("--"));
|
|
const [version, assetsDirArg] = positional;
|
|
|
|
if (!version || flags.has("--help") || flags.has("-h")) {
|
|
console.error(
|
|
[
|
|
"Usage: node scripts/build_npm_release_assets.mjs <version> [assets-dir] [--skip-install] [--no-verify]",
|
|
"",
|
|
"Builds the TypeScript SDK and OpenClaw npm release tarballs, rewrites",
|
|
"OpenClaw release metadata to depend on the just-built SDK version,",
|
|
"regenerates dist/package.json, and verifies the resulting assets.",
|
|
].join("\n"),
|
|
);
|
|
process.exit(flags.has("--help") || flags.has("-h") ? 0 : 2);
|
|
}
|
|
|
|
if (!/^\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?$/.test(version)) {
|
|
console.error(`Invalid version: ${version}`);
|
|
process.exit(2);
|
|
}
|
|
|
|
const timestamp = new Date().toISOString().replace(/\D/g, "").slice(0, 14);
|
|
const assetsDir = path.resolve(
|
|
rootDir,
|
|
assetsDirArg || path.join("release-assets-local", `${version}-${timestamp}`),
|
|
);
|
|
|
|
const trackedFiles = [
|
|
path.join(sdkDir, "package.json"),
|
|
path.join(sdkDir, "package-lock.json"),
|
|
path.join(openClawDir, "package.json"),
|
|
path.join(openClawDir, "package-lock.json"),
|
|
path.join(openClawDir, "dist", "package.json"),
|
|
];
|
|
|
|
const snapshots = new Map(
|
|
trackedFiles.map((filePath) => [
|
|
filePath,
|
|
existsSync(filePath) ? readFileSync(filePath, "utf8") : null,
|
|
]),
|
|
);
|
|
|
|
function quoteCmdArg(value) {
|
|
const arg = String(value);
|
|
if (/^[A-Za-z0-9_./:=\\-]+$/.test(arg)) {
|
|
return arg;
|
|
}
|
|
return `"${arg.replace(/"/g, '""')}"`;
|
|
}
|
|
|
|
function run(command, args, cwd) {
|
|
console.log(`\n> ${command} ${args.map(quoteCmdArg).join(" ")}`);
|
|
const result = spawnSync(command, args, {
|
|
cwd,
|
|
encoding: "utf8",
|
|
stdio: "inherit",
|
|
});
|
|
|
|
if (result.error) {
|
|
throw new Error(`${command} failed: ${result.error.message}`);
|
|
}
|
|
|
|
if (result.status !== 0) {
|
|
throw new Error(`${command} failed with exit code ${result.status ?? "unknown"}`);
|
|
}
|
|
}
|
|
|
|
// npm on Windows is a .cmd shim, which only runs through cmd.exe, and cmd.exe
|
|
// reinterprets characters such as & and | inside arguments (paths included).
|
|
// Run npm's own CLI script under this Node binary instead, so no shell is
|
|
// involved on any platform.
|
|
function npmCommand(args) {
|
|
if (process.platform !== "win32") {
|
|
return ["npm", args];
|
|
}
|
|
return [process.execPath, [findNpmCli(), ...args]];
|
|
}
|
|
|
|
// Node and npm can live in different directories (a node shim, or npm
|
|
// upgraded into the global prefix), so look where npm itself says it is, then
|
|
// next to node, then next to every npm.cmd on PATH. Every npm.cmd install
|
|
// keeps its CLI at node_modules/npm/bin/npm-cli.js beside the shim.
|
|
function findNpmCli() {
|
|
const cliPath = path.join("node_modules", "npm", "bin", "npm-cli.js");
|
|
const candidates = [];
|
|
if (process.env.npm_execpath?.endsWith("npm-cli.js")) {
|
|
candidates.push(process.env.npm_execpath);
|
|
}
|
|
candidates.push(path.join(path.dirname(process.execPath), cliPath));
|
|
for (const dir of (process.env.PATH || "").split(path.delimiter)) {
|
|
if (dir && existsSync(path.join(dir, "npm.cmd"))) {
|
|
candidates.push(path.join(dir, cliPath));
|
|
}
|
|
}
|
|
const found = candidates.find((candidate) => existsSync(candidate));
|
|
if (!found) {
|
|
throw new Error(`npm CLI (npm-cli.js) not found. Looked in: ${candidates.join(", ")}`);
|
|
}
|
|
return found;
|
|
}
|
|
|
|
function runNpm(args, cwd) {
|
|
const [command, commandArgs] = npmCommand(args);
|
|
run(command, commandArgs, cwd);
|
|
}
|
|
|
|
function runNode(args, cwd) {
|
|
run(process.execPath, args, cwd);
|
|
}
|
|
|
|
function readJson(filePath) {
|
|
return JSON.parse(readFileSync(filePath, "utf8"));
|
|
}
|
|
|
|
function writeJson(filePath, data) {
|
|
writeFileSync(filePath, `${JSON.stringify(data, null, 2)}\n`, "utf8");
|
|
}
|
|
|
|
function ensureEmptyAssetsDir() {
|
|
mkdirSync(assetsDir, { recursive: true });
|
|
const existing = readdirSync(assetsDir);
|
|
if (existing.length > 0) {
|
|
throw new Error(
|
|
`Assets directory must be empty to avoid stale tarballs: ${assetsDir}`,
|
|
);
|
|
}
|
|
}
|
|
|
|
function restoreTrackedFiles() {
|
|
for (const [filePath, contents] of snapshots.entries()) {
|
|
if (contents === null) {
|
|
rmSync(filePath, { force: true });
|
|
} else {
|
|
mkdirSync(path.dirname(filePath), { recursive: true });
|
|
writeFileSync(filePath, contents, "utf8");
|
|
}
|
|
}
|
|
}
|
|
|
|
function relativeFileSpec(fromDir, targetPath) {
|
|
let relativePath = path.relative(fromDir, targetPath).split(path.sep).join("/");
|
|
if (!relativePath.startsWith(".")) {
|
|
relativePath = `./${relativePath}`;
|
|
}
|
|
return `file:${relativePath}`;
|
|
}
|
|
|
|
function rewriteOpenClawDependency(spec) {
|
|
const packageJsonPath = path.join(openClawDir, "package.json");
|
|
const pkg = readJson(packageJsonPath);
|
|
pkg.dependencies = pkg.dependencies || {};
|
|
pkg.dependencies["headroom-ai"] = spec;
|
|
writeJson(packageJsonPath, pkg);
|
|
}
|
|
|
|
function rewriteOpenClawLocalDependency(sdkTarballPath) {
|
|
rewriteOpenClawDependency(relativeFileSpec(openClawDir, sdkTarballPath));
|
|
}
|
|
|
|
function rewriteOpenClawReleaseDependency() {
|
|
rewriteOpenClawDependency(`^${version}`);
|
|
}
|
|
|
|
function assertTarballBuilt(name) {
|
|
const tarballPath = path.join(assetsDir, `${name}-${version}.tgz`);
|
|
if (!existsSync(tarballPath)) {
|
|
throw new Error(`Expected npm pack to produce ${tarballPath}`);
|
|
}
|
|
return tarballPath;
|
|
}
|
|
|
|
try {
|
|
ensureEmptyAssetsDir();
|
|
|
|
if (!flags.has("--skip-install")) {
|
|
runNpm(["ci"], sdkDir);
|
|
}
|
|
runNpm(["run", "build"], sdkDir);
|
|
runNpm(["version", version, "--no-git-tag-version", "--allow-same-version"], sdkDir);
|
|
runNpm(["pack", "--pack-destination", assetsDir], sdkDir);
|
|
const sdkTarballPath = assertTarballBuilt("headroom-ai");
|
|
|
|
rewriteOpenClawLocalDependency(sdkTarballPath);
|
|
if (!flags.has("--skip-install")) {
|
|
runNpm(
|
|
["install", "--package-lock=false", "--no-audit", "--no-fund", "--ignore-scripts"],
|
|
openClawDir,
|
|
);
|
|
} else {
|
|
runNpm(["install", "--no-save", "--package-lock=false", sdkTarballPath], openClawDir);
|
|
}
|
|
runNpm(["run", "build"], openClawDir);
|
|
runNpm(["version", version, "--no-git-tag-version", "--allow-same-version"], openClawDir);
|
|
rewriteOpenClawReleaseDependency();
|
|
runNode(["prepare-dist.mjs"], openClawDir);
|
|
runNpm(["pack", "--pack-destination", assetsDir], openClawDir);
|
|
assertTarballBuilt("headroom-openclaw");
|
|
|
|
if (!flags.has("--no-verify")) {
|
|
runNode(["scripts/verify_npm_release_assets.mjs", assetsDir, version], rootDir);
|
|
}
|
|
|
|
console.log(`\nBuilt and verified npm release assets in ${assetsDir}`);
|
|
} finally {
|
|
restoreTrackedFiles();
|
|
runNode(["prepare-dist.mjs"], openClawDir);
|
|
}
|