Commit Graph
40 Commits
Author SHA1 Message Date
123XieCand123XieC d077263012 fix(openclaw): preserve assistant text blocks when tool arguments change (#4829)
## Description

OpenClaw history can contain text on both sides of an assistant tool
call. If the compression response changes the call arguments while
leaving the wire text intact, restoring that message currently moves
both pieces of text ahead of the call and joins them into a single
block.

## Type of Change

- [x] Bug fix

## Changes Made

- Compare the returned assistant text with the text sent for that
message. Keep the original blocks when the text matches, then apply the
changed arguments to the matching tool call.
- Extend the assemble regression to check the order and content of both
text blocks, the signed thinking block, and the updated tool call.

## Testing

- [x] Regression and existing OpenClaw tests pass.
- [x] Manual testing performed.

### Test Output

```text
cd plugins/openclaw
npm test          # 11 files, 162 tests passed
npm run typecheck # passed
npm run build     # passed
git diff --check  # passed
```

The repository commit hooks and Rust formatting check passed. The full
local pre-push suite stopped at Rust dependency resolution after the
registry fetch stalled. I used the hook's documented push override for
this TypeScript-only change. The verification above covers the changed
plugin.

## Real Behavior Proof

- Environment: macOS arm64, Node 22.22.2, built OpenClaw plugin with the
pinned JavaScript SDK, and a Node HTTP server listening on loopback. The
payload was synthetic and did not need a model.
- Exact command / steps: Run `npm run build` in `plugins/openclaw` on
base `9f2cc01` and the patched tree. In Node, import
`HeadroomContextEngine` from the built `dist/index.js`. Start a loopback
server that answers `GET /v1/retrieve/stats` with `{"store":{}}`. For
`POST /v1/compress`, echo the submitted messages and set the first tool
call's argument string to `{"id":7}`. Return `tokens_before: 100`,
`tokens_after: 25`, and `tokens_saved: 75`. Create the engine with that
`proxyUrl`, `autoStart: false`, and `minContextChars: 0`. Await
`ensureProxyUrl()`, then call `assemble()` with one assistant message
whose blocks are signed thinking, text `Checking.`, a `lookup` call with
ID `lookup-1` and arguments `{id: 7, notes: "fixture ".repeat(80)}`, and
text `Awaiting result.`.
- Observed result: the SDK sent one actual compression request in both
runs. Base returned `thinking, text("Checking.Awaiting result."),
toolCall({id:7})`. The patch returned `thinking, text("Checking."),
toolCall({id:7}), text("Awaiting result.")`. The signature and call ID
were preserved, and the engine reported 25 tokens in both runs.
- Not tested: a remote proxy, model calls, or an OpenClaw host process.
The runtime proof exercised the built plugin and SDK over HTTP with a
deterministic compression response.

## Runtime Rollout Safety

- Rollout-managed feature(s): OpenClaw context restoration.
- Minimum rollout channel: existing plugin channel.
- Stable/default behavior changed: text block placement is preserved
when only assistant tool arguments change.
- Kill switch / disable path: existing OpenClaw plugin enablement.
- Unsafe override required: none.
- Qualification impact: OpenClaw plugin suite and loopback verification
above.
- Rollback path: revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

This is a small correction to the restoration path introduced by #3826.
AI assistance was used to investigate, implement, and verify the change.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4829"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: 123XieC <323077714+123XieC@users.noreply.github.com>
2026-11-16 23:05:24 -06:00
Parideboy 1372f95524 fix(openclaw): let the circuit breaker open on proxy failures (#4755)
## Description

`assemble()` in the openclaw plugin passed `fallback: true` to the SDK's
`compress()`. In `sdk/typescript/src/client.ts`, that flag makes a
connection error, a timeout or a 5xx retry once and then **resolve** to
`makeFallbackResult(messages)` — `tokensBefore: 0`, `tokensSaved: 0`,
`compressed: false` — instead of throwing.

That result never reached the `catch` that calls `tripCircuit()`. Worse,
`assemble()` treated it as a successful no-op and called
`resetCircuit()`, so the error counter went back to zero on every turn.
Against a proxy that accepts connections but never answers, the circuit
breaker could never open: every `assemble()` paid up to `2 ×
requestTimeoutMs` and then bypassed compression anyway.

Closes #4738

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Pass `fallback: false` on the plugin's `compress()` call, so proxy
failures throw into the existing `catch` → `tripCircuit()`. Graceful
degradation is unchanged: that `catch` already returns the original
messages.
- Treat a zero `tokensBefore` on **nonempty** input as a failure too, so
a result that resolved empty cannot keep the breaker closed either.
Empty input is exempt, because the proxy legitimately answers
`tokens_before: 0` for an empty `messages` array — caught in review, see
the `minContextChars: 0` case below.
- `README.md`: the post-cool-down turn does not call `/health`; nothing
in the engine ever did. It re-probes by attempting a real compression.
Fixed, and the threshold row now says which failures count.

## Testing

- [ ] Unit tests pass (`pytest`) — N/A, this is the TypeScript plugin;
ran `vitest` instead
- [ ] Linting passes (`ruff check .`) — N/A, no Python changed
- [ ] Type checking passes (`mypy headroom`) — N/A, no Python changed
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ npx vitest run test/engine.test.ts
 Test Files  1 passed (1)
      Tests  41 passed (41)

$ npm run typecheck
> node node_modules/@typescript/native/bin/tsc --noEmit
(clean)

$ npm run build
ESM Build success in 18ms
DTS Build success in 515ms
```

The three new tests each fail on the unpatched tree (verified by
stashing only `src/engine.ts` and re-running):

```text
× passes fallback: false so proxy failures reach the circuit breaker
× opens the circuit on zero-token results, even if the SDK swallows the error
× does not count an empty context as a failure
```

Full suite: 158 passed, 2 failed — both pre-existing and unrelated, in
`test/advancement-key-store.test.ts`, caused by this machine's Node
rejecting `--experimental-transform-types` in the child-process fixture
(`node: bad option: --experimental-transform-types`). They fail
identically on `upstream/main` without this patch.

## Real Behavior Proof

- Environment: Windows 11, Node v26.10.0, `plugins/openclaw` built with
`npm run build`, in-tree `headroom-ai@0.42.0`. Stub proxy is a
`node:http` server that accepts the connection and never responds.
Engine constructed with `requestTimeoutMs: 500`,
`circuitBreakerThreshold: 3`, `circuitBreakerCooldownMs: 60000`,
`minContextChars: 0`, driven through five `assemble()` calls with a
2000-char message.
- Exact command / steps: `npm run build` in `plugins/openclaw`, then
`node proof-circuit.mjs`, which starts the hung proxy on an ephemeral
port, points the engine at it, and makes five sequential `assemble()`
calls. Run once on the unpatched build and once on the patched build.
- Observed result: before the patch the breaker never opened and every
call paid the full timeout. After the patch it opens after the third
failure and calls 4 and 5 skip the proxy entirely (0 ms, 0 requests).
Messages returned are the originals in every case, so model output is
unchanged; only the latency is removed.

Before the patch:

```text
call 1: ms=1004 requestsSeenByProxy=2 totalProxyRequests=2
call 2: ms=1001 requestsSeenByProxy=2 totalProxyRequests=4
call 3: ms=1003 requestsSeenByProxy=2 totalProxyRequests=6
call 4: ms=1001 requestsSeenByProxy=2 totalProxyRequests=8
call 5: ms=1002 requestsSeenByProxy=2 totalProxyRequests=10
circuit-opened=false
```

After the patch:

```text
call 1: ms=1004 requestsSeenByProxy=2 totalProxyRequests=2 messagesAreOriginal=true
call 2: ms=1002 requestsSeenByProxy=2 totalProxyRequests=4 messagesAreOriginal=true
call 3: ms=1002 requestsSeenByProxy=2 totalProxyRequests=6 messagesAreOriginal=true
call 4: ms=0 requestsSeenByProxy=0 totalProxyRequests=6 messagesAreOriginal=true
call 5: ms=0 requestsSeenByProxy=0 totalProxyRequests=6 messagesAreOriginal=true
circuit-opened=true
  warn: [headroom] Circuit breaker opened after 3 errors (last: HeadroomConnectionError: Failed to connect to Headroom at http://127.0.0.1:53545: TimeoutError: The operation was aborted due to timeout); bypassing compression for 60000ms
  warn: [headroom] Circuit open — using uncompressed messages
  warn: [headroom] Circuit open — using uncompressed messages
```

- Not tested: a live OpenClaw host with a real provider (the engine
needs a host to drive, so I drove `assemble()` directly against the
built plugin), and the published `headroom-ai` beyond `0.42.0`.
`fallback` has been in the SDK's `CompressOptions` since the first TS
release, so the flag resolves against the plugin's `^0.40.0` range too.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — this is a plugin-local circuit
breaker inside `headroom-openclaw`, with no rollout flag or channel
gating involved.
- Minimum rollout channel: N/A, no gating is introduced.
- Stable/default behavior changed: yes, deliberately. The circuit
breaker now opens on proxy failures instead of never opening, so an
unhealthy proxy causes compression to be skipped for
`circuitBreakerCooldownMs` (default 60 s) rather than being attempted on
every turn. Messages are returned unchanged either way, so model output
is unaffected; the change removes repeated timeout latency.
- Kill switch / disable path: `enabled: false` disables the plugin
entirely; `circuitBreakerThreshold` can be raised to make the breaker
effectively unreachable; `circuitBreakerCooldownMs` bounds how long each
bypass lasts.
- Unsafe override required: no.
- Qualification impact: none — no provider, transport or config surface
changes, and no new dependency.
- Rollback path: revert the commit. The prior behavior was a single-line
`fallback: true`, and nothing here migrates or persists state, so
reverting restores the old behavior exactly.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Review caught one defect in my first attempt, now fixed and covered by a
test: with `minContextChars: 0`, `assemble()` forwards empty `messages:
[]` to the proxy, which answers `200` with `tokens_before: 0`. Counting
that as a failure meant a handful of empty-context turns could open the
breaker and then skip compression for a healthy proxy. The zero-token
check is now scoped to nonempty input.

Found with AI assistance (opencode) operated by me.
2026-11-10 12:34:14 -06:00
123XieC e6b5777806 fix(openclaw): return structured retrieval tool results (#4707)
## Description

The registered `headroom_retrieve` tool returns a string today. OpenClaw
Code Mode expects an `AgentToolResult` and rejects the string during
transcript capture. Return a text block and structured details for
success, validation errors, HTTP errors, and network failures. Both
invocation signatures now use this result shape. Consumers of the
exported factory can read the text from `content[0].text`.

<!-- Briefly explain the change and why it is needed. -->

Closes #4700.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Preserve the original successful response text and expose the parsed
response as `details`.
- Use OpenClaw’s `details.status: "error"` convention and preserve the
error message, response detail, or expiry hint.

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
cd plugins/openclaw
npm test         # 11 files, 158 tests passed on Node 22.22.2
npm run typecheck
npm run build
git diff --check
```

## Real Behavior Proof

- Environment: macOS arm64, Node 26.12.1, built plugin, loopback HTTP
fixture with synthetic stored content.
- Exact command / steps: run `npm run build` in `plugins/openclaw`.
Import `registerHeadroomPlugin` from `dist/index.js`, configure the
loopback `proxyUrl` with `autoStart: false` and `routeCodexViaProxy:
false`, and capture the factory passed to `registerTool`. Call
`execute(callId, { hash })` with an uppercase 12-hex hash and a 24-hex
hash against successful JSON object and string responses. Then call it
with an invalid hash, a missing key returning HTTP 404, and a valid hash
after stopping the server.
- Observed result: all five paths returned strings on `main` and result
objects after the patch. Request counts were `1, 1, 0, 1, 0` in both
runs. The fix preserved success text and exposed the three failures as
`details.status: "error"`. Seven expectations in the focused regression
file failed on `main`.
- Not tested: full OpenClaw host dispatch or a remote Headroom proxy.
The smoke test used actual plugin registration and HTTP with a small
host API shim.

## Runtime Rollout Safety

- Rollout-managed feature(s): retrieval tool in the OpenClaw plugin.
- Minimum rollout channel: existing plugin channel.
- Stable/default behavior changed: the registered tool returns a
structured result.
- Kill switch / disable path: existing OpenClaw plugin enablement.
- Unsafe override required: none.
- Qualification impact: plugin test suite and loopback verification
above.
- Rollback path: revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

AI assistance was used for investigation, implementation, and tests.

<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4707"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
2026-11-09 21:27:43 -06:00
APandJerrettDavis 8d3e9e5ca7 fix(openclaw): accept 12-character hashes in headroom_retrieve (#4640)
<!-- maintainer-current-main -->
## Maintainer latest-main checkpoint
Published `930d7f115250cb8d0344f0be8e4fa3fc9f40faa3` incorporates
current upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948` after a
clean merge. 153 plugin tests pass, 2 Windows skips; typecheck/build
pass. Eight actual TCP/store retrieval cases again recover exact
originals through direct/OpenClaw calls and lowercase/uppercase
12/24-character keys. Contributor correctness logic was not rewritten.
Earlier qualification captures below retain their original
commit/platform scope. Hosted CI must finish on this exact head before
approval; nothing merged.
<!-- /maintainer-current-main -->

## Description

The OpenClaw `headroom_retrieve` tool rejected 12-character hashes with
"Invalid hash format. Expected 24 hex characters." before the request
could reach `GET /v1/retrieve/{hash}`. The SmartCrusher row-drop path
keys its CCR markers with a 12-char SHA-256 hex prefix, mirrored into
the compression store through `explicit_hash`, so 12-char keys are valid
retrieval keys. Every other marker consumer in the repo already accepts
both lengths; this validator was the odd one out.

Closes #4639

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `plugins/openclaw/src/tools/headroom-retrieve.ts`: validator now
accepts 12- or 24-character hex hashes
(`/^(?:[a-f0-9]{12}|[a-f0-9]{24})$/i`); error message and both the tool
and parameter descriptions updated to say 12 or 24.
- `plugins/openclaw/test/headroom-retrieve.test.ts`: new regression
tests — 12-char accepted (reaches fetch), uppercase 12-char accepted,
24-char still accepted, non-hex / 13-char / empty rejected without a
fetch call, and both descriptions mention both lengths.

## Testing

- [x] New tests added for new functionality
- [x] Manual testing performed

The template's pytest/ruff/mypy items are N/A here: this change is
TypeScript-only, in the OpenClaw plugin. What actually ran (fresh clone
at upstream main 31dcaf41, Node v22.23.1, plugin dir `plugins/openclaw`,
`npm ci`):

### Test Output

```text
$ npx vitest run
 Test Files  10 passed (10)
      Tests  150 passed (150)

$ npm run typecheck
(no output — clean)
```

Before the fix, the issue's repro form showed a 12-char hash refused
locally (`{"error":"Invalid hash format. Expected 24 hex characters."}`,
fetch not called) while a 24-char hash reached the stub backend. After
the fix, both lengths reach the stub backend. The new test file fails
6/7 on the unpatched validator and passes 7/7 after; the one test that
passes on both sides is the "24-char still accepted" guard, which exists
to prove the fix did not regress the length that already worked. Full
suite went from 9 files / 143 tests to 10 files / 150 tests.

## Real Behavior Proof

- Environment: fresh clone of headroomlabs-ai/headroom at main 31dcaf41,
Node v22.23.1, npm 11.20.0, vitest, plugin dir `plugins/openclaw`.
- Exact command / steps: `npm ci`, then `npx vitest run` (full plugin
suite) and `npm run typecheck`; plus the issue's repro form (stubbed
fetch, call `headroom_retrieve` with a 12-char hash, observe whether a
request goes out) run before and after the patch.
- Observed result: 12-char hash now produces `GET
/v1/retrieve/{12-char-hash}` against the stub and returns the stored
content; suite 150/150 green, typecheck clean.
- Not tested: no live Headroom proxy (fetch stubbed throughout, so no
real round trip against a running Headroom); no Python or Rust suites
(change is TypeScript-only); no `npm run build`; no in-OpenClaw agent
session; other plugins were not checked for a copy of the same
validator.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: yes, intentionally — 12-character
hashes are now accepted instead of rejected; 24-character behavior is
unchanged, and all other lengths and non-hex input are still rejected.
- Kill switch / disable path: none needed; revert the commit to restore
the old behavior.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — no UI change.

## Additional Notes

One honest caveat on the tests: the "24-char still accepted" test passes
before and after by design (it guards against regression, so it cannot
fail on the unpatched code). The file as a whole fails before the fix
and passes after, which is what the repo's bug-fix rule asks for. The
three rejection tests fail pre-fix only on the changed error string;
their accept/reject verdicts are unchanged.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4640"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer follow-up qualification
Published `9bcb14fc447abdc844b77c399691e6bbc9e208f2` includes latest
main `ddbbb07746a274da5c9bd7380a42ba2a46e7515d`, including #4641's
OpenClaw execute adapter and the tsup update. The merge is clean;
contributor normalization logic is unchanged. The previous uppercase
finding is fixed by lowercasing accepted keys before the GET request,
with URL assertions for both lengths.

Windows, Node 22.22.0, Python 3.12.13: `npm ci --no-audit --no-fund`,
`npm test`, `npm run typecheck`, `npm run build` all pass. Full plugin
suite: **153 passed, 2 skipped**, 11 test files; package and
declarations build successfully.

Actual runtime proof: start an offline/beacon-opt-out localhost TCP
Uvicorn `create_app(ProxyConfig(stateless=True, optimize=False,
cache_enabled=False, rate_limit_enabled=False, log_requests=False))`;
store a Unicode original with `get_compression_store().store(original,
"short")` and again with `explicit_hash="a1b2c3d4e5f6"`; import
`createHeadroomRetrieveTool` from the built `dist/index.js`. For each
12/24-character lowercase and uppercase key, call both
`tool.execute({hash})` and `tool.execute("call_fixture", {hash})`,
JSON-parse the response, and assert `original_content` equals the exact
original and returned `hash` equals lowercase input. **All eight real
GET round-trips succeed**. The prior uppercase 404 no longer reproduces.
Both Devin findings are addressed by the code/test follow-up and this
live proof.

Not tested: an installed OpenClaw agent session, model/provider calls,
native rebuild, non-Windows runtime. Existing development native
extension was used only for proxy startup. Hosted checks must complete
on the final published head; nothing merged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-05 10:55:26 -06:00
Vivek SinghandJerrettDavis 362e1fc65b fix(openclaw): read the retrieve hash from params, not the tool-call id (#4641)
Fixes #4638

## Description

### Problem

`headroom_retrieve` is registered through `api.registerTool` as a
concrete OpenClaw tool. OpenClaw calls such a tool as
`execute(toolCallId, params, signal?, onUpdate?)`
(`ErasedAgentToolExecute` in `src/agents/tools/common.ts`), but the tool
read its first argument as `{ hash }`. At runtime `hash` was therefore
the tool-call id string, the 24-hex check failed, and every real call
returned `Invalid hash format` without sending a request. The same input
passed as the only argument reached the proxy, which is why the
reproducer in the issue isolates it cleanly.

## Change

`execute` now takes the OpenClaw shape: when the first argument is a
string (the tool-call id) the input comes from the second argument; when
it is an object, as in a direct `execute({ hash })` call, it is used as
before. A missing or non-string hash takes the existing format-error
path, so a bad call still fails closed without a request.

The hash format check and the return shape are unchanged. #4640 widens
the accepted hash lengths in the same function; the two changes touch
different lines of it.

## Tests

`plugins/openclaw/test/headroom-retrieve-execute-args.test.ts`, with
`fetch` stubbed:
- `execute("call_fixture", { hash })` sends `GET /v1/retrieve/<hash>`
and does not return the format error;
- `execute({ hash })` still sends the request;
- `execute("call_fixture", { hash: "not-a-hash" })` and
`execute("call_fixture")` return the format error without a request.

I could not run `vitest` here; the tests follow `proxy-manager.test.ts`
in the same folder (`vi.stubGlobal("fetch", ...)`).

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4641"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Type of Change
- [x] Bug fix

## Changes Made
- Select tool input from OpenClaw's second execute argument while
preserving direct object calls and the existing hash policy.
- Maintainer follow-up: assert the exact retrieved content in both
successful argument-shape tests. Contributor runtime logic is unchanged.

## Testing
- [x] Unit tests pass (full OpenClaw Vitest suite; Python tests are N/A
for this TypeScript change)

Maintainer qualification on Windows, Node 22.22.0, Python 3.12.13: `npm
ci --no-audit --no-fund`, `npm test`, `npm run typecheck`, and `npm run
build` in `plugins/openclaw`. Full plugin suite: 145 passed, 2 Windows
skips. Typecheck and package/declaration build passed. Test result
assertions were strengthened after the initial run; the complete suite
and typecheck were rerun successfully.

### Test Output

```text
$ npm test
Test Files 10 passed (10)
Tests 145 passed | 2 skipped (147)
$ npm run typecheck
(exit 0)
$ npm run build
ESM Build success
DTS Build success
```

## Real Behavior Proof
- Environment: Windows; built plugin; actual loopback TCP Uvicorn
Headroom proxy from this PR, actual in-memory CompressionStore; offline
and beacon opt-out enabled. Existing development native extension used
for startup; no fresh native build claimed.
- Exact command / steps: build the package; start
`create_app(ProxyConfig(stateless=True, optimize=False,
cache_enabled=False, rate_limit_enabled=False, log_requests=False))` on
a free localhost port; `get_compression_store().store(original, "short",
original_tokens=15, compressed_tokens=1)`; import
`createHeadroomRetrieveTool` from `dist/index.js`; call `await
tool.execute("call_fixture", {hash})` and `await tool.execute({hash})`;
JSON-parse and compare `original_content` to the stored Unicode
original. Compare the same two-argument call with the unchanged execute
implementation on #4640.
- Observed result: both supported calls on this PR recovered the exact
original, including Unicode, through real GET `/v1/retrieve/{hash}`. The
prior implementation returned `Invalid hash format` for the OpenClaw
call shape. Invalid hash input remained rejected before retrieval. The
current upstream OpenClaw `ErasedAgentToolExecute` declaration
independently confirms `(toolCallId, params, signal?, onUpdate?)`.
- Not tested: an installed OpenClaw agent session, model/provider calls,
abort/update behavior, cross-platform runtime, fresh native build. The
separate 12-character validator issue is scoped to #4640 and not fixed
by this PR.

## Runtime Rollout Safety
- Rollout-managed feature(s): none; existing retrieval tool.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: valid OpenClaw tool calls can reach
retrieval; direct calls remain supported.
- Kill switch / disable path: existing plugin disable path.
- Unsafe override required: no.
- Qualification impact: focused argument-shape coverage, full plugin
validation, real local proxy retrieval.
- Rollback path: revert the change; no schema migration.

## Review Readiness
Maintainer qualification and review of the test-only follow-up (not an
inferred author attestation):
- [x] I have performed a self-review
- [x] This PR is ready for human review
Hosted CI on the published final head must finish independently. No
merge has been performed.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 22:36:54 -06:00
24acbe8f32 fix(openclaw): record turn advancements as one marker file per key (#4212)
## Description

OpenClaw 2026.9.9 started calling the context engine's `commitTurn`
after every accepted turn. While a commit is pending, it runs every
later turn of that session on its legacy context path, which means no
Headroom compression. The plugin's turn-commit store (from #3442) has
three ways to leave a commit pending, and this PR removes all three.
Each committed key becomes one small marker file, created with exclusive
creation (`O_CREAT | O_EXCL`). There's no lock and no shared file to
rewrite, and only the key is stored.

**What we saw.** We run the plugin on Kubernetes with a read-only root
filesystem. After upgrading OpenClaw to 2026.9.9, every turn logged:

```
[context-engine] durable turn advancement remains queued: <key>: ENOENT: no such file or directory, mkdir '/home/openclaw/.headroom'
```

On the session's next turn, OpenClaw retries the pending commit before
assembling context. When the retry fails, it degrades that turn to the
legacy engine (`degradeBeforeStart("pending durable turn advancement
could not be completed before the next turn")` in OpenClaw's
`context-engine-turn-attempt`). A failed commit only increments
`attempt_count`, so the row stays pending and every later turn of the
session degrades too.

**The three problems in `advancement-key-store.ts`:**

1. **Read-only home.** The default
`~/.headroom/openclaw/commit-log.json` can't be created on a read-only
root. Node reports the failed `mkdir` as `ENOENT`, which reads like a
bug rather than a setting. `commitLogPath` exists on the engine but not
in the manifest's `configSchema` (`additionalProperties: false`), so
`HEADROOM_WORKSPACE_DIR` is the only way to move it.
2. **Unbounded growth.** Every accepted turn's `messages` (OpenClaw caps
a turn at 8 MiB) are kept forever in one JSON file. Each commit
re-reads, parses, copies and rewrites the whole file on the gateway's
event loop, so the file and each commit's latency grow with every turn
ever committed. Nothing reads the messages back (`get` is test-only),
and OpenClaw's transcript already holds them.
3. **A lock nothing reclaims.** By design, the lock is never
force-broken. A gateway killed mid-commit (an OOM kill, a node drain)
leaves `commit-log.json.lock` behind, and every later commit waits 60 s
and then fails until someone deletes it. With the retry above, every
turn then starts 60 s late before degrading.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **One marker per committed key**, at
`commit-log.d/<sha256[0:2]>/<sha256(key)>` next to the old log path,
created with `open(path, "wx")`. The first creator in any process gets
`"committed"`; every later caller gets `EEXIST`, i.e. `"duplicate"`.
Only that exclusive create can mean `"duplicate"`, and only when what
exists at the path is a marker file. An `EEXIST` from creating the shard
directory (a file where the directory belongs), or anything but a file
at the marker's path, is a storage error. The marker holds
`{advancementKey, committedAt}`: 99 bytes for OpenClaw's UUID keys.
- **Only the key is recorded.** For this engine, the advancement *is*
the key: `assemble()` works from the messages OpenClaw passes in, and
the plugin keeps no turn state. The contract asks for "one atomic,
idempotent write keyed by `advancementKey`" (OpenClaw
`docs/concepts/context-engine.md`), and creating the marker is that
write.
- **Keys are still never evicted**, for the reason #3442 gives.
- **Migration.** Before every commit and lookup, a store checks for a
`commit-log.json` from an earlier version. With no log there, that check
is one failed `open()`. If a log is there and it's a version the store
hasn't migrated, the store moves its keys into markers and records which
version that was (size, mtime, inode) in
`commit-log.d/migrated-legacy-log.json`. That record only saves work, so
a store that can't write it carries on and reads the log again later.
The log's version and its contents come from one open handle. The log
itself is never moved or changed, so an older version still running
against the same workspace keeps its own record, and nothing has to
write to the log's directory (which may be read-only, with a writable
marker volume mounted beneath it). An unchanged log is skipped by its
metadata alone, even if it can no longer be read. A changed one is
migrated again, so a key an older version commits at any moment, even
while a store is migrating the log, is never accepted as new. Only keys
the store hasn't seen get markers, so a rewrite costs only the keys it
adds. Both the object and the older array format are read. Several
processes can migrate at once safely, since creating an existing marker
is a no-op. A corrupt log still fails loudly rather than being read as
empty, and the store recovers once it's fixed.
- **An actionable error.** When the marker directory can't be created,
the message says what to do: `Cannot record turn advancements under
<dir>: <error>. Point HEADROOM_WORKSPACE_DIR (or the plugin's
commitLogPath) at a writable directory.` A lookup in a store whose path
a file blocks fails the same way on every platform, although Windows
reports such a path as `ENOENT`, which is also what a missing marker
gives.
- **`commitLogPath` is settable.** It's declared in
`openclaw.plugin.json`, with a UI hint and a README row.
- **Tests.** The store suite is rewritten for the new design. It keeps
the existing guarantees: committed, then duplicate; survives a restart;
never evicts across 600 keys; exactly one winner in-process and across
two OS processes; a failed write is retried, not remembered. New tests
cover:
  - a commit stores one small marker holding only the key;
- committing more keys never rewrites an earlier marker (same inode and
mtime);
  - a leftover `.lock` from a crashed process doesn't block a commit;
- a file where the marker directory or a shard directory belongs, and a
directory where a marker belongs: each is an error, never `"duplicate"`,
and the same key commits once the path is cleared;
- a file blocking the store's directory or its marker directory,
reported the way Windows reports it (`ENOENT`, not `ENOTDIR`): a lookup
is a storage error, not an absent key;
  - both legacy formats, including two stores migrating concurrently;
- a legacy log in a read-only directory: commits go through; a restart
doesn't read the unchanged log again; a migrated log that can no longer
be read doesn't block commits; and once the directory is writable, a key
an older version commits the old way (temp file + rename) is migrated
before a running store's next commit;
- a key an older version commits while a store is migrating the log: the
store's next lookup finds it, and its next commit, like a new store's,
is `"duplicate"`;
- an older version rewriting the log: only the key it added gets a
marker;
- a migration record that can't be written: commits go through, in that
store and in one started later;
  - a corrupt legacy log.

In `engine.test.ts`, a commit carrying 1 MB of tool output stores under
1 KB. The lock-holder fixture is removed, since there is no lock.

## Testing

- [ ] Unit tests pass (`pytest`) (n/a, TypeScript plugin only)
- [ ] Linting passes (`ruff check .`) (n/a)
- [ ] Type checking passes (`mypy headroom`) (n/a; `npm run typecheck`
passes)
- [x] New tests added for new functionality
- [x] Manual testing performed (see Real Behavior Proof)

### Test Output

```text
$ cd plugins/openclaw && npm run typecheck        # exit 0
$ npx vitest run                                  # Node v24.21.0, macOS
 Test Files  9 passed (9)
      Tests  139 passed (139)
$ npx vitest run                                  # Node v22.22.0, GitHub Actions windows-latest
 Test Files  9 passed (9)
      Tests  137 passed | 2 skipped (139)
```

## Real Behavior Proof

- Environment: macOS, Node v24.21.0, for the benchmark below: "old" is
`advancement-key-store.ts` from `main`, "new" is this PR, and the same
script drives both. In production: OpenClaw 2026.9.9 on Kubernetes with
a read-only root filesystem and `HEADROOM_WORKSPACE_DIR` on a writable
volume, running the plugin at v0.40.0 from a fork tag, with
`advancement-key-store.ts` identical to a9e9ea6e.
- Exact command / steps: one script drives both stores through the first
three cases, and the last compares production logs:
- commit 300 accepted turns, each carrying 100 KB of tool output, and
time commits #1, #100 and #300;
- commit once with a `commit-log.json.lock` left behind, as a killed
process would leave it;
- commit with the workspace's parent directory not writable (mode 0555);
- in production, compare one gateway's OpenClaw logs over a 12-hour
night on v0.40.0's store with its first 3 hours on this store.
- Observed result: the new store stays small and fast and ignores a
leftover lock. In production, no commit failed and no turn fell back to
legacy.

  | | Old store | New store |
  |---|---|---|
  | Store on disk after 300 turns | 30.0 MB | 0.02 MB |
| Commit #1 / #100 / #300 | 3.3 / 14.3 / 40.4 ms | 2.6 / 0.1 / 0.2 ms |
| Leftover lock file from a crashed process | fails after 60.0 s |
committed immediately |
| Workspace not writable | `EACCES: permission denied, mkdir
'…/.headroom'` | same error, plus which setting fixes it |

In production, v0.40.0's store failed all 53 commits overnight with the
`remains queued` line above, and all 8 later turns of a session fell
back to legacy. On this store, the same gateway made 38 commits in its
first 3 hours, none failed, and no turn fell back. The 5 later turns we
could trace to their session all ran on headroom. The 38 markers take
296 KiB on disk, shard directories included.
- Not tested: the two permission-bit tests (a read-only directory, an
unwritable migration record) on Windows or as root, where they are
skipped, since permission bits don't restrict either. Our production
build predates the last three commits, which change only how a legacy
log is migrated and how a blocked path is reported, and that deployment
never had a legacy log.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: the commit store's on-disk format (an
existing log is migrated on first use). The default location is
unchanged.
- Kill switch / disable path: none needed. Pointing `commitLogPath` or
`HEADROOM_WORKSPACE_DIR` elsewhere starts a fresh store.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert. The old log stays where it was, so an older
version still finds every key it wrote. Keys committed after the upgrade
live only in markers, so an older version could re-accept a turn that
was mid-retry during the downgrade.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes (on Node
24; see Additional Notes)
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- A version older than this one, running at the same moment against the
same workspace, keeps writing the old file, and this store migrates
whatever it adds. The older version doesn't read markers, though, so it
could still accept a key only this version committed. That's the same
mixed-version caveat any format change has.
- Unrelated to this change: the cross-process tests run `node
--experimental-transform-types`, which Node 26 no longer accepts (`bad
option`). They pass on Node 24, which CI uses.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4212"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-03 22:46:13 -06:00
liuyongcheng0000andJerrett Davis 33f204406c fix(openclaw/release): ship responsive npm integrations (#1458)
## Description

Fixes #977 and #2353.

The v0.35.0 release built all three JavaScript packages successfully,
but npm rejected every publish with ENEEDAUTH. Because each publish step
used continue-on-error, the publish-npm job still concluded success. npm
therefore remains on headroom-ai 0.22.4 and headroom-openclaw 0.1.0,
while headroom-opencode is absent.

The checked-in OpenClaw source had already corrected the old
strict-schema mismatch and added a circuit breaker, but its assemble
hook still called compression for every tiny conversation and allowed a
30-second default wait. This PR completes both the runtime correction
and the delivery path instead of merely exposing one failure.

## Changes

### OpenClaw runtime

- Skip proxy compression below an 800-character context floor (roughly
200 tokens), avoiding synchronous HTTP work on short conversations that
cannot produce meaningful savings.
- Measure actual content, including tool-call arguments, so a single
large tool result remains eligible.
- Allow operators to disable the floor with minContextChars=0.
- Lower the interactive request ceiling from 30 seconds to 2 seconds.
- Pass that timeout into the SDK, whose AbortSignal cancels the HTTP
request, instead of racing a promise while the fetch continues in the
background.
- Preserve circuit-breaker fail-open behavior and document/schema the
new defaults.

### npm release integrity

- Authenticate with npm before publishing any package, so a missing or
invalid NPM_TOKEN fails before a partial release begins.
- Make SDK, OpenClaw, and OpenCode npm publish failures fatal when npm
publishing is enabled.
- Preserve NPM_SKIP=true as the explicit opt-out.
- Make retries safe after partial publication by skipping only the exact
package version already present, then continuing to missing packages.
- Remove the false-success notice path.

## Existing #977 corrections verified

- The current OpenClaw schema declares startupTimeoutMs,
routeCodexViaProxy, gatewayProviderIds, request timeout, and
circuit-breaker fields written/used by the installer and runtime.
- wrap defaults to the real published package name, headroom-openclaw.
- unwrap removes managed values while preserving unrelated user
configuration.
- The coding savings profile uses protect_recent=0 paired with
type-based recent-read protection, avoiding the old blanket
short-session protection without weakening tool-result fidelity.

## Testing

- 80 OpenClaw plugin tests pass.
- TypeScript typecheck and production build pass.
- 123 OpenClaw wrap/provider, savings-profile, compress endpoint, and
MCP contract tests pass.
- 47 release-workflow and MCP contract tests pass.
- The npm release asset builder installed/imported and verified the
actual 0.35.1 SDK and OpenClaw tarballs.
- Workflow YAML parses; Ruff, format, and git diff checks pass.

## Required repository action

A maintainer must replace or restore the npm publishing credential in
NPM_TOKEN, then rerun the v0.35.0 release workflow (or publish the next
version). This PR makes that failure visible and safely rerunnable;
repository code cannot provision the external npm credential.

Co-authored with @liuyongcheng0000, whose original PR identified the
masked npm failures.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/1458"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
2026-10-28 17:40:52 -05:00
Mansurshakh Japarov 37f3f9efe1 fix(openclaw): preserve tool-call thought signatures (#4038)
## Description

Preserve provider metadata while canonicalizing assistant tool calls.
The existing normalization rebuilt a `toolCall` from four fields and
dropped `thoughtSignature`, including on unchanged assistant turns
restored after compressing a different tool result.

Closes #4036. The original dynamic-notice behavior in #3809 is already
addressed on main and is not claimed as a new fix here.

## Type of Change

- [x] Bug fix (non-breaking)
- [x] Documentation update

## Changes Made

- Retain the original tool-call block properties before applying the
existing canonical type/id/name/arguments normalization.
- Keep existing missing-metadata defaults and legacy `tool_use`
normalization; no new options, routes, or dependencies.
- Add a failing-before regression for a signed canonical tool call and
document metadata retention in the existing plugin README.

## Testing

- Plugin: `npm test` — **8 files, 121 tests passed**.
- Plugin: `npm run typecheck` and `npm run build` — passed.
- Regression before the production change: **1 failed, 4 passed**, with
`thoughtSignature` observed as `undefined`.
- Required pre-push `make ci-precheck` — **passed**: complete Rust
workspace formatting, strict Clippy, workspace tests; native extension
build/import verification; Python qualification suite **177 passed, 4
skipped**; Conventional Commit lint.
- Complete Ruff formatting check — **1768 files already formatted**.
- Whole-tree Ruff lint reported five unchanged executable OAuth2 Python
files lacking shebangs; no unrelated chmod or suppression was applied.
- Whole-tree `mypy headroom` checked 566 files and reported one
unchanged `truthy-function` error in
`headroom/integrations/langchain/chat_model.py:394`; no Python
implementation was changed by this PR.
- Complete Python suite — **15,630 passed, 612 skipped, 7 failed**. The
failures are outside the changed plugin: two CLI-import shape errors,
one WebSocket lifecycle timeout, and four Kompress readiness
expectations. These broad failures are not suppressed or claimed green.
- Published GitHub checks — **19 passed, 19 skipped**, with no failed or
pending check at the recorded readiness check.

## Real Behavior Proof

Environment: Linux x86_64, Bun 1.3.13, Node 22.23.1, Python 3.13.14,
repository-pinned Rust 1.95.0; base
`3c418f6f0b190ad1060413c0eadd77f20ab3cf46`.

Actual converter/restorer command: `bun -e` importing
`agentToOpenAIIndexed` and `restoreAgentMessages` from
`plugins/openclaw/src/convert.ts`. The input contains a user turn, a
Google-provider assistant turn with a signed canonical tool call, and a
separate tool result. Only the tool-result text is compressed in the
returned wire messages.

Before: the identical Google-provider fixture run against untouched main
fails its unchanged-assistant deep equality assertion because the tool
call loses `thoughtSignature`.

After:

```text
unchanged_user_byte_exact: true
unchanged_assistant_byte_exact: true
thinking_signature_preserved: true
compressed_tool_text: short tool result
tool_metadata_preserved: true
```

The unchanged-assistant equality includes the tool-call signature and
arguments. This is an actual converter round trip, not a mocked provider
or a forwarding-only test.

Not tested: full OpenClaw gateway installation, paid provider calls,
provider cache-hit rates/billing, macOS, Windows.

## Runtime Rollout Safety

No rollout-managed feature/channel or unsafe override is added. Default
plugin normalization now retains opaque provider tool-call metadata;
canonical defaults remain intact. Disable the OpenClaw plugin to avoid
its context assembly, or revert this three-file commit as rollback.
Existing Rust/Python qualification remains passing as recorded above.

## Review Readiness

- [x] Existing conversion consumers, normalization defaults, and scope
reviewed
- [x] This PR is ready for human review; all broad-check outcomes are
recorded, including unrelated Python limitations

## Checklist

- [x] Corresponding documentation updated
- [x] Regression proves the fix is effective
- [x] No new warnings observed in the changed plugin
- [x] No `CHANGELOG.md` edit (release-please owns it)

## Additional Notes

Fresh duplicate check covered all 195 open PRs. #4034 preserves
signatures in Python CCR handling and does not modify this OpenClaw
normalizer. AI-assisted: omp / GPT-6.1 Sol.
2026-10-08 10:19:33 -05:00
f8851878a4 fix(plugins/openclaw): keep compression system-prompt notice cache-stable (#3811)
## Description

`HeadroomContextEngine.assemble()` in the OpenClaw plugin returns a
`systemPromptAddition` string that OpenClaw prepends to the system
prompt every turn. It interpolated the per-turn `tokensSaved` count, so
the system prompt — the start of the provider's cached prefix — changed
on every compressing turn, defeating prompt caching. Reported cost: a
turn saving 267 tokens caused a ~93K-token cache miss on Bedrock/Claude
Opus 5.5, roughly 4,000x the tokens the compression saved.

Refs #3809

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `plugins/openclaw/src/engine.ts`: replaced the interpolated
`${result.tokensSaved}` notice with a fixed constant
(`HEADROOM_COMPRESSION_NOTICE`), so any turn that includes it is
byte-identical to any other. Added a `hasAnnouncedCompression` instance
flag so the notice, once shown (tokensSaved > 100), stays present and
unchanged on later turns even when a given turn has nothing new to
compress
- `plugins/openclaw/src/engine.ts`: added `announceCompression`
(boolean, default `true`) to `HeadroomEngineConfig` to disable the
notice entirely.
- `plugins/openclaw/openclaw.plugin.json`: added `announceCompression`
to `configSchema.properties` (required — schema has
`additionalProperties: false`) and a matching `uiHints` entry.
- `plugins/openclaw/README.md`: documented the new option in the
configuration table.
- `plugins/openclaw/test/engine.test.ts`: 5 new tests, static notice
with no interpolated count, byte-identical notices across differing
`tokensSaved`, notice staying present on a turn with nothing to compress
(the flap-prevention regression case), `announceCompression: false`
opt-out, and the existing below-threshold no-op case.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ npx vitest run test/engine.test.ts
 RUN  v4.1.10 /Users/sarah_ling/Documents/GitHub/headroom/plugins/openclaw

 Test Files  1 passed (1)
      Tests  27 passed (27)
   Start at  18:17:43
   Duration  527ms

$ npx vitest run
 Test Files  1 failed | 6 passed (7)
      Tests  3 failed | 104 passed (107)

# The 3 failures are in test/advancement-key-store.test.ts, unrelated to this change
```

## Real Behavior Proof

- Environment: local vitest run against `plugins/openclaw` (mocked
`compress()`/proxy — no live OpenClaw host or real Bedrock/Anthropic
request).
- Exact command / steps: added a temporary test that mocks a 6-turn
session mirroring the reported scenario (tokensSaved: 0, 267, 0, 9000,
0, 50) and recorded `assemble()`'s `systemPromptAddition` per turn.
- Observed result: notice absent on turn 1, appears static and identical
on turns 2–6 (including turns 3/5 with nothing to compress, and turn 6's
below-threshold 50-token save) — 1 total transition across 6 turns,
versus the old behavior's distinct string on every compressing turn.
- Not tested: no live OpenClaw + real provider run to directly observe
the Bedrock cache-read/cache-write fields improve, as the original
issue's report did. The fix is verified at the unit level (the exact
value the plugin hands to OpenClaw), not against a real provider cache.

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: N/A
- Stable/default behavior changed: yes, the notice text changes (no
longer reports a live token count) and stays present for the rest of the
session instead of tracking that turn's compression state. Default
(`announceCompression` unset) still shows the notice, matching prior
default-on behavior.
- Kill switch / disable path: set `announceCompression: false` in the
plugin config to fully disable the notice.
- Unsafe override required: None
- Qualification impact: None
- Rollback path: Revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3811"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-06 08:37:07 -05:00
d1ad189819 fix(plugins/openclaw): return messages the proxy did not change exactly as they came in (#3826)
## Description

When `/v1/compress` compresses anything, `assemble()` returns
`openAIToAgent(result.messages)`: every history message converted
AgentMessage → OpenAI → AgentMessage. That round trip is lossy. It drops
assistant `thinking` blocks and flattens user and tool content blocks
into one string, so on every compressing turn the whole history changes,
and the provider prompt cache misses from the first message instead of
from the first compressed one. Details and measurements are in #3809.

This change returns every message the proxy didn't change exactly as
OpenClaw passed it in, and rebuilds only the compressed ones, keeping
their original fields and non-text blocks.

Refs #3809. The other half of the cache problem, the note, is #3811 (see
Real Behavior Proof).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `plugins/openclaw/src/convert.ts`: `agentToOpenAIIndexed()` stamps
each converted message with the index of its source AgentMessage in
`_headroomMeta`, which the proxy returns unchanged.
`restoreAgentMessages()` maps the proxy result back onto the originals.
An unchanged message passes through exactly as it came in. A changed one
keeps every original field and block except the text compression
replaced, and compressed tool-call arguments are applied to the matching
block. Messages the proxy dropped for the token budget stay dropped, and
a message it added (no index) is converted as before.
- `plugins/openclaw/src/engine.ts`: `assemble()` uses the two instead of
`agentToOpenAI` / `openAIToAgent`.
- `plugins/openclaw/test/assemble-cache-stability.test.ts`: 7 new tests.

## Testing

- [ ] Unit tests pass (`pytest`): N/A, TypeScript-only change. I ran the
plugin's vitest suite instead.
- [ ] Linting passes (`ruff check .`): N/A, no Python changed.
- [ ] Type checking passes (`mypy headroom`): N/A. I ran the plugin's
`npm run typecheck` instead.
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ npx vitest run test/assemble-cache-stability.test.ts
 Test Files  1 passed (1)
      Tests  7 passed (7)

# The same 7 tests against the unmodified engine.ts on main: 5 fail, 2 pass.
# The 2 that pass guard the unchanged fallback path and check that nothing leaks.

$ npx vitest run
 Test Files  1 failed | 7 passed (8)
      Tests  3 failed | 106 passed (109)
# The 3 failures are in test/advancement-key-store.test.ts and fail identically on
# unmodified main here (Node 26.7: the spawned `node --experimental-transform-types` worker exits non-zero).

$ npm run typecheck   # exit 0
$ npm run build       # exit 0
```

## Real Behavior Proof

- Environment: openclaw 2026.9.6 with headroom 0.39.0 (proxy + plugin),
Claude Opus 5.5 on Amazon Bedrock. The plugin's `dist/index.js` was
rebuilt from v0.39.0 plus exactly this diff and swapped in temporarily.
`engine.ts` and `convert.ts` are identical between v0.39.0 and main.
- Exact command / steps: a fresh Slack DM session with 3 turns. (1) A
~400-word answer with no tools. (2) One exec producing a JSON array of
~170 pods. (3) A one-line follow-up sent inside the 5-minute cache TTL,
so turn 3's `assemble()` compresses turn 2's tool result. The cache
figures come from the provider's usage fields for the first model call
of turn 3. `headroom_tokens_saved_total` rose by ~5,000 on that turn,
confirming the compression.
- Observed result:

| | note disabled | current note |
| --- | --- | --- |
| main (lossy round trip) | read 57,369 (tools + stable system prompt
only) | read 57,369 |
| this PR | **read 62,150** (everything through turn 2's user message) |
read 56,906 |

With this PR and the note disabled (a local edit for the test), turn 3
read the full prefix up to the compressed message. With the current
note, the note first appears on that turn. It lives in openclaw's
dynamic system-prompt suffix, which sits before every message, so the
message cache still misses. This PR is one half of the fix; a stable
note (#3811, or `announceCompression: false`) is the other.
- Not tested: #3811's static note together with this change; I'd expect
one miss when the note first appears in a session, then stable. Also not
tested: long agent turns. The provider's cache lookback is limited, so
with many content blocks between the compressed tool result and the next
cache breakpoint, a call may still fall back to the system-prompt entry.

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: N/A
- Stable/default behavior changed: Yes, for the messages `assemble()`
returns when compression happens. Untouched messages are now returned as
passed in instead of round-tripped. Content and order are otherwise
unchanged.
- Kill switch / disable path: the plugin's existing `enabled: false`.
- Unsafe override required: No
- Qualification impact: None expected
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review. It ran on real traffic for two
days first; results are in the comments.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation: N/A, no
user-facing option changed.
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes, apart
from the 3 pre-existing advancement-key-store failures above
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

This relies on the proxy's contract that it returns the same messages in
the same order, echoes `_headroomMeta` unchanged, and changes only the
messages it compressed. I checked that against a live 0.39.0 proxy: 5
messages in, 5 out, only the tool result changed. The proxy's
`config.frozen_message_count` doesn't cover this case, because the loss
happens in the plugin's own conversion after the proxy returns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_017qUMQxaNhAa8xrjsausyyi

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-01 15:55:22 -05:00
Radhakrishnan PachyappanandJerrett Davis a28dd4eab7 fix(openclaw): declare transcriptSemantics and implement commitTurn (#3442)
## Description

Adds OpenClaw transcript-semantics declarations and an accepted-turn log
for admitted-turn advancement. Fixes #3441.

Messages and advancement keys are written together in
`HEADROOM_WORKSPACE_DIR/openclaw/commit-log.json` (default workspace:
`~/.headroom`). Each commit re-reads the log inside an exclusive
sibling-file lock and replaces the log atomically. Keys are retained
without eviction; prior key-only logs are recognized for migration.
Failed writes do not create an in-memory phantom duplicate.

## Locking and recovery

The lock serializes processes sharing the workspace and is never
reclaimed merely because it is old. A waiting caller times out after 60
seconds instead of committing without ownership. Release checks the
acquisition token.

A crashed holder can leave an orphan `.lock` file. Recovery is
operational: stop writers using that workspace, establish that the
holder is no longer running, remove the orphan lock, then resume. Do not
delete a live writer's lock. Automatic crash recovery and power-loss
durability are not claimed.

## Testing

- 99 plugin tests pass, including distinct-key and same-key
child-process commits and an aged live-lock regression.
- TypeScript checking passes.
- The aged-lock regression fails against the prior age-reclaim
implementation and passes against the current source. Test cleanup
releases the child even on assertion failure.
- Current-head CI is pending.

## Real Behavior Proof

- Environment: macOS, two real Node child processes importing
`DurableAdvancementKeyStore`.
- Exact command / steps: A acquires the lock and reads the empty log;
pause A while alive for 31 seconds. Start B committing another key and
verify it remains pending, then resume A.
- Observed result: B does not finish while A holds the lock. After
release, both return `committed` and both message payloads remain in the
log. The prior implementation acknowledged both but lost one turn.
- Not tested: automatic recovery from crashed holders (requires the
operator procedure above); power-loss durability.

## Runtime Rollout Safety

- Rollout-managed feature(s): OpenClaw transcript-semantics declaration.
- Minimum rollout channel: Host capability negotiation.
- Stable/default behavior changed: Accepted messages and keys persist in
a workspace-shared log. The log grows without eviction, and an orphan
lock requires operator recovery.
- Kill switch / disable path: Revert the plugin change or use the host's
legacy context-engine path.
- Unsafe override required: No.
- Qualification impact: Admission uses atomic/idempotent commit
behavior; contention fails closed.
- Rollback path: Revert the plugin change. Preserve the log if retry
history must remain available.

## Review Readiness

- [x] Prior message persistence, retry, cross-process, and stale-lock
findings resolved.
- [x] Local tests and type checking pass.
- [ ] Current-head CI complete.
- Final human review is required.

---------

Signed-off-by: Radhakrishnan P <gingeekrishna@gmail.com>
Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
2026-09-24 17:17:46 -05:00
Gautam SharmaandJerrettDavis eb4da64774 fix(openclaw): delegate native compaction (#2304)
## Description

Fix the OpenClaw context engine claiming ownership of persistent
compaction while its `compact()` implementation performed no compaction.

Previously, `ownsCompaction` was `true`, which disabled OpenClaw's
built-in automatic compaction. However, `compact()` did not read or
rewrite
the session transcript and still returned `{ ok: true, compacted: true
}`.

Headroom now declares that it does not own persistent compaction and
delegates `/compact` and overflow-recovery requests through OpenClaw's
official `delegateCompactionToRuntime(...)` API. Headroom continues to
perform zero-cost compression while assembling model input.

OpenClaw's context-engine contract requires non-owning engines to
delegate `compact()` explicitly; setting `ownsCompaction: false` alone
does
not restore manual or overflow-recovery compaction:

https://docs.openclaw.ai/concepts/context-engine#ownscompaction

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Changed `HeadroomContextEngine.info.ownsCompaction` from `true` to
`false`.
- Replaced the successful no-op `compact()` implementation with OpenClaw
native-runtime delegation.
- Added a small runtime adapter that dynamically loads
`openclaw/plugin-sdk/core` without bundling the optional OpenClaw peer
dependency.
- Return the real delegated compaction result instead of manufacturing a
successful result.
- Increment compaction statistics only when OpenClaw reports that
compaction occurred.
- Propagate native compaction failures so OpenClaw can apply its normal
failure-isolation behavior.
- Removed stale claims that RollingWindow drops messages during Headroom
compression.
- Updated the OpenClaw README to distinguish input compression from
persistent compaction.
- Added an Unreleased changelog entry.
- Added regression coverage for delegated success, delegated no-op, and
delegated failure behavior.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ cd plugins/openclaw
$ npx vitest run test/engine.test.ts

Test Files  1 passed (1)
Tests       13 passed (13)

$ npm run typecheck
> tsc --noEmit
Passed

$ npm run build
ESM Build success
DTS Build success

$ npm pack --dry-run
npm notice total files: 13
npm notice package size: 35.2 kB

$ uv run --extra dev --frozen ruff check .
All checks passed!

$ uv run --extra dev --frozen mypy headroom --ignore-missing-imports
$ uv run --extra dev --frozen pytest
9360 passed, 565 skipped, 4 failed
- Anthropic compaction transform ordering
- macOS AF_UNIX temporary socket path length
- Dashboard cached-stats expectation
- Content-router log fallback expectation

The repository-wide format check also flags pre-existing formatting in the untouched `headroom/proxy/handlers/anthropic.py`.

The complete OpenClaw plugin suite reports:

```text
Test Files  1 failed | 5 passed (6)
Tests       1 failed | 77 passed (78)
```

The remaining failure expects `sh -lc` while the existing implementation uses `sh -c`. The identical failure was reproduced in a detached,
untouched `origin/main` worktree, where it reported `1 failed | 74 passed`.

## Real Behavior Proof

- Environment: macOS 26.5.1 on arm64, Node.js v24.14.1, npm 11.11.0.
- Exact command / steps: Ran the focused Vitest suite, TypeScript type-check, production build, and package dry run. Inspected the emitted
`dist/index.js` to verify that it contains `ownsCompaction: false`, preserves the runtime import of `openclaw/plugin-sdk/core`, and calls
`delegateCompactionToRuntime(params)` from `compact()`.
- Observed result: A delegated successful result containing `tokensBefore: 20000` and `tokensAfter: 8000` was returned unchanged and
incremented the compaction counter. A delegated `{ compacted: false }` result did not increment the counter. A rejected native compaction
propagated the original error and did not report or count a successful compaction. Headroom's `compress()` was not invoked as a substitute for
persistent compaction.
- Not tested: A live installed OpenClaw gateway, an end-to-end `/compact` command against a persisted production transcript, or provider-
triggered overflow recovery. Those paths enter the same official OpenClaw delegation API exercised by the engine boundary.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did not edit CHANGELOG.md; release notes are generated automatically

## Additional Notes

The complete-suite checkbox remains unchecked because of the unrelated baseline failures documented above. The focused OpenClaw compaction
tests, TypeScript type-check, production build, Ruff lint, and mypy all pass.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-09 22:47:05 -05:00
6576ef639c fix(openclaw-plugin): circuit breaker + per-request timeout for proxy resilience (#639)
## Description

This change adds bounded timeout and circuit-breaker behavior so
OpenClaw can degrade safely when Headroom or the upstream stream stalls,
while returning structured proxy errors instead of hanging.

Closes #638 by improving OpenClaw/proxy resilience when the Headroom
proxy stalls or Anthropic resets a stream. The PR adds proxy-side
handling for `httpx.RemoteProtocolError`, returns structured 502
responses for otherwise unhandled proxy middleware errors, and adds
OpenClaw plugin timeout/circuit-breaker fallback behavior.

## Type of Change

- [x] Bug fix
- [ ] New feature
- [x] Documentation
- [ ] Refactor
- [x] Tests only

## Changes Made

- Added OpenClaw plugin per-request compression timeout and circuit
breaker fallback.
- Cleared timeout timers after successful or failed compression so
successful calls do not leave pending timers.
- Added a focused Vitest regression for timeout cleanup.
- Added `contracts.tools` for `headroom_retrieve` without whole-file
manifest reformatting.
- Added proxy handling for mid-stream `httpx.RemoteProtocolError` and
structured 502 fallback behavior.
- Documented the new OpenClaw resilience configuration fields.

## Testing

- [x] Unit tests
- [x] Integration-style proxy tests
- [x] Typecheck/build
- [ ] Manual testing

### Test Output

```text
cd plugins/openclaw && npm test
Test Files 6 passed (6), Tests 55 passed (55)

cd plugins/openclaw && npm run typecheck
passed

cd plugins/openclaw && npm run build
Build success

UV_SKIP_WHEEL_FILENAME_CHECK=1 uv run --with pytest --with pytest-asyncio --with fastapi --with httpx --with uvicorn --with h2 python -m pytest tests/test_proxy_streaming_resilience.py -q
24 passed in 2.16s
```

## Real Behavior Proof

- Environment: Windows 11, Node/npm from local plugin worktree, Python
3.13.3, focused local worktree for PR #639.
- Exact command / steps: Installed plugin dependencies, ran OpenClaw
plugin tests/typecheck/build, and ran the proxy streaming resilience
suite with required async/FastAPI/httpx extras.
- Observed result: Plugin tests, typecheck, build, and proxy resilience
tests all passed.
- Not tested: Live OpenClaw gateway session in this pass; original
reporter previously verified patched files in a container and OpenClaw
degraded/recovered cleanly.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: Sergei Baikin <sergei.baikin@fotograf.de>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
2026-08-13 11:52:01 -05:00
JD DavisandJD Davis 01161fe019 test(openclaw): match inherited PATH shell check (#2821)
## Description

Fix the OpenClaw test failure on `main` by aligning its PATH-launcher
expectation with the intentionally shipped `sh -c` behavior from #1459.

The non-login shell preserves the PATH inherited from the OpenClaw
process. Changing production code back to `sh -lc` would risk a login
shell resetting that PATH and would undo the compatibility fix. This PR
therefore corrects only the stale assertion; runtime behavior and
defaults do not change.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Expect `sh -c` for the non-Windows lightweight `command -v headroom`
check.
- Preserve the existing Windows `where.exe` behavior and all launcher
behavior.

## Testing

- [x] Unit tests pass (`npm test`)
- [x] Linting passes (`npm run typecheck`)
- [x] Type checking passes (`npm run typecheck`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ npm test
Test Files  6 passed (6)
Tests  75 passed (75)

$ npm run typecheck
> tsc --noEmit

$ npm run build
ESM Build success
DTS Build success

$ npm ci
found 0 vulnerabilities
```

## Real Behavior Proof

- Environment: macOS, Node/npm, clean install from `origin/main` at
`2954e37048f8dcffe16e1c37b8f71afb0094a0a2`.
- Exact command / steps: `cd plugins/openclaw && npm ci && npm test &&
npm run typecheck && npm run build`.
- Observed result: all 75 OpenClaw tests pass, TypeScript typechecking
succeeds, and both ESM and declaration builds succeed.
- Not tested: Windows execution; its separate `where.exe` expectation
and implementation are unchanged.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — test-only correction with no UI changes.

## Additional Notes

History confirms #1459 deliberately changed `sh -lc` to `sh -c` while
adding explicit uv-tool path detection. Reverting the implementation
would change runtime discovery semantics; updating the stale test
preserves the accepted behavior.

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
2026-08-06 19:21:52 -07:00
5709291914 chore(release): harden local artifact smokes (#1824)
## Description

Harden the release artifact workflow so maintainers can reproduce npm
and Python release checks locally and so OpenClaw packaging does not
depend on a not-yet-published SDK version. This follow-up also keeps the
OpenClaw loader export contract explicit and updates the PR after the
branch was merged with current `headroomlabs/main`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

## Changes Made

- Added reusable local release smoke scripts for npm assets, Python
wheel/sdist artifacts, and the combined release gate.
- Switched the release workflow npm asset build to the reusable npm
builder.
- Made OpenClaw release asset building install against the just-built
local SDK tarball before rewriting packed metadata to the release
dependency range.
- Kept the OpenClaw source dependency registry-installable at
`headroom-ai@^0.22.3` while the packed artifact still ships
`^<release-version>`.
- Added `registerHeadroomPlugin` as a named export while preserving the
default `{ register }` OpenClaw loader contract.
- Added Windows development bootstrap docs/script and regression tests
for version sync, npm asset ordering, OpenClaw source installability,
and Python wheel smoke import isolation.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run --with pytest python -m pytest tests/test_release_workflows.py scripts/tests/test_version_sync.py -q
44 passed, 1 warning

node --check scripts/build_npm_release_assets.mjs
node --check scripts/verify_npm_release_assets.mjs

python -m py_compile scripts/build_python_release_smoke.py scripts/release_smoke_all.py scripts/version-sync.py
python scripts/verify-versions.py
All versions aligned at 0.31.0

npm ci (plugins/openclaw)
added 88 packages, audited 89 packages, found 0 vulnerabilities

python scripts/release_smoke_all.py --out release-assets-local/all-pr1824-postmerge-fixed-20260710-104739
Verified npm release assets for 0.31.0
wheel metadata OK: headroom_ai-0.31.0-cp310-abi3-win_amd64.whl contains headroom/_core.pyd
sdist License-File metadata OK: ['LICENSE', 'NOTICE']
smoke-import OK: version=0.31.0 hello=headroom-core
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.14.3, Node v24.14.0, npm 11.9.0, uv
0.11.15, Rust/Cargo already installed in the local development
environment.
- Exact command / steps: Ran `python scripts/release_smoke_all.py --out
release-assets-local/all-pr1824-postmerge-fixed-20260710-104739` after
syncing this branch with current `headroomlabs/main`.
- Observed result: The command built `headroom-ai-0.31.0.tgz`,
`headroom-openclaw-0.31.0.tgz`,
`headroom_ai-0.31.0-cp310-abi3-win_amd64.whl`, and
`headroom_ai-0.31.0.tar.gz`; npm verifier passed; the wheel installed
into a fresh venv and imported `headroom._core`.
- Not tested: Full GitHub Actions release matrix and publish jobs with
real PyPI/npm/GitHub Packages credentials.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A.

## Additional Notes

The post-merge full smoke initially failed because the OpenClaw source
package depended on `headroom-ai@^0.31.0`, which is not available on
public npm yet. The builder now installs OpenClaw against the just-built
local SDK tarball before build/pack, and then rewrites packed metadata
to `^0.31.0`.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-14 16:07:34 -04:00
felixboenkost-droidandBjörn-Christian Bönkost 6d116b15f1 Harden OpenClaw plugin proxy routing (#1074)
## Description

Hardens the bundled OpenClaw plugin so configured proxy routing is
fail-closed and `autoStart` is opt-in.

Closes: N/A

This follow-up is intentionally separate from the ContentRouter cache
fix because it changes plugin/gateway behavior rather than core
compression routing.

The plugin should not mutate upstream provider routing unless a
configured proxy URL is reachable and looks like Headroom. It should
also avoid unhandled startup promise rejections when proxy startup is
fire-and-forget.

Why this shape:

- `autoStart: false` by default matches deployments where Headroom is
supervised externally, for example by systemd. The plugin should not
silently start or assume ownership of a proxy unless the operator opted
in.
- Provider routing is fail-closed: a configured URL must first respond
like Headroom, not merely expose a generic liveness endpoint. This
prevents accidentally routing model traffic through the wrong local
service.
- `/readyz` is treated as liveness, not identity. Identity comes from
Headroom-shaped stats endpoints (`/v1/retrieve/stats` or `/stats`)
because those are harder for unrelated services to satisfy by accident.
- Startup remains asynchronous, but errors are captured and exposed
instead of becoming unhandled promise rejections.
- This is a separate PR because the core cache fix is about compression
correctness, while this patch is about integration safety around
OpenClaw gateway routing.

## Type of Change

- [x] Bug fix (non-breaking change fixes issue)
- [ ] New feature (non-breaking change adds functionality)
- [ ] Breaking change (fix or feature would cause existing functionality
change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Make proxy `autoStart` opt-in (`default: false`).
- Probe configured `proxyUrl` before applying provider routing.
- Treat `/readyz` as liveness only; require Headroom-shaped
`/v1/retrieve/stats` or `/stats` for identity.
- Observe fire-and-forget startup promise rejection and expose startup
error for callers.
- Isolate proxy-ready listener failures.
- Keep provider routing deferred when no active/probed Headroom proxy
exists.
- Register retrieve tool with explicit `headroom_retrieve` name.
- Extend plugin/unit tests for configured proxy failures, generic
non-Headroom endpoints, path collisions, and routing behavior.

Changed files:

- `plugins/openclaw/README.md`
- `plugins/openclaw/openclaw.plugin.json`
- `plugins/openclaw/src/engine.ts`
- `plugins/openclaw/src/plugin/index.ts`
- `plugins/openclaw/src/proxy-manager.ts`
- `plugins/openclaw/test/engine.test.ts`
- `plugins/openclaw/test/gateway-config.test.ts`
- `plugins/openclaw/test/plugin-runtime-routing.test.ts`
- `plugins/openclaw/test/proxy-manager.test.ts`

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added new functionality
- [x] Manual testing performed

### Test Output

```text
$ npm test

Test Files  6 passed (6)
Tests  74 passed (74)

$ npm run typecheck
tsc --noEmit

$ npm run build
tsup && node prepare-dist.mjs
Build success
```

## Real Behavior Proof

- Environment: local OpenClaw plugin package in the Headroom repo.
- Exact command / steps:
  - Run plugin test suite.
  - Run TypeScript typecheck.
  - Run plugin build.
- Observed result:
  - Tests passed: `74/74`.
  - Typecheck passed.
  - Build passed.
- Not tested:
- Full OpenClaw Gateway integration as part of this standalone PR prep.

## Review Readiness

- [x] I performed self-review
- [x] This PR ready for human review

## Checklist

- [x] My code follows project's style guidelines
- [x] I performed self-review my code
- [ ] I commented my code, particularly in hard-to-understand areas
- [x] I made corresponding changes documentation
- [x] My changes generate no new warnings
- [x] I added tests prove fix is effective or feature works
- [x] New and existing unit tests pass locally my changes
- [ ] I updated CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A.

## Additional Notes

Checklist items left unchecked intentionally:

- No CHANGELOG update included.
- No extra comments were needed beyond existing code structure.

Co-authored-by: Björn-Christian Bönkost <bjoern@v2202603344248440850.hotsrv.de>
2026-06-22 22:52:59 -05:00
JerrettDavisandCopilot f63cfe7a1e fix: lazy-load proxy startup dependencies
Keep the default Headroom proxy startup path lightweight so OpenClaw can launch it reliably on Windows. This defers heavyweight provider, cache, transform, pricing, and detector imports until they are actually needed, adds a lightweight version module, and keeps the OpenClaw launcher on the configured Python path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-10 20:33:46 -05:00
JerrettDavisandCopilot 37f32a8922 test(openclaw): cover branch routing paths
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 23:46:16 -05:00
JerrettDavisandCopilot 2f05705043 fix(openclaw): normalize provider proxy routing
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 22:28:17 -05:00
JerrettDavisandCopilot d4f6e3938f fix(proxy): add fast-fail launch settings
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 22:03:21 -05:00
JerrettDavisandCopilot 4d8b76f7da fix(openclaw): use lightweight headroom launcher checks
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 21:39:14 -05:00
JerrettDavisandCopilot afb339059b fix(openclaw): launch Windows headroom shims via shell
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 21:16:32 -05:00
JerrettDavis 0da1a7b4de Add OpenClaw unwrap flow and non-blocking proxy startup 2026-04-08 21:16:32 -05:00
JerrettDavis 8134939efb fix(openclaw): preserve upstream response paths 2026-04-08 21:15:52 -05:00
JerrettDavis 621c3b6f43 fix(openclaw): wait for proxy before gateway routing 2026-04-08 21:10:49 -05:00
JerrettDavis e0383675aa fix(openclaw): keep gateway routing runtime-only 2026-04-08 21:10:49 -05:00
JerrettDavis 6f500dfa9c feat(openclaw): support configurable gateway providers 2026-04-08 21:10:49 -05:00
JerrettDavis a4fe13d62f docs(openclaw): document codex gateway routing 2026-04-08 21:10:49 -05:00
JerrettDavis 92b7b09f70 fix(openclaw): route codex through headroom proxy 2026-04-08 21:10:49 -05:00
JerrettDavis 2edff08668 fix(openclaw): normalize engine messages for discord 2026-04-06 22:18:37 -05:00
JerrettDavis 5f075dee81 fix(openclaw): preserve tool call linkage in converted history 2026-04-03 22:49:30 -05:00
JerrettDavis 3e368f13c8 refactor(openclaw): DRY up duplicate code across plugin
- Extract shared defaultLogger constant, use in both ProxyManager and
  HeadroomContextEngine (was duplicated inline in both constructors)
- Extract parseProxyUrl() helper, reused by normalizeAndValidateProxyUrl
  and withDefaultPort (eliminates redundant URL parsing)
- Add test helpers: stubProbeSuccess/NonHeadroom/Unreachable to replace
  6 identical inline fetch mock constructions
- Remove duplicate probe tests from engine.test.ts (already covered in
  proxy-manager.test.ts)
- Clean up unused probeHeadroomProxy import from engine.test.ts

Net: -26 lines, same coverage (28 tests)
2026-04-03 10:30:10 -05:00
JerrettDavis 89f3e5f01f fix(openclaw): proxyPort only applies to local URLs, consistency pass
- proxyPort default (8787) now only applied to local URLs; remote URLs
  use their protocol default (80/443) as expected
- Remove redundant localhost-only entries from network capabilities
  (covered by http://*:* and https://*:* wildcards)
- Sync package.json capabilities with openclaw.plugin.json
- Export isLocalProxyUrl from barrel index
- Fix JSDoc indentation in start() method
- Rename "Required Proxy Setup" to "Manual Proxy Setup" with accurate
  description for remote proxy users
- Clarify autoStart description: local-only, ignored for remote
- Add test: remote URL does not get proxyPort applied

Build: tsc clean, 30 tests passed
2026-04-03 10:30:10 -05:00
JerrettDavis 4c3d9c4e28 feat(openclaw): support remote Headroom proxy (connect-only)
Remove localhost-only restriction from proxyUrl validation. Remote URLs
are connect-only: the plugin probes and uses them but never attempts to
spawn a subprocess. Auto-start remains available for local addresses
(127.0.0.1/localhost) only.

- Accept http:// and https:// URLs for any hostname
- Add isLocalProxyUrl() helper to gate auto-start on local addresses
- Remote URLs that fail probe get a clear "Remote proxy not reachable"
  error instead of falling through to auto-start
- Update plugin.json schema, network capabilities, and UI hints
- Update plugin README with local vs remote proxy sections
- Add tests: remote connect, remote fail-fast, isLocalProxyUrl

Build: tsc --noEmit clean
Tests: 29 passed (0 skipped)
2026-04-03 10:30:09 -05:00
JerrettDavis fa99ba785e docs: add OpenClaw plugin section to main README
Add dedicated OpenClaw plugin section with install instructions,
explanation of --dangerously-force-unsafe-install requirement (proxy
subprocess spawning), and quick config example. Links to plugin README
for full details. Update integration tables to reference new section.

Also fix proxy port handling: apply default proxyPort when explicit
proxyUrl omits port, and allow trailing slash in proxyUrl validation
pattern. Add test coverage for port defaulting behavior.
2026-04-03 10:30:09 -05:00
JerrettDavis 3e11dc6316 Remove skipped integration test registration 2026-04-02 22:51:13 -05:00
JerrettDavis 3569eabdbd Add comprehensive tests for proxy detection and launcher logic 2026-04-02 22:28:48 -05:00
JerrettDavis 205ec54df2 Harden openclaw plugin proxy handling and metadata 2026-04-02 21:27:12 -05:00
chopratejas a41c7f0c89 OpenClaw plugin fixes, telemetry fix, cost tracker improvement, tokenBudget support
OpenClaw plugin:
- Fix assistant content always returned as array (fixes flatMap crash)
- Clean up debug logging, restore clean plugin entry point
- Add test for text-only assistant round-trip
- Set headroom-ai dependency to npm (not local path)

Telemetry:
- Switch from sb_publishable_ to JWT anon key (PostgREST needs JWT for RLS)
- Switch from upsert to plain INSERT (anon role upsert was failing with 42501)
- Verified: full payload with all JSONB columns writes successfully (201)

Cost tracker:
- Use output_buffer instead of hardcoded 500 for cost estimation in client.py
- CostTracker counterfactual: value removed tokens at avg effective $/token
  (actual_spend / billed_tokens) instead of uncached list price — aligns
  savings with real billing mix (cache reads, writes, uncached)
- Add debug log for /stats summary payload
- Add 3 tests for new cost savings calculation

TypeScript SDK:
- Add tokenBudget param to compress(), HeadroomClient, CompressOptions
- Proxy /v1/compress accepts optional token_budget to override model limit
2026-03-28 23:46:03 -07:00
chopratejas 053c4a06af feat: add OpenClaw ContextEngine plugin (@headroom-ai/openclaw)
- HeadroomContextEngine: implements ContextEngine interface (bootstrap, ingest,
  assemble, compact, afterTurn, prepareSubagentSpawn, dispose)
- ProxyManager: auto-detects running proxy or spawns one as child process,
  health checks, restart on crash, graceful shutdown
- AgentMessage ↔ OpenAI format conversion (user, assistant/tool_use, toolResult)
- headroom_retrieve CCR tool: agent can retrieve original uncompressed content
- Plugin manifest (openclaw.plugin.json) with config schema
- 11 tests (6 unit + 5 integration), all passing
- Integration tested: 100 servers compressed, 3,735 tokens saved (82%)
2026-03-28 13:43:40 -07:00