fix(plugins/openclaw): return messages the proxy did not change exactly as they came in (#3826)

## Description

When `/v1/compress` compresses anything, `assemble()` returns
`openAIToAgent(result.messages)`: every history message converted
AgentMessage → OpenAI → AgentMessage. That round trip is lossy. It drops
assistant `thinking` blocks and flattens user and tool content blocks
into one string, so on every compressing turn the whole history changes,
and the provider prompt cache misses from the first message instead of
from the first compressed one. Details and measurements are in #3809.

This change returns every message the proxy didn't change exactly as
OpenClaw passed it in, and rebuilds only the compressed ones, keeping
their original fields and non-text blocks.

Refs #3809. The other half of the cache problem, the note, is #3811 (see
Real Behavior Proof).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `plugins/openclaw/src/convert.ts`: `agentToOpenAIIndexed()` stamps
each converted message with the index of its source AgentMessage in
`_headroomMeta`, which the proxy returns unchanged.
`restoreAgentMessages()` maps the proxy result back onto the originals.
An unchanged message passes through exactly as it came in. A changed one
keeps every original field and block except the text compression
replaced, and compressed tool-call arguments are applied to the matching
block. Messages the proxy dropped for the token budget stay dropped, and
a message it added (no index) is converted as before.
- `plugins/openclaw/src/engine.ts`: `assemble()` uses the two instead of
`agentToOpenAI` / `openAIToAgent`.
- `plugins/openclaw/test/assemble-cache-stability.test.ts`: 7 new tests.

## Testing

- [ ] Unit tests pass (`pytest`): N/A, TypeScript-only change. I ran the
plugin's vitest suite instead.
- [ ] Linting passes (`ruff check .`): N/A, no Python changed.
- [ ] Type checking passes (`mypy headroom`): N/A. I ran the plugin's
`npm run typecheck` instead.
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ npx vitest run test/assemble-cache-stability.test.ts
 Test Files  1 passed (1)
      Tests  7 passed (7)

# The same 7 tests against the unmodified engine.ts on main: 5 fail, 2 pass.
# The 2 that pass guard the unchanged fallback path and check that nothing leaks.

$ npx vitest run
 Test Files  1 failed | 7 passed (8)
      Tests  3 failed | 106 passed (109)
# The 3 failures are in test/advancement-key-store.test.ts and fail identically on
# unmodified main here (Node 26.7: the spawned `node --experimental-transform-types` worker exits non-zero).

$ npm run typecheck   # exit 0
$ npm run build       # exit 0
```

## Real Behavior Proof

- Environment: openclaw 2026.9.6 with headroom 0.39.0 (proxy + plugin),
Claude Opus 5.5 on Amazon Bedrock. The plugin's `dist/index.js` was
rebuilt from v0.39.0 plus exactly this diff and swapped in temporarily.
`engine.ts` and `convert.ts` are identical between v0.39.0 and main.
- Exact command / steps: a fresh Slack DM session with 3 turns. (1) A
~400-word answer with no tools. (2) One exec producing a JSON array of
~170 pods. (3) A one-line follow-up sent inside the 5-minute cache TTL,
so turn 3's `assemble()` compresses turn 2's tool result. The cache
figures come from the provider's usage fields for the first model call
of turn 3. `headroom_tokens_saved_total` rose by ~5,000 on that turn,
confirming the compression.
- Observed result:

| | note disabled | current note |
| --- | --- | --- |
| main (lossy round trip) | read 57,369 (tools + stable system prompt
only) | read 57,369 |
| this PR | **read 62,150** (everything through turn 2's user message) |
read 56,906 |

With this PR and the note disabled (a local edit for the test), turn 3
read the full prefix up to the compressed message. With the current
note, the note first appears on that turn. It lives in openclaw's
dynamic system-prompt suffix, which sits before every message, so the
message cache still misses. This PR is one half of the fix; a stable
note (#3811, or `announceCompression: false`) is the other.
- Not tested: #3811's static note together with this change; I'd expect
one miss when the note first appears in a session, then stable. Also not
tested: long agent turns. The provider's cache lookback is limited, so
with many content blocks between the compressed tool result and the next
cache breakpoint, a call may still fall back to the system-prompt entry.

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: N/A
- Stable/default behavior changed: Yes, for the messages `assemble()`
returns when compression happens. Untouched messages are now returned as
passed in instead of round-tripped. Content and order are otherwise
unchanged.
- Kill switch / disable path: the plugin's existing `enabled: false`.
- Unsafe override required: No
- Qualification impact: None expected
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review. It ran on real traffic for two
days first; results are in the comments.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation: N/A, no
user-facing option changed.
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes, apart
from the 3 pre-existing advancement-key-store failures above
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

This relies on the proxy's contract that it returns the same messages in
the same order, echoes `_headroomMeta` unchanged, and changes only the
messages it compressed. I checked that against a live 0.39.0 proxy: 5
messages in, 5 out, only the tool result changed. The proxy's
`config.frozen_message_count` doesn't cover this case, because the loss
happens in the plugin's own conversion after the proxy returns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_017qUMQxaNhAa8xrjsausyyi

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
jedi-tal
2026-10-01 15:55:22 -05:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent db90b93da9
commit d1ad189819
3 changed files with 327 additions and 4 deletions
+133
View File
@@ -260,6 +260,139 @@ export function normalizeAgentMessages(messages: any[]): any[] {
return messages.map((message) => normalizeAgentMessage(message));
}
/** `_headroomMeta` key recording which AgentMessage a converted message came from. */
const SOURCE_INDEX_KEY = "headroomSourceIndex";
/**
* Like `agentToOpenAI`, but stamps every converted message with the index of the AgentMessage it came from.
* The proxy returns `_headroomMeta` unchanged, so `restoreAgentMessages` can map its result back onto the
* originals even when it drops messages to fit a token budget.
*/
export function agentToOpenAIIndexed(messages: any[]): OpenAIMessage[] {
const result: OpenAIMessage[] = [];
messages.forEach((message, index) => {
for (const converted of agentToOpenAI([message])) {
result.push({
...converted,
_headroomMeta: { ...(converted._headroomMeta ?? {}), [SOURCE_INDEX_KEY]: index },
});
}
});
return result;
}
/**
* Map the proxy's result back onto the AgentMessages OpenClaw passed in.
*
* `openAIToAgent(agentToOpenAI(m))` is lossy: it drops `thinking` blocks and flattens user and tool content
* blocks into one string. Applying it to every message whenever anything is compressed rewrites the whole
* history, which invalidates the provider's prompt cache from the first message on every compressing turn.
* So a message the proxy returned unchanged is passed through exactly as it came in, and a message it did
* change keeps every original field and block except the text that compression replaced.
*
* `sent` must be the `agentToOpenAIIndexed(originals)` output that was given to the proxy.
*/
export function restoreAgentMessages(
originals: any[],
sent: OpenAIMessage[],
returned: OpenAIMessage[],
): any[] {
const sentByIndex = new Map<number, OpenAIMessage>();
for (const message of sent) {
const index = sourceIndexOf(message);
if (index !== undefined) sentByIndex.set(index, message);
}
const result: any[] = [];
for (const message of returned) {
const index = sourceIndexOf(message);
const original = index === undefined ? undefined : originals[index];
const before = index === undefined ? undefined : sentByIndex.get(index);
if (original === undefined || before === undefined) {
// Not one of ours (e.g. a message the proxy added): convert it the way it always was.
result.push(...openAIToAgent([withoutSourceIndex(message)]));
} else if (sameWireMessage(before, message)) {
result.push(normalizeAgentMessage(original));
} else {
result.push(applyCompressedMessage(original, message));
}
}
return result;
}
function sourceIndexOf(message: OpenAIMessage): number | undefined {
const index = message._headroomMeta?.[SOURCE_INDEX_KEY];
return typeof index === "number" ? index : undefined;
}
function withoutSourceIndex(message: OpenAIMessage): OpenAIMessage {
if (!message._headroomMeta || !(SOURCE_INDEX_KEY in message._headroomMeta)) return message;
const { [SOURCE_INDEX_KEY]: _index, ...meta } = message._headroomMeta;
return { ...message, _headroomMeta: meta };
}
function sameWireMessage(a: OpenAIMessage, b: OpenAIMessage): boolean {
return (
a.role === b.role &&
a.content === b.content &&
a.tool_call_id === b.tool_call_id &&
a.name === b.name &&
JSON.stringify(a.tool_calls ?? null) === JSON.stringify(b.tool_calls ?? null)
);
}
function applyCompressedMessage(original: any, compressed: OpenAIMessage): any {
const base = normalizeAgentMessage(original);
const text = typeof compressed.content === "string" ? compressed.content : "";
if (base.role === "assistant") {
let content = replaceTextBlocks(base.content, text);
if (Array.isArray(compressed.tool_calls)) {
content = content.map((block: any) => withCompressedArguments(block, compressed.tool_calls!));
}
return { ...base, content };
}
if (typeof base.content === "string") return { ...base, content: text };
return { ...base, content: replaceTextBlocks(base.content, text) };
}
/** Replace the text blocks with one block holding `text`, where the first one was; keep every other block. */
function replaceTextBlocks(content: unknown, text: string): any[] {
const blocks = Array.isArray(content) ? content : [{ type: "text", text: String(content ?? "") }];
const result: any[] = [];
let placed = false;
for (const block of blocks) {
const isText = typeof block === "string" || (isRecord(block) && block.type === "text");
if (!isText) {
result.push(block);
continue;
}
if (!placed) {
if (text) result.push(isRecord(block) ? { ...block, text } : { type: "text", text });
placed = true;
}
}
if (!placed && text) result.unshift({ type: "text", text });
return result;
}
function withCompressedArguments(block: any, toolCalls: any[]): any {
if (!isRecord(block) || (block.type !== "toolCall" && block.type !== "tool_use")) return block;
const call = toolCalls.find((tc) => tc?.id === block.id);
const raw = call?.function?.arguments;
if (typeof raw !== "string") return block;
const current = block.type === "toolCall" ? block.arguments : block.input;
if ((typeof current === "string" ? current : JSON.stringify(current ?? {})) === raw) return block;
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
parsed = raw;
}
return block.type === "toolCall" ? { ...block, arguments: parsed } : { ...block, input: parsed };
}
/**
* Extract text from content blocks.
*/
+6 -4
View File
@@ -9,7 +9,7 @@
import { compress } from "headroom-ai";
import { ProxyManager, defaultLogger, type ProxyManagerConfig, type ProxyManagerLogger } from "./proxy-manager.js";
import { agentToOpenAI, normalizeAgentMessages, openAIToAgent } from "./convert.js";
import { agentToOpenAIIndexed, normalizeAgentMessages, restoreAgentMessages } from "./convert.js";
import { DurableAdvancementKeyStore, defaultCommitLogPath } from "./advancement-key-store.js";
import {
delegateCompactionToRuntime,
@@ -114,7 +114,9 @@ export class HeadroomContextEngine {
/**
* Assemble context for the model — THE CORE HOOK.
*
* Converts AgentMessage[] → OpenAI format → compress() → AgentMessage[]
* Converts AgentMessage[] → OpenAI format → compress() → AgentMessage[]. Only messages the proxy
* actually changed are rebuilt; the rest are returned exactly as OpenClaw passed them, so the
* provider prompt cache survives up to the first compressed message (see restoreAgentMessages).
*/
async assemble(params: {
sessionId: string;
@@ -140,7 +142,7 @@ export class HeadroomContextEngine {
try {
// Convert AgentMessage → OpenAI format
const openaiMessages = agentToOpenAI(params.messages);
const openaiMessages = agentToOpenAIIndexed(params.messages);
// Compress via proxy — pass tokenBudget so RollingWindow enforces it
const result = await withTimeout(
@@ -162,7 +164,7 @@ export class HeadroomContextEngine {
}
// Convert back to AgentMessage format
const compressedAgentMessages = openAIToAgent(result.messages);
const compressedAgentMessages = restoreAgentMessages(params.messages, openaiMessages, result.messages);
this.resetCircuit();
// Track stats
@@ -0,0 +1,188 @@
import { afterEach, describe, expect, it, vi } from "vitest";
const mocked = vi.hoisted(() => ({
compress: vi.fn(),
start: vi.fn(async () => "http://127.0.0.1:8787"),
stop: vi.fn(async () => undefined),
logger: { debug: vi.fn(), error: vi.fn(), info: vi.fn(), warn: vi.fn() },
}));
vi.mock("headroom-ai", () => ({ compress: mocked.compress }));
vi.mock("../src/proxy-manager.js", () => ({
ProxyManager: class {
start = mocked.start;
stop = mocked.stop;
},
defaultLogger: mocked.logger,
}));
import { normalizeAgentMessages } from "../src/convert.js";
import { HeadroomContextEngine } from "../src/engine.js";
afterEach(() => {
mocked.compress.mockReset();
});
// An OpenClaw-shaped history: a multi-block user message, an assistant turn with a signed thinking block
// and a tool call, a large tool result, the final answer, and the next user message.
function history() {
return [
{
role: "user",
content: [
{ type: "text", text: "[Slack DM from tal]" },
{ type: "text", text: "list the pods" },
],
timestamp: 1,
senderId: "U123",
},
{
role: "assistant",
content: [
{ type: "thinking", thinking: "run kubectl", thinkingSignature: "sig-1" },
{ type: "text", text: "Running it." },
{ type: "toolCall", id: "t1", name: "exec", arguments: { command: "kubectl get pods -A -o json" } },
],
api: "bedrock-converse-stream",
provider: "amazon-bedrock",
model: "claude",
stopReason: "toolUse",
timestamp: 2,
},
{
role: "toolResult",
toolCallId: "t1",
toolName: "exec",
content: [{ type: "text", text: "[" + '{"ns":"argocd","name":"a"},'.repeat(200) + "]" }],
isError: false,
timestamp: 3,
details: { exitCode: 0 },
},
{
role: "assistant",
content: [
{ type: "thinking", thinking: "count them", thinkingSignature: "sig-2" },
{ type: "text", text: "There are 200 pods." },
],
api: "bedrock-converse-stream",
provider: "amazon-bedrock",
model: "claude",
stopReason: "stop",
timestamp: 4,
},
{ role: "user", content: [{ type: "text", text: "which namespace?" }], timestamp: 5 },
];
}
// A proxy that compresses only tool results, the way the real /v1/compress does for JSON tool output:
// same messages, same order, `_headroomMeta` echoed back unchanged.
function compressToolResults(messages: any[]) {
return {
compressed: true,
tokensBefore: 1000,
tokensAfter: 400,
tokensSaved: 600,
messages: messages.map((m) => (m.role === "tool" ? { ...m, content: "[compressed]" } : m)),
};
}
function engine() {
const e = new HeadroomContextEngine();
(e as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
return e;
}
describe("assemble() keeps the provider prompt cache", () => {
it("returns every message it did not compress exactly as OpenClaw passed it in", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
const input = history();
const expected = normalizeAgentMessages(history());
const { messages } = await engine().assemble({ sessionId: "s", messages: input });
expect(messages).toHaveLength(5);
for (const i of [0, 1, 3, 4]) expect(messages[i]).toEqual(expected[i]);
// The two losses the old full round trip caused, on messages nothing was compressed in:
expect(messages[0].content).toEqual(history()[0].content); // content blocks not flattened
expect(messages[1].content.map((b: any) => b.type)).toEqual(["thinking", "text", "toolCall"]); // thinking kept
expect(messages[3].content.map((b: any) => b.type)).toEqual(["thinking", "text"]);
});
it("keeps the history before the first compressed message byte-identical", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
expect(JSON.stringify(messages.slice(0, 2))).toBe(JSON.stringify(normalizeAgentMessages(history()).slice(0, 2)));
});
it("replaces only the text of a compressed tool result and keeps its other fields and blocks", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
const input = history();
input[2].content.push({ type: "image", data: "aGk=", mimeType: "image/png" } as any);
const { messages } = await engine().assemble({ sessionId: "s", messages: input });
expect(messages[2]).toEqual({
...normalizeAgentMessages([input[2]])[0],
content: [
{ type: "text", text: "[compressed]" },
{ type: "image", data: "aGk=", mimeType: "image/png" },
],
});
expect(messages[2].details).toEqual({ exitCode: 0 });
});
it("follows a result the proxy shortened to fit the token budget", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => {
const r = compressToolResults(messages);
return { ...r, messages: r.messages.slice(2) }; // rolling window dropped the two oldest
});
const expected = normalizeAgentMessages(history());
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
expect(messages).toHaveLength(3);
expect(messages[0].toolCallId).toBe("t1");
expect(messages[0].content).toEqual([{ type: "text", text: "[compressed]" }]);
expect(messages[1]).toEqual(expected[3]);
expect(messages[2]).toEqual(expected[4]);
});
it("converts a message the proxy added itself the same way as before", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => {
const r = compressToolResults(messages);
return { ...r, messages: [{ role: "user", content: "[3 earlier messages omitted]" }, ...r.messages] };
});
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
expect(messages).toHaveLength(6);
expect(messages[0]).toMatchObject({ role: "user", content: "[3 earlier messages omitted]" });
});
it("applies compressed tool-call arguments without dropping the assistant's thinking", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => ({
...compressToolResults(messages),
messages: messages.map((m) =>
m.role === "assistant" && m.tool_calls
? { ...m, tool_calls: [{ ...m.tool_calls[0], function: { ...m.tool_calls[0].function, arguments: '{"command":"kubectl get pods"}' } }] }
: m,
),
}));
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
expect(messages[1].content.map((b: any) => b.type)).toEqual(["thinking", "text", "toolCall"]);
expect(messages[1].content[2].arguments).toEqual({ command: "kubectl get pods" });
});
it("never leaks its bookkeeping into the messages it returns", async () => {
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
expect(JSON.stringify(messages)).not.toContain("headroomSourceIndex");
});
});