mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(plugins/openclaw): return messages the proxy did not change exactly as they came in (#3826)
## Description When `/v1/compress` compresses anything, `assemble()` returns `openAIToAgent(result.messages)`: every history message converted AgentMessage → OpenAI → AgentMessage. That round trip is lossy. It drops assistant `thinking` blocks and flattens user and tool content blocks into one string, so on every compressing turn the whole history changes, and the provider prompt cache misses from the first message instead of from the first compressed one. Details and measurements are in #3809. This change returns every message the proxy didn't change exactly as OpenClaw passed it in, and rebuilds only the compressed ones, keeping their original fields and non-text blocks. Refs #3809. The other half of the cache problem, the note, is #3811 (see Real Behavior Proof). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `plugins/openclaw/src/convert.ts`: `agentToOpenAIIndexed()` stamps each converted message with the index of its source AgentMessage in `_headroomMeta`, which the proxy returns unchanged. `restoreAgentMessages()` maps the proxy result back onto the originals. An unchanged message passes through exactly as it came in. A changed one keeps every original field and block except the text compression replaced, and compressed tool-call arguments are applied to the matching block. Messages the proxy dropped for the token budget stay dropped, and a message it added (no index) is converted as before. - `plugins/openclaw/src/engine.ts`: `assemble()` uses the two instead of `agentToOpenAI` / `openAIToAgent`. - `plugins/openclaw/test/assemble-cache-stability.test.ts`: 7 new tests. ## Testing - [ ] Unit tests pass (`pytest`): N/A, TypeScript-only change. I ran the plugin's vitest suite instead. - [ ] Linting passes (`ruff check .`): N/A, no Python changed. - [ ] Type checking passes (`mypy headroom`): N/A. I ran the plugin's `npm run typecheck` instead. - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ npx vitest run test/assemble-cache-stability.test.ts Test Files 1 passed (1) Tests 7 passed (7) # The same 7 tests against the unmodified engine.ts on main: 5 fail, 2 pass. # The 2 that pass guard the unchanged fallback path and check that nothing leaks. $ npx vitest run Test Files 1 failed | 7 passed (8) Tests 3 failed | 106 passed (109) # The 3 failures are in test/advancement-key-store.test.ts and fail identically on # unmodified main here (Node 26.7: the spawned `node --experimental-transform-types` worker exits non-zero). $ npm run typecheck # exit 0 $ npm run build # exit 0 ``` ## Real Behavior Proof - Environment: openclaw 2026.9.6 with headroom 0.39.0 (proxy + plugin), Claude Opus 5.5 on Amazon Bedrock. The plugin's `dist/index.js` was rebuilt from v0.39.0 plus exactly this diff and swapped in temporarily. `engine.ts` and `convert.ts` are identical between v0.39.0 and main. - Exact command / steps: a fresh Slack DM session with 3 turns. (1) A ~400-word answer with no tools. (2) One exec producing a JSON array of ~170 pods. (3) A one-line follow-up sent inside the 5-minute cache TTL, so turn 3's `assemble()` compresses turn 2's tool result. The cache figures come from the provider's usage fields for the first model call of turn 3. `headroom_tokens_saved_total` rose by ~5,000 on that turn, confirming the compression. - Observed result: | | note disabled | current note | | --- | --- | --- | | main (lossy round trip) | read 57,369 (tools + stable system prompt only) | read 57,369 | | this PR | **read 62,150** (everything through turn 2's user message) | read 56,906 | With this PR and the note disabled (a local edit for the test), turn 3 read the full prefix up to the compressed message. With the current note, the note first appears on that turn. It lives in openclaw's dynamic system-prompt suffix, which sits before every message, so the message cache still misses. This PR is one half of the fix; a stable note (#3811, or `announceCompression: false`) is the other. - Not tested: #3811's static note together with this change; I'd expect one miss when the note first appears in a session, then stable. Also not tested: long agent turns. The provider's cache lookback is limited, so with many content blocks between the compressed tool result and the next cache breakpoint, a call may still fall back to the system-prompt entry. ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: N/A - Stable/default behavior changed: Yes, for the messages `assemble()` returns when compression happens. Untouched messages are now returned as passed in instead of round-tripped. Content and order are otherwise unchanged. - Kill switch / disable path: the plugin's existing `enabled: false`. - Unsafe override required: No - Qualification impact: None expected - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review. It ran on real traffic for two days first; results are in the comments. ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation: N/A, no user-facing option changed. - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes, apart from the 3 pre-existing advancement-key-store failures above - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes This relies on the proxy's contract that it returns the same messages in the same order, echoes `_headroomMeta` unchanged, and changes only the messages it compressed. I checked that against a live 0.39.0 proxy: 5 messages in, 5 out, only the tool result changed. The proxy's `config.frozen_message_count` doesn't cover this case, because the loss happens in the plugin's own conversion after the proxy returns. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017qUMQxaNhAa8xrjsausyyi Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
db90b93da9
commit
d1ad189819
@@ -260,6 +260,139 @@ export function normalizeAgentMessages(messages: any[]): any[] {
|
||||
return messages.map((message) => normalizeAgentMessage(message));
|
||||
}
|
||||
|
||||
/** `_headroomMeta` key recording which AgentMessage a converted message came from. */
|
||||
const SOURCE_INDEX_KEY = "headroomSourceIndex";
|
||||
|
||||
/**
|
||||
* Like `agentToOpenAI`, but stamps every converted message with the index of the AgentMessage it came from.
|
||||
* The proxy returns `_headroomMeta` unchanged, so `restoreAgentMessages` can map its result back onto the
|
||||
* originals even when it drops messages to fit a token budget.
|
||||
*/
|
||||
export function agentToOpenAIIndexed(messages: any[]): OpenAIMessage[] {
|
||||
const result: OpenAIMessage[] = [];
|
||||
messages.forEach((message, index) => {
|
||||
for (const converted of agentToOpenAI([message])) {
|
||||
result.push({
|
||||
...converted,
|
||||
_headroomMeta: { ...(converted._headroomMeta ?? {}), [SOURCE_INDEX_KEY]: index },
|
||||
});
|
||||
}
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Map the proxy's result back onto the AgentMessages OpenClaw passed in.
|
||||
*
|
||||
* `openAIToAgent(agentToOpenAI(m))` is lossy: it drops `thinking` blocks and flattens user and tool content
|
||||
* blocks into one string. Applying it to every message whenever anything is compressed rewrites the whole
|
||||
* history, which invalidates the provider's prompt cache from the first message on every compressing turn.
|
||||
* So a message the proxy returned unchanged is passed through exactly as it came in, and a message it did
|
||||
* change keeps every original field and block except the text that compression replaced.
|
||||
*
|
||||
* `sent` must be the `agentToOpenAIIndexed(originals)` output that was given to the proxy.
|
||||
*/
|
||||
export function restoreAgentMessages(
|
||||
originals: any[],
|
||||
sent: OpenAIMessage[],
|
||||
returned: OpenAIMessage[],
|
||||
): any[] {
|
||||
const sentByIndex = new Map<number, OpenAIMessage>();
|
||||
for (const message of sent) {
|
||||
const index = sourceIndexOf(message);
|
||||
if (index !== undefined) sentByIndex.set(index, message);
|
||||
}
|
||||
|
||||
const result: any[] = [];
|
||||
for (const message of returned) {
|
||||
const index = sourceIndexOf(message);
|
||||
const original = index === undefined ? undefined : originals[index];
|
||||
const before = index === undefined ? undefined : sentByIndex.get(index);
|
||||
if (original === undefined || before === undefined) {
|
||||
// Not one of ours (e.g. a message the proxy added): convert it the way it always was.
|
||||
result.push(...openAIToAgent([withoutSourceIndex(message)]));
|
||||
} else if (sameWireMessage(before, message)) {
|
||||
result.push(normalizeAgentMessage(original));
|
||||
} else {
|
||||
result.push(applyCompressedMessage(original, message));
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function sourceIndexOf(message: OpenAIMessage): number | undefined {
|
||||
const index = message._headroomMeta?.[SOURCE_INDEX_KEY];
|
||||
return typeof index === "number" ? index : undefined;
|
||||
}
|
||||
|
||||
function withoutSourceIndex(message: OpenAIMessage): OpenAIMessage {
|
||||
if (!message._headroomMeta || !(SOURCE_INDEX_KEY in message._headroomMeta)) return message;
|
||||
const { [SOURCE_INDEX_KEY]: _index, ...meta } = message._headroomMeta;
|
||||
return { ...message, _headroomMeta: meta };
|
||||
}
|
||||
|
||||
function sameWireMessage(a: OpenAIMessage, b: OpenAIMessage): boolean {
|
||||
return (
|
||||
a.role === b.role &&
|
||||
a.content === b.content &&
|
||||
a.tool_call_id === b.tool_call_id &&
|
||||
a.name === b.name &&
|
||||
JSON.stringify(a.tool_calls ?? null) === JSON.stringify(b.tool_calls ?? null)
|
||||
);
|
||||
}
|
||||
|
||||
function applyCompressedMessage(original: any, compressed: OpenAIMessage): any {
|
||||
const base = normalizeAgentMessage(original);
|
||||
const text = typeof compressed.content === "string" ? compressed.content : "";
|
||||
|
||||
if (base.role === "assistant") {
|
||||
let content = replaceTextBlocks(base.content, text);
|
||||
if (Array.isArray(compressed.tool_calls)) {
|
||||
content = content.map((block: any) => withCompressedArguments(block, compressed.tool_calls!));
|
||||
}
|
||||
return { ...base, content };
|
||||
}
|
||||
|
||||
if (typeof base.content === "string") return { ...base, content: text };
|
||||
return { ...base, content: replaceTextBlocks(base.content, text) };
|
||||
}
|
||||
|
||||
/** Replace the text blocks with one block holding `text`, where the first one was; keep every other block. */
|
||||
function replaceTextBlocks(content: unknown, text: string): any[] {
|
||||
const blocks = Array.isArray(content) ? content : [{ type: "text", text: String(content ?? "") }];
|
||||
const result: any[] = [];
|
||||
let placed = false;
|
||||
for (const block of blocks) {
|
||||
const isText = typeof block === "string" || (isRecord(block) && block.type === "text");
|
||||
if (!isText) {
|
||||
result.push(block);
|
||||
continue;
|
||||
}
|
||||
if (!placed) {
|
||||
if (text) result.push(isRecord(block) ? { ...block, text } : { type: "text", text });
|
||||
placed = true;
|
||||
}
|
||||
}
|
||||
if (!placed && text) result.unshift({ type: "text", text });
|
||||
return result;
|
||||
}
|
||||
|
||||
function withCompressedArguments(block: any, toolCalls: any[]): any {
|
||||
if (!isRecord(block) || (block.type !== "toolCall" && block.type !== "tool_use")) return block;
|
||||
const call = toolCalls.find((tc) => tc?.id === block.id);
|
||||
const raw = call?.function?.arguments;
|
||||
if (typeof raw !== "string") return block;
|
||||
const current = block.type === "toolCall" ? block.arguments : block.input;
|
||||
if ((typeof current === "string" ? current : JSON.stringify(current ?? {})) === raw) return block;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
parsed = raw;
|
||||
}
|
||||
return block.type === "toolCall" ? { ...block, arguments: parsed } : { ...block, input: parsed };
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract text from content blocks.
|
||||
*/
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
|
||||
import { compress } from "headroom-ai";
|
||||
import { ProxyManager, defaultLogger, type ProxyManagerConfig, type ProxyManagerLogger } from "./proxy-manager.js";
|
||||
import { agentToOpenAI, normalizeAgentMessages, openAIToAgent } from "./convert.js";
|
||||
import { agentToOpenAIIndexed, normalizeAgentMessages, restoreAgentMessages } from "./convert.js";
|
||||
import { DurableAdvancementKeyStore, defaultCommitLogPath } from "./advancement-key-store.js";
|
||||
import {
|
||||
delegateCompactionToRuntime,
|
||||
@@ -114,7 +114,9 @@ export class HeadroomContextEngine {
|
||||
/**
|
||||
* Assemble context for the model — THE CORE HOOK.
|
||||
*
|
||||
* Converts AgentMessage[] → OpenAI format → compress() → AgentMessage[]
|
||||
* Converts AgentMessage[] → OpenAI format → compress() → AgentMessage[]. Only messages the proxy
|
||||
* actually changed are rebuilt; the rest are returned exactly as OpenClaw passed them, so the
|
||||
* provider prompt cache survives up to the first compressed message (see restoreAgentMessages).
|
||||
*/
|
||||
async assemble(params: {
|
||||
sessionId: string;
|
||||
@@ -140,7 +142,7 @@ export class HeadroomContextEngine {
|
||||
|
||||
try {
|
||||
// Convert AgentMessage → OpenAI format
|
||||
const openaiMessages = agentToOpenAI(params.messages);
|
||||
const openaiMessages = agentToOpenAIIndexed(params.messages);
|
||||
|
||||
// Compress via proxy — pass tokenBudget so RollingWindow enforces it
|
||||
const result = await withTimeout(
|
||||
@@ -162,7 +164,7 @@ export class HeadroomContextEngine {
|
||||
}
|
||||
|
||||
// Convert back to AgentMessage format
|
||||
const compressedAgentMessages = openAIToAgent(result.messages);
|
||||
const compressedAgentMessages = restoreAgentMessages(params.messages, openaiMessages, result.messages);
|
||||
this.resetCircuit();
|
||||
|
||||
// Track stats
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const mocked = vi.hoisted(() => ({
|
||||
compress: vi.fn(),
|
||||
start: vi.fn(async () => "http://127.0.0.1:8787"),
|
||||
stop: vi.fn(async () => undefined),
|
||||
logger: { debug: vi.fn(), error: vi.fn(), info: vi.fn(), warn: vi.fn() },
|
||||
}));
|
||||
|
||||
vi.mock("headroom-ai", () => ({ compress: mocked.compress }));
|
||||
|
||||
vi.mock("../src/proxy-manager.js", () => ({
|
||||
ProxyManager: class {
|
||||
start = mocked.start;
|
||||
stop = mocked.stop;
|
||||
},
|
||||
defaultLogger: mocked.logger,
|
||||
}));
|
||||
|
||||
import { normalizeAgentMessages } from "../src/convert.js";
|
||||
import { HeadroomContextEngine } from "../src/engine.js";
|
||||
|
||||
afterEach(() => {
|
||||
mocked.compress.mockReset();
|
||||
});
|
||||
|
||||
// An OpenClaw-shaped history: a multi-block user message, an assistant turn with a signed thinking block
|
||||
// and a tool call, a large tool result, the final answer, and the next user message.
|
||||
function history() {
|
||||
return [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "[Slack DM from tal]" },
|
||||
{ type: "text", text: "list the pods" },
|
||||
],
|
||||
timestamp: 1,
|
||||
senderId: "U123",
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "thinking", thinking: "run kubectl", thinkingSignature: "sig-1" },
|
||||
{ type: "text", text: "Running it." },
|
||||
{ type: "toolCall", id: "t1", name: "exec", arguments: { command: "kubectl get pods -A -o json" } },
|
||||
],
|
||||
api: "bedrock-converse-stream",
|
||||
provider: "amazon-bedrock",
|
||||
model: "claude",
|
||||
stopReason: "toolUse",
|
||||
timestamp: 2,
|
||||
},
|
||||
{
|
||||
role: "toolResult",
|
||||
toolCallId: "t1",
|
||||
toolName: "exec",
|
||||
content: [{ type: "text", text: "[" + '{"ns":"argocd","name":"a"},'.repeat(200) + "]" }],
|
||||
isError: false,
|
||||
timestamp: 3,
|
||||
details: { exitCode: 0 },
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "thinking", thinking: "count them", thinkingSignature: "sig-2" },
|
||||
{ type: "text", text: "There are 200 pods." },
|
||||
],
|
||||
api: "bedrock-converse-stream",
|
||||
provider: "amazon-bedrock",
|
||||
model: "claude",
|
||||
stopReason: "stop",
|
||||
timestamp: 4,
|
||||
},
|
||||
{ role: "user", content: [{ type: "text", text: "which namespace?" }], timestamp: 5 },
|
||||
];
|
||||
}
|
||||
|
||||
// A proxy that compresses only tool results, the way the real /v1/compress does for JSON tool output:
|
||||
// same messages, same order, `_headroomMeta` echoed back unchanged.
|
||||
function compressToolResults(messages: any[]) {
|
||||
return {
|
||||
compressed: true,
|
||||
tokensBefore: 1000,
|
||||
tokensAfter: 400,
|
||||
tokensSaved: 600,
|
||||
messages: messages.map((m) => (m.role === "tool" ? { ...m, content: "[compressed]" } : m)),
|
||||
};
|
||||
}
|
||||
|
||||
function engine() {
|
||||
const e = new HeadroomContextEngine();
|
||||
(e as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
|
||||
return e;
|
||||
}
|
||||
|
||||
describe("assemble() keeps the provider prompt cache", () => {
|
||||
it("returns every message it did not compress exactly as OpenClaw passed it in", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
|
||||
const input = history();
|
||||
const expected = normalizeAgentMessages(history());
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: input });
|
||||
|
||||
expect(messages).toHaveLength(5);
|
||||
for (const i of [0, 1, 3, 4]) expect(messages[i]).toEqual(expected[i]);
|
||||
// The two losses the old full round trip caused, on messages nothing was compressed in:
|
||||
expect(messages[0].content).toEqual(history()[0].content); // content blocks not flattened
|
||||
expect(messages[1].content.map((b: any) => b.type)).toEqual(["thinking", "text", "toolCall"]); // thinking kept
|
||||
expect(messages[3].content.map((b: any) => b.type)).toEqual(["thinking", "text"]);
|
||||
});
|
||||
|
||||
it("keeps the history before the first compressed message byte-identical", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
|
||||
|
||||
expect(JSON.stringify(messages.slice(0, 2))).toBe(JSON.stringify(normalizeAgentMessages(history()).slice(0, 2)));
|
||||
});
|
||||
|
||||
it("replaces only the text of a compressed tool result and keeps its other fields and blocks", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
|
||||
const input = history();
|
||||
input[2].content.push({ type: "image", data: "aGk=", mimeType: "image/png" } as any);
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: input });
|
||||
|
||||
expect(messages[2]).toEqual({
|
||||
...normalizeAgentMessages([input[2]])[0],
|
||||
content: [
|
||||
{ type: "text", text: "[compressed]" },
|
||||
{ type: "image", data: "aGk=", mimeType: "image/png" },
|
||||
],
|
||||
});
|
||||
expect(messages[2].details).toEqual({ exitCode: 0 });
|
||||
});
|
||||
|
||||
it("follows a result the proxy shortened to fit the token budget", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => {
|
||||
const r = compressToolResults(messages);
|
||||
return { ...r, messages: r.messages.slice(2) }; // rolling window dropped the two oldest
|
||||
});
|
||||
const expected = normalizeAgentMessages(history());
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
|
||||
|
||||
expect(messages).toHaveLength(3);
|
||||
expect(messages[0].toolCallId).toBe("t1");
|
||||
expect(messages[0].content).toEqual([{ type: "text", text: "[compressed]" }]);
|
||||
expect(messages[1]).toEqual(expected[3]);
|
||||
expect(messages[2]).toEqual(expected[4]);
|
||||
});
|
||||
|
||||
it("converts a message the proxy added itself the same way as before", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => {
|
||||
const r = compressToolResults(messages);
|
||||
return { ...r, messages: [{ role: "user", content: "[3 earlier messages omitted]" }, ...r.messages] };
|
||||
});
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
|
||||
|
||||
expect(messages).toHaveLength(6);
|
||||
expect(messages[0]).toMatchObject({ role: "user", content: "[3 earlier messages omitted]" });
|
||||
});
|
||||
|
||||
it("applies compressed tool-call arguments without dropping the assistant's thinking", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => ({
|
||||
...compressToolResults(messages),
|
||||
messages: messages.map((m) =>
|
||||
m.role === "assistant" && m.tool_calls
|
||||
? { ...m, tool_calls: [{ ...m.tool_calls[0], function: { ...m.tool_calls[0].function, arguments: '{"command":"kubectl get pods"}' } }] }
|
||||
: m,
|
||||
),
|
||||
}));
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
|
||||
|
||||
expect(messages[1].content.map((b: any) => b.type)).toEqual(["thinking", "text", "toolCall"]);
|
||||
expect(messages[1].content[2].arguments).toEqual({ command: "kubectl get pods" });
|
||||
});
|
||||
|
||||
it("never leaks its bookkeeping into the messages it returns", async () => {
|
||||
mocked.compress.mockImplementation(async (messages: any[]) => compressToolResults(messages));
|
||||
|
||||
const { messages } = await engine().assemble({ sessionId: "s", messages: history() });
|
||||
|
||||
expect(JSON.stringify(messages)).not.toContain("headroomSourceIndex");
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user