Files
headroom/plugins/openclaw/test/engine.test.ts
Parideboy 1372f95524 fix(openclaw): let the circuit breaker open on proxy failures (#4755)
## Description

`assemble()` in the openclaw plugin passed `fallback: true` to the SDK's
`compress()`. In `sdk/typescript/src/client.ts`, that flag makes a
connection error, a timeout or a 5xx retry once and then **resolve** to
`makeFallbackResult(messages)` — `tokensBefore: 0`, `tokensSaved: 0`,
`compressed: false` — instead of throwing.

That result never reached the `catch` that calls `tripCircuit()`. Worse,
`assemble()` treated it as a successful no-op and called
`resetCircuit()`, so the error counter went back to zero on every turn.
Against a proxy that accepts connections but never answers, the circuit
breaker could never open: every `assemble()` paid up to `2 ×
requestTimeoutMs` and then bypassed compression anyway.

Closes #4738

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Pass `fallback: false` on the plugin's `compress()` call, so proxy
failures throw into the existing `catch` → `tripCircuit()`. Graceful
degradation is unchanged: that `catch` already returns the original
messages.
- Treat a zero `tokensBefore` on **nonempty** input as a failure too, so
a result that resolved empty cannot keep the breaker closed either.
Empty input is exempt, because the proxy legitimately answers
`tokens_before: 0` for an empty `messages` array — caught in review, see
the `minContextChars: 0` case below.
- `README.md`: the post-cool-down turn does not call `/health`; nothing
in the engine ever did. It re-probes by attempting a real compression.
Fixed, and the threshold row now says which failures count.

## Testing

- [ ] Unit tests pass (`pytest`) — N/A, this is the TypeScript plugin;
ran `vitest` instead
- [ ] Linting passes (`ruff check .`) — N/A, no Python changed
- [ ] Type checking passes (`mypy headroom`) — N/A, no Python changed
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ npx vitest run test/engine.test.ts
 Test Files  1 passed (1)
      Tests  41 passed (41)

$ npm run typecheck
> node node_modules/@typescript/native/bin/tsc --noEmit
(clean)

$ npm run build
ESM Build success in 18ms
DTS Build success in 515ms
```

The three new tests each fail on the unpatched tree (verified by
stashing only `src/engine.ts` and re-running):

```text
× passes fallback: false so proxy failures reach the circuit breaker
× opens the circuit on zero-token results, even if the SDK swallows the error
× does not count an empty context as a failure
```

Full suite: 158 passed, 2 failed — both pre-existing and unrelated, in
`test/advancement-key-store.test.ts`, caused by this machine's Node
rejecting `--experimental-transform-types` in the child-process fixture
(`node: bad option: --experimental-transform-types`). They fail
identically on `upstream/main` without this patch.

## Real Behavior Proof

- Environment: Windows 11, Node v26.10.0, `plugins/openclaw` built with
`npm run build`, in-tree `headroom-ai@0.42.0`. Stub proxy is a
`node:http` server that accepts the connection and never responds.
Engine constructed with `requestTimeoutMs: 500`,
`circuitBreakerThreshold: 3`, `circuitBreakerCooldownMs: 60000`,
`minContextChars: 0`, driven through five `assemble()` calls with a
2000-char message.
- Exact command / steps: `npm run build` in `plugins/openclaw`, then
`node proof-circuit.mjs`, which starts the hung proxy on an ephemeral
port, points the engine at it, and makes five sequential `assemble()`
calls. Run once on the unpatched build and once on the patched build.
- Observed result: before the patch the breaker never opened and every
call paid the full timeout. After the patch it opens after the third
failure and calls 4 and 5 skip the proxy entirely (0 ms, 0 requests).
Messages returned are the originals in every case, so model output is
unchanged; only the latency is removed.

Before the patch:

```text
call 1: ms=1004 requestsSeenByProxy=2 totalProxyRequests=2
call 2: ms=1001 requestsSeenByProxy=2 totalProxyRequests=4
call 3: ms=1003 requestsSeenByProxy=2 totalProxyRequests=6
call 4: ms=1001 requestsSeenByProxy=2 totalProxyRequests=8
call 5: ms=1002 requestsSeenByProxy=2 totalProxyRequests=10
circuit-opened=false
```

After the patch:

```text
call 1: ms=1004 requestsSeenByProxy=2 totalProxyRequests=2 messagesAreOriginal=true
call 2: ms=1002 requestsSeenByProxy=2 totalProxyRequests=4 messagesAreOriginal=true
call 3: ms=1002 requestsSeenByProxy=2 totalProxyRequests=6 messagesAreOriginal=true
call 4: ms=0 requestsSeenByProxy=0 totalProxyRequests=6 messagesAreOriginal=true
call 5: ms=0 requestsSeenByProxy=0 totalProxyRequests=6 messagesAreOriginal=true
circuit-opened=true
  warn: [headroom] Circuit breaker opened after 3 errors (last: HeadroomConnectionError: Failed to connect to Headroom at http://127.0.0.1:53545: TimeoutError: The operation was aborted due to timeout); bypassing compression for 60000ms
  warn: [headroom] Circuit open — using uncompressed messages
  warn: [headroom] Circuit open — using uncompressed messages
```

- Not tested: a live OpenClaw host with a real provider (the engine
needs a host to drive, so I drove `assemble()` directly against the
built plugin), and the published `headroom-ai` beyond `0.42.0`.
`fallback` has been in the SDK's `CompressOptions` since the first TS
release, so the flag resolves against the plugin's `^0.40.0` range too.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — this is a plugin-local circuit
breaker inside `headroom-openclaw`, with no rollout flag or channel
gating involved.
- Minimum rollout channel: N/A, no gating is introduced.
- Stable/default behavior changed: yes, deliberately. The circuit
breaker now opens on proxy failures instead of never opening, so an
unhealthy proxy causes compression to be skipped for
`circuitBreakerCooldownMs` (default 60 s) rather than being attempted on
every turn. Messages are returned unchanged either way, so model output
is unaffected; the change removes repeated timeout latency.
- Kill switch / disable path: `enabled: false` disables the plugin
entirely; `circuitBreakerThreshold` can be raised to make the breaker
effectively unreachable; `circuitBreakerCooldownMs` bounds how long each
bypass lasts.
- Unsafe override required: no.
- Qualification impact: none — no provider, transport or config surface
changes, and no new dependency.
- Rollback path: revert the commit. The prior behavior was a single-line
`fallback: true`, and nothing here migrates or persists state, so
reverting restores the old behavior exactly.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Review caught one defect in my first attempt, now fixed and covered by a
test: with `minContextChars: 0`, `assemble()` forwards empty `messages:
[]` to the proxy, which answers `200` with `tokens_before: 0`. Counting
that as a failure meant a handful of empty-context turns could open the
breaker and then skip compression for a healthy proxy. The zero-token
check is now scoped to nonempty input.

Found with AI assistance (opencode) operated by me.
2026-11-10 12:34:14 -06:00

799 lines
30 KiB
TypeScript

import * as fs from "node:fs/promises";
import * as os from "node:os";
import * as path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const mocked = vi.hoisted(() => ({
compress: vi.fn(),
delegateCompactionToRuntime: vi.fn(),
start: vi.fn(async () => "http://127.0.0.1:8787"),
stop: vi.fn(async () => undefined),
logger: {
debug: vi.fn(),
error: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
},
}));
vi.mock("headroom-ai", () => ({
compress: mocked.compress,
}));
vi.mock("../src/openclaw-compaction.js", () => ({
delegateCompactionToRuntime: mocked.delegateCompactionToRuntime,
}));
vi.mock("../src/proxy-manager.js", () => ({
ProxyManager: class {
start = mocked.start;
stop = mocked.stop;
},
defaultLogger: mocked.logger,
}));
import { HeadroomContextEngine, type HeadroomEngineConfig } from "../src/engine.js";
import { compress } from "headroom-ai";
afterEach(() => {
mocked.compress.mockReset();
mocked.delegateCompactionToRuntime.mockReset();
mocked.start.mockReset();
mocked.start.mockResolvedValue("http://127.0.0.1:8787");
mocked.stop.mockClear();
mocked.logger.debug.mockClear();
mocked.logger.error.mockClear();
mocked.logger.info.mockClear();
mocked.logger.warn.mockClear();
});
describe("HeadroomContextEngine compaction", () => {
it("delegates persistent compaction to OpenClaw without claiming ownership", async () => {
const engine = new HeadroomContextEngine();
const params = {
sessionId: "session-1",
sessionKey: "agent:main:session-1",
tokenBudget: 12_000,
force: true,
runtimeContext: { workspaceDir: "/tmp/workspace" },
};
const delegatedResult = {
ok: true,
compacted: true,
result: {
tokensBefore: 20_000,
tokensAfter: 8_000,
},
};
mocked.delegateCompactionToRuntime.mockResolvedValueOnce(delegatedResult);
expect(engine.info.ownsCompaction).toBe(false);
await expect(engine.compact(params)).resolves.toEqual(delegatedResult);
expect(mocked.delegateCompactionToRuntime).toHaveBeenCalledWith(params);
expect(mocked.compress).not.toHaveBeenCalled();
expect(engine.getStats().compactions).toBe(1);
});
it("does not count a delegated no-op as a compaction", async () => {
const engine = new HeadroomContextEngine();
mocked.delegateCompactionToRuntime.mockResolvedValueOnce({
ok: true,
compacted: false,
reason: "Below compaction threshold",
});
await expect(
engine.compact({
sessionId: "session-1",
sessionKey: "agent:main:session-1",
}),
).resolves.toEqual({
ok: true,
compacted: false,
reason: "Below compaction threshold",
});
expect(engine.getStats().compactions).toBe(0);
});
it("propagates delegated compaction failures without reporting success", async () => {
const engine = new HeadroomContextEngine();
const failure = new Error("native compaction failed");
mocked.delegateCompactionToRuntime.mockRejectedValueOnce(failure);
await expect(
engine.compact({
sessionId: "session-1",
sessionKey: "agent:main:session-1",
}),
).rejects.toBe(failure);
expect(engine.getStats().compactions).toBe(0);
expect(mocked.logger.info).not.toHaveBeenCalled();
});
});
describe("HeadroomContextEngine proxy startup helpers", () => {
it("bootstraps by scheduling proxy startup when enabled", async () => {
const engine = new HeadroomContextEngine();
await expect(
engine.bootstrap({
sessionId: "session-1",
sessionFile: "session.jsonl",
}),
).resolves.toEqual({
bootstrapped: true,
reason: "proxy startup scheduled",
});
expect(mocked.start).toHaveBeenCalledTimes(1);
});
it("removes unsubscribed proxy listeners before notifying readiness", async () => {
const engine = new HeadroomContextEngine();
const first = vi.fn();
const second = vi.fn();
const unsubscribeFirst = engine.onProxyReady(first);
engine.onProxyReady(second);
unsubscribeFirst();
engine.ensureProxyStarted();
await engine.ensureProxyUrl();
expect(first).not.toHaveBeenCalled();
expect(second).toHaveBeenCalledWith("http://127.0.0.1:8787");
});
it("returns the existing proxy URL without starting again", async () => {
const engine = new HeadroomContextEngine();
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await expect(engine.ensureProxyUrl()).resolves.toBe("http://127.0.0.1:8787");
expect(mocked.start).not.toHaveBeenCalled();
});
it("throws when proxy startup is disabled", async () => {
const engine = new HeadroomContextEngine({ enabled: false });
await expect(engine.ensureProxyUrl()).rejects.toThrow("Headroom proxy startup is disabled");
expect(mocked.start).not.toHaveBeenCalled();
});
it("does not emit an unhandledRejection when fire-and-forget startup fails", async () => {
mocked.start.mockReset();
mocked.start.mockRejectedValue(new Error("proxy boom"));
const engine = new HeadroomContextEngine();
const unhandled: unknown[] = [];
const onUnhandled = (reason: unknown) => unhandled.push(reason);
process.on("unhandledRejection", onUnhandled);
try {
// Fire-and-forget: caller intentionally does not await.
engine.ensureProxyStarted();
// Let the startup promise settle and any microtasks/macrotasks flush.
await new Promise((resolve) => setTimeout(resolve, 0));
expect(unhandled).toEqual([]);
expect(mocked.logger.warn).toHaveBeenCalledWith(
expect.stringContaining("Headroom proxy unavailable"),
);
} finally {
process.off("unhandledRejection", onUnhandled);
}
});
it("stores the startup failure in getProxyStartupError()", async () => {
const failure = new Error("proxy boom");
mocked.start.mockReset();
mocked.start.mockRejectedValue(failure);
const engine = new HeadroomContextEngine();
expect(engine.getProxyStartupError()).toBeNull();
engine.ensureProxyStarted();
await new Promise((resolve) => setTimeout(resolve, 0));
expect(engine.getProxyStartupError()).toBe(failure);
});
it("allows retrying startup after a failure", async () => {
mocked.start.mockReset();
mocked.start
.mockRejectedValueOnce(new Error("proxy boom"))
.mockResolvedValueOnce("http://127.0.0.1:8787");
const engine = new HeadroomContextEngine();
engine.ensureProxyStarted();
await new Promise((resolve) => setTimeout(resolve, 0));
expect(engine.getProxyStartupError()).toBeInstanceOf(Error);
// A second attempt is possible once the failed promise has cleared.
const url = await engine.ensureProxyUrl();
expect(url).toBe("http://127.0.0.1:8787");
expect(engine.getProxyStartupError()).toBeNull();
expect(mocked.start).toHaveBeenCalledTimes(2);
});
it("ensureProxyUrl rejects cleanly on startup failure without unhandledRejection", async () => {
const failure = new Error("proxy boom");
mocked.start.mockReset();
mocked.start.mockRejectedValue(failure);
const engine = new HeadroomContextEngine();
const unhandled: unknown[] = [];
const onUnhandled = (reason: unknown) => unhandled.push(reason);
process.on("unhandledRejection", onUnhandled);
try {
await expect(engine.ensureProxyUrl()).rejects.toBe(failure);
await new Promise((resolve) => setTimeout(resolve, 0));
expect(unhandled).toEqual([]);
} finally {
process.off("unhandledRejection", onUnhandled);
}
});
it("isolates and logs proxy-ready listener rejections", async () => {
const engine = new HeadroomContextEngine();
const failing = vi.fn(async () => {
throw new Error("listener boom");
});
const healthy = vi.fn();
engine.onProxyReady(failing);
engine.onProxyReady(healthy);
engine.ensureProxyStarted();
// ensureProxyUrl must still resolve despite the listener throwing.
await expect(engine.ensureProxyUrl()).resolves.toBe("http://127.0.0.1:8787");
expect(failing).toHaveBeenCalled();
expect(healthy).toHaveBeenCalledWith("http://127.0.0.1:8787");
expect(mocked.logger.warn).toHaveBeenCalledWith(
expect.stringContaining("Headroom proxy ready listener failed"),
);
expect(engine.getProxyStartupError()).toBeNull();
});
it("schedules startup and returns original messages when assembling before proxy readiness", async () => {
const engine = new HeadroomContextEngine();
const messages = [{ role: "user", content: "hello" }];
await expect(
engine.assemble({
sessionId: "session-1",
messages,
}),
).resolves.toEqual({
messages,
estimatedTokens: 0,
});
expect(mocked.start).toHaveBeenCalledTimes(1);
});
it("passes the configured request timeout to the aborting SDK client", async () => {
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [{ role: "user", content: "hello" }],
tokensBefore: 5,
tokensAfter: 5,
tokensSaved: 0,
});
const engine = new HeadroomContextEngine({ requestTimeoutMs: 1_234, minContextChars: 0 });
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({
sessionId: "session-1",
messages: [{ role: "user", content: "hello" }],
});
expect(compress).toHaveBeenCalledWith(
expect.any(Array),
expect.objectContaining({ timeout: 1_234 }),
);
});
it("skips proxy compression for short contexts with no meaningful savings", async () => {
const messages = [{ role: "user", content: "hello" }];
const engine = new HeadroomContextEngine();
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await expect(engine.assemble({ sessionId: "session-1", messages })).resolves.toEqual({
messages,
estimatedTokens: 0,
});
expect(compress).not.toHaveBeenCalled();
});
it("counts a single large tool result as eligible context", async () => {
const messages = [{ role: "toolResult", toolCallId: "call-1", content: "x".repeat(800) }];
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [{ role: "tool", tool_call_id: "call-1", content: "x".repeat(800) }],
tokensBefore: 200,
tokensAfter: 200,
tokensSaved: 0,
});
const engine = new HeadroomContextEngine();
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages });
expect(compress).toHaveBeenCalledTimes(1);
});
it("allows the short-context gate to be disabled", async () => {
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [{ role: "user", content: "hello" }],
tokensBefore: 2,
tokensAfter: 2,
tokensSaved: 0,
});
const engine = new HeadroomContextEngine({ minContextChars: 0 });
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages: [{ role: "user", content: "hello" }] });
expect(compress).toHaveBeenCalledTimes(1);
});
it("opens the circuit after consecutive compression failures", async () => {
vi.mocked(compress).mockRejectedValue(new Error("proxy stalled"));
const messages = [{ role: "user", content: "hello" }];
const engine = new HeadroomContextEngine({
minContextChars: 0,
circuitBreakerThreshold: 2,
circuitBreakerCooldownMs: 60_000,
});
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages });
await engine.assemble({ sessionId: "session-1", messages });
await expect(engine.assemble({ sessionId: "session-1", messages })).resolves.toEqual({
messages,
estimatedTokens: 0,
});
expect(compress).toHaveBeenCalledTimes(2);
expect(mocked.logger.warn).toHaveBeenCalledWith(
expect.stringContaining("Circuit breaker opened"),
);
});
// Regression: engine.ts used to pass `fallback: true`, so the SDK resolved a
// connection error, timeout or 5xx to a zero-token stub instead of throwing.
// That stub looked like a successful no-op, so every assemble() reset the
// breaker and it could never open on an unhealthy proxy.
it("passes fallback: false so proxy failures reach the circuit breaker", async () => {
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [{ role: "user", content: "hello" }],
tokensBefore: 0,
tokensAfter: 0,
tokensSaved: 0,
});
const engine = new HeadroomContextEngine({ minContextChars: 0 });
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages: [{ role: "user", content: "hello" }] });
expect(vi.mocked(compress).mock.calls[0][1]).toMatchObject({ fallback: false });
});
it("opens the circuit on zero-token results, even if the SDK swallows the error", async () => {
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [{ role: "user", content: "hello" }],
tokensBefore: 0,
tokensAfter: 0,
tokensSaved: 0,
});
const messages = [{ role: "user", content: "hello" }];
const engine = new HeadroomContextEngine({
minContextChars: 0,
circuitBreakerThreshold: 2,
circuitBreakerCooldownMs: 60_000,
});
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages });
await engine.assemble({ sessionId: "session-1", messages });
await expect(engine.assemble({ sessionId: "session-1", messages })).resolves.toEqual({
messages,
estimatedTokens: 0,
});
// The third call is served from the open circuit, not the proxy.
expect(compress).toHaveBeenCalledTimes(2);
expect(mocked.logger.warn).toHaveBeenCalledWith(
expect.stringContaining("Circuit breaker opened"),
);
});
it("does not count an empty context as a failure", async () => {
// The proxy answers 200 with tokens_before: 0 for an empty message list, so
// zero tokens there is a real answer, not a failure.
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [],
tokensBefore: 0,
tokensAfter: 0,
tokensSaved: 0,
});
const engine = new HeadroomContextEngine({
minContextChars: 0,
circuitBreakerThreshold: 1,
});
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages: [] });
await engine.assemble({ sessionId: "session-1", messages: [] });
// The breaker must stay closed, so the next call still reaches the proxy.
expect(compress).toHaveBeenCalledTimes(2);
expect(mocked.logger.warn).not.toHaveBeenCalledWith(
expect.stringContaining("Circuit breaker opened"),
);
});
it("does not count a genuine no-savings result as a failure", async () => {
vi.mocked(compress).mockResolvedValue({
compressed: false,
messages: [{ role: "user", content: "hello" }],
tokensBefore: 5,
tokensAfter: 5,
tokensSaved: 0,
});
const engine = new HeadroomContextEngine({
minContextChars: 0,
circuitBreakerThreshold: 1,
});
(engine as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
await engine.assemble({ sessionId: "session-1", messages: [{ role: "user", content: "hello" }] });
await engine.assemble({ sessionId: "session-1", messages: [{ role: "user", content: "hello" }] });
// Nothing was saved, but the proxy answered — the breaker must stay closed.
expect(compress).toHaveBeenCalledTimes(2);
expect(mocked.logger.warn).not.toHaveBeenCalledWith(
expect.stringContaining("Circuit breaker opened"),
);
});
});
describe("HeadroomContextEngine transcriptSemantics contract", () => {
let commitLogDir: string;
let commitLogPath: string;
beforeEach(async () => {
commitLogDir = await fs.mkdtemp(path.join(os.tmpdir(), "headroom-commit-log-"));
commitLogPath = path.join(commitLogDir, "commit-log.json");
});
afterEach(async () => {
await fs.rm(commitLogDir, { recursive: true, force: true });
});
it("declares the durable-commit transcript semantics OpenClaw requires", () => {
const engine = new HeadroomContextEngine({ commitLogPath });
expect(engine.info.transcriptSemantics).toEqual({
currentTurnFence: "before-current-turn-entry-v1",
turnAdvancementIdempotency: "atomic-idempotent-v1",
});
});
it("commits a new advancement key", async () => {
const engine = new HeadroomContextEngine({ commitLogPath });
await expect(
engine.commitTurn({ advancementKey: "turn-1", messages: [] }),
).resolves.toEqual({ status: "committed" });
});
it("reports duplicate on a retried advancement key", async () => {
const engine = new HeadroomContextEngine({ commitLogPath });
await expect(
engine.commitTurn({ advancementKey: "turn-1", messages: [] }),
).resolves.toEqual({ status: "committed" });
await expect(
engine.commitTurn({ advancementKey: "turn-1", messages: [] }),
).resolves.toEqual({ status: "duplicate" });
});
it("treats distinct advancement keys independently", async () => {
const engine = new HeadroomContextEngine({ commitLogPath });
await expect(
engine.commitTurn({ advancementKey: "turn-1", messages: [] }),
).resolves.toEqual({ status: "committed" });
await expect(
engine.commitTurn({ advancementKey: "turn-2", messages: [] }),
).resolves.toEqual({ status: "committed" });
});
it("reports duplicate for a key committed before a process restart", async () => {
// Simulate a restart: a brand new engine instance (no shared in-memory
// state) pointed at the same durable commit-log path.
const before = new HeadroomContextEngine({ commitLogPath });
await expect(
before.commitTurn({ advancementKey: "turn-restart", messages: [] }),
).resolves.toEqual({ status: "committed" });
const after = new HeadroomContextEngine({ commitLogPath });
await expect(
after.commitTurn({ advancementKey: "turn-restart", messages: [] }),
).resolves.toEqual({ status: "duplicate" });
});
it(
"never forgets a key regardless of how many other keys were committed since",
async () => {
// Regression: the old implementation evicted the oldest key past a
// 512-entry cap, so a retry of an early key was wrongly re-accepted as
// new instead of reported as a duplicate. There is no such cap now.
const engine = new HeadroomContextEngine({ commitLogPath });
for (let i = 0; i < 600; i++) {
await engine.commitTurn({ advancementKey: `turn-${i}`, messages: [] });
}
await expect(
engine.commitTurn({ advancementKey: "turn-0", messages: [] }),
).resolves.toEqual({ status: "duplicate" });
},
20_000,
);
it("records only the advancement key, so the store doesn't grow with the turn's size", async () => {
// Regression: every accepted turn's messages used to be kept forever in one
// file that was rewritten on each commit. OpenClaw's transcript already holds
// them, and this engine never reads them back.
const engine = new HeadroomContextEngine({ commitLogPath });
const toolOutput = "x".repeat(1_000_000);
await expect(
engine.commitTurn({
advancementKey: "turn-1",
messages: [{ role: "toolResult", content: [{ type: "text", text: toolOutput }] }],
}),
).resolves.toEqual({ status: "committed" });
let storedBytes = 0;
for (const entry of await fs.readdir(commitLogDir, { recursive: true, withFileTypes: true })) {
if (entry.isFile()) {
storedBytes += (await fs.stat(path.join(entry.parentPath, entry.name))).size;
}
}
expect(storedBytes).toBeGreaterThan(0);
expect(storedBytes).toBeLessThan(1_000);
});
});
describe("HeadroomContextEngine assemble() compression notice", () => {
const HEADROOM_COMPRESSION_NOTICE =
"[Headroom is compressing tool outputs in this session. Use headroom_retrieve if you need the original, uncompressed content.]";
const messages = [{ role: "user", content: "hello" }];
function mockCompressResult(overrides: Partial<{
compressed: boolean;
tokensSaved: number;
tokensBefore: number;
tokensAfter: number;
}>) {
const tokensSaved = overrides.tokensSaved ?? 0;
return {
compressed: overrides.compressed ?? tokensSaved > 0,
messages: [{ role: "user", content: "hello" }],
tokensBefore: overrides.tokensBefore ?? 1000,
tokensAfter: overrides.tokensAfter ?? 1000 - tokensSaved,
tokensSaved,
};
}
function readyEngine(config?: HeadroomEngineConfig) {
const engine = new HeadroomContextEngine({ minContextChars: 0, ...config });
(engine as unknown as { proxyUrl: string | null }).proxyUrl = "http://127.0.0.1:8787";
return engine;
}
it("returns the static notice, with no interpolated count, once tokensSaved crosses the threshold", async () => {
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }));
const engine = readyEngine();
const result = await engine.assemble({ sessionId: "s1", messages });
expect(result.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
});
it("returns byte-identical notices across turns with different tokensSaved amounts", async () => {
vi.mocked(compress)
.mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }))
.mockResolvedValueOnce(mockCompressResult({ tokensSaved: 9000 }));
const engine = readyEngine();
const first = await engine.assemble({ sessionId: "s1", messages });
const second = await engine.assemble({ sessionId: "s1", messages });
expect(first.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
expect(second.systemPromptAddition).toBe(first.systemPromptAddition);
});
it("keeps the notice present on a later turn that has nothing to compress", async () => {
vi.mocked(compress)
.mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }))
.mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }));
const engine = readyEngine();
const first = await engine.assemble({ sessionId: "s1", messages });
const second = await engine.assemble({ sessionId: "s1", messages });
expect(first.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
expect(second.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
});
it("keeps an earned notice on a later context below the compression threshold", async () => {
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }));
const engine = readyEngine({ minContextChars: 800 });
const first = await engine.assemble({
sessionId: "s1", messages: [{ role: "user", content: "x".repeat(1000) }],
});
const second = await engine.assemble({ sessionId: "s1", messages });
expect(first.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
expect(second.systemPromptAddition).toBe(first.systemPromptAddition);
expect(compress).toHaveBeenCalledTimes(1);
});
it("never returns a notice when announceCompression is false", async () => {
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 500 }));
const engine = readyEngine({ announceCompression: false });
const result = await engine.assemble({ sessionId: "s1", messages });
expect(result.systemPromptAddition).toBeUndefined();
});
it("does not announce for compression below the noise threshold", async () => {
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 50 }));
const engine = readyEngine();
const result = await engine.assemble({ sessionId: "s1", messages });
expect(result.systemPromptAddition).toBeUndefined();
});
it("does not leak a session's announcement to a different session on the same engine", async () => {
vi.mocked(compress)
.mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }))
.mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }));
const engine = readyEngine();
const sessionA = await engine.assemble({ sessionId: "a", messages });
const sessionB = await engine.assemble({ sessionId: "b", messages });
expect(sessionA.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
expect(sessionB.systemPromptAddition).toBeUndefined();
});
it("keeps announcing for the session that earned the notice even after another session is seen", async () => {
vi.mocked(compress)
.mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }))
.mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }))
.mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }));
const engine = readyEngine();
await engine.assemble({ sessionId: "a", messages });
await engine.assemble({ sessionId: "b", messages });
const sessionAAgain = await engine.assemble({ sessionId: "a", messages });
expect(sessionAAgain.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
});
it("evicts the oldest tracked session once the announced-session bound is exceeded", async () => {
vi.mocked(compress).mockResolvedValue(mockCompressResult({ tokensSaved: 150 }));
const engine = readyEngine();
for (let i = 0; i < 1000; i++) {
await engine.assemble({ sessionId: `session-${i}`, messages });
}
const overflow = await engine.assemble({ sessionId: "session-overflow", messages });
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }));
const oldest = await engine.assemble({ sessionId: "session-0", messages });
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }));
const newest = await engine.assemble({ sessionId: "session-overflow", messages });
expect(overflow.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
expect(oldest.systemPromptAddition).toBeUndefined();
expect(newest.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
});
it("keeps the earned notice byte-identical through a timeout, a circuit-open turn, and recovery", async () => {
vi.useFakeTimers();
try {
const requestTimeoutMs = 5_000;
const circuitBreakerCooldownMs = 10_000;
const engine = readyEngine({
requestTimeoutMs,
circuitBreakerThreshold: 1,
circuitBreakerCooldownMs,
});
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }));
const turn1 = await engine.assemble({ sessionId: "s1", messages });
const firstNotice = turn1.systemPromptAddition;
expect(firstNotice).toBe(HEADROOM_COMPRESSION_NOTICE);
vi.mocked(compress).mockImplementationOnce(() => new Promise((_, reject) => {
setTimeout(() => reject(new Error("SDK request timed out")), requestTimeoutMs);
}));
const turn2Promise = engine.assemble({ sessionId: "s1", messages });
await vi.advanceTimersByTimeAsync(requestTimeoutMs);
const turn2 = await turn2Promise;
expect(turn2.systemPromptAddition).toBe(firstNotice);
const turn3 = await engine.assemble({ sessionId: "s1", messages });
expect(turn3.systemPromptAddition).toBe(firstNotice);
expect(compress).toHaveBeenCalledTimes(2);
await vi.advanceTimersByTimeAsync(circuitBreakerCooldownMs);
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ compressed: false, tokensSaved: 0 }));
const turn4 = await engine.assemble({ sessionId: "s1", messages });
expect(turn4.systemPromptAddition).toBe(firstNotice);
expect(turn4.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
} finally {
vi.useRealTimers();
}
});
it("preserves an earned notice when the proxy becomes unavailable", async () => {
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }));
const engine = readyEngine();
const first = await engine.assemble({ sessionId: "s1", messages });
expect(first.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
(engine as unknown as { proxyUrl: string | null }).proxyUrl = null;
const second = await engine.assemble({ sessionId: "s1", messages });
expect(second.systemPromptAddition).toBe(HEADROOM_COMPRESSION_NOTICE);
});
it("never surfaces a notice in any fallback path when announceCompression is false", async () => {
vi.useFakeTimers();
try {
const requestTimeoutMs = 5_000;
const engine = readyEngine({
announceCompression: false,
requestTimeoutMs,
circuitBreakerThreshold: 1,
circuitBreakerCooldownMs: 10_000,
});
vi.mocked(compress).mockResolvedValueOnce(mockCompressResult({ tokensSaved: 150 }));
const turn1 = await engine.assemble({ sessionId: "s1", messages });
expect(turn1.systemPromptAddition).toBeUndefined();
vi.mocked(compress).mockImplementationOnce(() => new Promise((_, reject) => {
setTimeout(() => reject(new Error("SDK request timed out")), requestTimeoutMs);
}));
const turn2Promise = engine.assemble({ sessionId: "s1", messages });
await vi.advanceTimersByTimeAsync(requestTimeoutMs);
const turn2 = await turn2Promise;
expect(turn2.systemPromptAddition).toBeUndefined();
const turn3 = await engine.assemble({ sessionId: "s1", messages });
expect(turn3.systemPromptAddition).toBeUndefined();
} finally {
vi.useRealTimers();
}
});
});