fix(sdk): send Anthropic messages to /v1/compress natively so thinking blocks survive (#4489)

## Description

The TypeScript SDK's Anthropic adapter converts every request to OpenAI
chat shape before `/v1/compress` and back afterwards. The converters
only model `text`, `tool_use`, `tool_result`, `image` and `document`, so
an assistant turn's **`thinking` and `redacted_thinking` blocks are
dropped**, `cache_control` markers on text blocks are lost, server-tool
blocks vanish, and a `tool_result` whose content is a block array is
flattened to text (an image inside it becomes a JSON string).
`compress()` does the same for Anthropic-format input.

Anthropic requires the thinking blocks back during a tool-use turn. From
the Thinking docs: "**Required:** within a tool-use turn, pass thinking
blocks back" and "Within the latest assistant message, the sequence of
consecutive `thinking` blocks must match what the model generated in the
original request: you can't rearrange, edit, or partially drop them.
This includes `redacted_thinking` blocks." The tool-workflows page:
"rebuilding the message or filtering out `redacted_thinking` blocks
triggers a 400 error." So a thinking-enabled tool-use turn through
`withHeadroom(anthropic)` reaches Anthropic with its `tool_use` block
but no thinking block.

Verified on `main` through the public converters (the adapter's exact
path):

```text
assistant blocks in : thinking, redacted_thinking, text, tool_use
assistant blocks out: text, tool_use
user cache_control kept: false
```

The conversion is unnecessary. Since #2743 (0.34.0) `/v1/compress` "does
no format conversion — callers send whichever wire shape they already
use"; the proxy docs list "content-block lists with `tool_use` /
`tool_result` / `thinking` blocks" among the natively compressed shapes
and say "an Anthropic-native caller does not need to convert to OpenAI
format first". The adapter page itself called the conversion "an SDK
choice, not an API requirement" while also calling it lossless. This PR
stops converting: the adapter and `compress()` send Anthropic messages
to the proxy as they are and forward what comes back.

Closes # (none — found while sweeping the converters after #4048)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`sdk/typescript/src/adapters/anthropic.ts`** — no conversion:
`compress(params.messages, …)` and forward `result.messages` (or the
original messages on fallback, as before). The converter imports go
away.
- **`sdk/typescript/src/compress.ts`** — when the detected format is
`anthropic`, the messages go to the proxy unconverted and the proxy's
messages are returned unconverted; every other format still goes through
`toOpenAI` / `fromOpenAI`. `hooks.computeBiases` receives the messages
exactly as they are sent to the proxy.
- **`sdk/typescript/src/utils/format.ts`** — `detectFormat` treats
`thinking` / `redacted_thinking` blocks as Anthropic markers (a
thinking-only turn used to fall through to the OpenAI default).
`anthropicToOpenAI` / `openAIToAnthropic` are unchanged and still
exported.
- **Tests** — `test/adapters/anthropic.test.ts` rewritten for native
pass-through: a thinking-enabled tool-use turn (user text with
`cache_control`, assistant `thinking` + `redacted_thinking` + `text` +
`tool_use`, array `tool_result` with an image) is posted unchanged and
the proxy's returned messages are forwarded unchanged;
tool_use/tool_result and image turns likewise; fallback unchanged.
`test/compress.test.ts`: native round trip for Anthropic input, and the
#4048 document-only case now asserts the native document block.
`test/utils/detect-format.test.ts`: thinking / redacted_thinking
detection.
- **`docs/content/docs/anthropic-sdk.mdx`** — "How it works" describes
the native path; the conversion table, the "This conversion is lossless"
claim and the "SDK choice" callout are replaced by what is actually
preserved.

**Not in this PR.** `simulate()` still converts Anthropic input (a dry
run; nothing reaches Anthropic). The Vercel middleware drops `reasoning`
parts and `providerOptions` the same way, and the Gemini converter
merges `thought` parts into text; both need carriers because the proxy
cannot read those shapes, so they are separate PRs.

## Testing

- [x] Unit tests pass (`npm test` — vitest, incl. typecheck)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
- [x] Manual testing performed (below)

### Test Output

```text
$ cd sdk/typescript && npm test
 Test Files  19 passed | 5 skipped (24)
      Tests  406 passed | 33 skipped (439)
Type Errors  no errors

$ npm run typecheck      # tsc --noEmit — clean
$ npm run build          # tsup — ESM, CJS and DTS built
```

## Real Behavior Proof

- Environment: published proxy `headroom-ai 0.41.0` (`uvx --from
'headroom-ai[proxy]' headroom proxy --port 8799`), this branch built
with `npm run build`, Node v22.16.0, macOS.
- Exact command / steps: one three-message conversation — a user text
block with `cache_control`, an assistant turn `[thinking,
redacted_thinking, text, tool_use]`, and a `tool_result` carrying a 72
KB JSON log — sent (a) through the public converters, which is what the
adapter did before, (b) through `withHeadroom()` wrapping a fake client
that captures the forwarded params, (c) through `compress()`.
- Observed result: before the change the conversion dropped both
thinking blocks and the `cache_control` marker; after it, the assistant
and user turns came back from the live proxy byte-identical while the
`tool_result` shrank from 72 432 to 34 916 chars (14 781 tokens saved).
Full output:

```text
== before: toOpenAI -> fromOpenAI (what the adapter used to do) ==
assistant blocks in : thinking, redacted_thinking, text, tool_use
assistant blocks out: text, tool_use
user cache_control kept: false

== after: withHeadroom(anthropic) against the live proxy ==
assistant turn byte-identical (thinking, redacted_thinking, text, tool_use): true
user turn byte-identical (cache_control kept): true
tool_result content: 72432 -> 34916 chars, still a tool_result block: true
compress(): tokensBefore=30471 tokensAfter=15690 tokensSaved=14781 transforms=["router:tool_result:mixed"]
compress() result thinking block byte-identical: true
```

The same tool result sent in OpenAI shape (`role: "tool"`) saves the
same 14 781 tokens (`router:mixed:0.39`), so the native path compresses
no less than the converted one did.
- Not tested: an end-to-end request to the Anthropic API with thinking
enabled (no key in this environment); the 400 is taken from Anthropic's
documentation quoted above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none (TypeScript SDK only, no proxy
change)
- Minimum rollout channel: n/a
- Stable/default behavior changed: for SDK users, Anthropic messages now
reach `/v1/compress` in Anthropic shape; the proxy has compressed that
shape natively since 0.34.0.
- Kill switch / disable path: unchanged — `fallback: true` forwards the
original messages when the proxy is unreachable; unwrap the client to
disable compression.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- Token counts for Claude models now come from the proxy's per-model
tokenizer registry instead of being counted on an OpenAI-shaped copy, so
`tokensBefore` / `tokensAfter` for the Anthropic adapter may differ
slightly from previous versions.
- Follow-ups, each its own PR: Vercel `reasoning` parts and
`providerOptions` (Anthropic signatures, Gemini `thoughtSignature`,
OpenAI `itemId`), and Gemini `thought` parts / `thoughtSignature`.

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Shubham Srivastava
2026-10-28 23:37:52 -05:00
committed by GitHub
co-authored by JD Davis
parent 21e8867504
commit cd5e2b9e73
7 changed files with 226 additions and 257 deletions
+9 -16
View File
@@ -41,26 +41,19 @@ Every call to `client.messages.create()` compresses messages first. The response
`withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`:
1. Converts Anthropic-format messages to OpenAI format
2. Sends them to the Headroom proxy's `/v1/compress` endpoint
3. Converts the compressed messages back to Anthropic format
4. Forwards the request to Anthropic as normal
1. Sends your `messages` to the Headroom proxy's `/v1/compress` endpoint as they are, in Anthropic format
2. Forwards the request to Anthropic with the compressed messages in place
### Message format conversion
### Message format
The adapter handles the full Anthropic message format including content blocks:
Nothing is converted. `POST /v1/compress` compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format) — and returns the same shape it received, so every block reaches Anthropic exactly as you built it:
| Anthropic format | OpenAI format |
|-----------------|---------------|
| `{ type: "text", text: "..." }` | `{ role: "user", content: "..." }` |
| `{ type: "tool_use", id, name, input }` | `{ tool_calls: [{ id, function: { name, arguments } }] }` |
| `{ type: "tool_result", tool_use_id, content }` | `{ role: "tool", tool_call_id, content }` |
- `thinking` and `redacted_thinking` blocks keep their `signature` and `data`, which Anthropic requires back unmodified within a tool-use turn
- `cache_control` markers stay on the blocks that carry them
- `tool_use` and `tool_result` blocks keep their ids; only the content of large `tool_result` blocks changes
- images, documents, citations and server-tool blocks pass through untouched
This conversion is lossless. Your request and response behave identically to an unwrapped client.
<Callout type="info" title="The conversion is an SDK choice, not an API requirement">
`POST /v1/compress` does no format conversion and compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format). If you are calling the endpoint directly (from a gateway, or LiteLLM's `headroom` guardrail), send Anthropic-shaped messages as-is; you get the same shape back. Only this TypeScript adapter converts, because it normalises on OpenAI types internally.
</Callout>
If the proxy is unreachable and `fallback` is enabled, the original messages are forwarded unchanged.
## Options
+6 -8
View File
@@ -1,6 +1,5 @@
import { compress } from "../compress.js";
import type { CompressOptions } from "../types.js";
import { anthropicToOpenAI, openAIToAnthropic } from "../utils/format.js";
/* eslint-disable @typescript-eslint/no-explicit-any */
@@ -15,6 +14,10 @@ interface AnthropicLike {
* Wrap an Anthropic client to auto-compress messages before each request.
*
* Intercepts `client.messages.create()` only. All other methods pass through.
* Messages reach `/v1/compress` in Anthropic shape and come back in Anthropic
* shape: the proxy compresses Anthropic content blocks natively, so thinking
* blocks, cache_control markers and tool_result blocks are forwarded exactly
* as the caller built them.
*
* @example
* ```typescript
@@ -40,20 +43,15 @@ export function withHeadroom<T extends AnthropicLike>(
const model =
options.model ?? params.model ?? "claude-sonnet-4-5-20250929";
const openaiMessages = anthropicToOpenAI(messages);
const result = await compress(openaiMessages, {
const result = await compress(messages, {
stack: "adapter_ts_anthropic",
...options,
model,
});
const anthropicMessages = result.compressed
? openAIToAnthropic(result.messages)
: messages;
return originalCreate({
...params,
messages: anthropicMessages,
messages: result.compressed ? result.messages : messages,
});
};
+12 -6
View File
@@ -14,7 +14,8 @@ import { extractUserQuery, countTurns, extractToolCalls } from "./hooks.js";
*
* Accepts messages in any format: OpenAI, Anthropic, Vercel AI SDK, or Google Gemini.
* Detects the format automatically, compresses via the proxy, and returns
* compressed messages in the same format as the input.
* compressed messages in the same format as the input. Anthropic messages are
* sent to the proxy unconverted (it compresses Anthropic blocks natively).
*/
export async function compress(
messages: any[],
@@ -47,21 +48,26 @@ export async function compress(
// 2. Detect input format
const inputFormat = detectFormat(processedMessages);
// 3. Convert to OpenAI format (the proxy's lingua franca)
const openaiMessages = toOpenAI(processedMessages);
// 3. Shape the messages for the proxy. Anthropic messages go as-is: /v1/compress
// compresses Anthropic content blocks natively, and converting them would drop
// the thinking blocks Anthropic requires back unchanged, along with cache_control
// markers and server-tool blocks. Every other format is converted to OpenAI
// shape, the proxy's lingua franca.
const native = inputFormat === "anthropic";
const proxyMessages = native ? processedMessages : toOpenAI(processedMessages);
// 4. Compute biases
let biases: Record<number, number> = {};
if (hooks) {
biases = await hooks.computeBiases(openaiMessages, ctx);
biases = await hooks.computeBiases(proxyMessages, ctx);
}
// 5. Compress via proxy
const client = providedClient ?? new HeadroomClient(clientOptions);
const result = await client.compress(openaiMessages, { model, tokenBudget, config });
const result = await client.compress(proxyMessages, { model, tokenBudget, config });
// 6. Convert compressed messages back to original format
const outputMessages = fromOpenAI(result.messages, inputFormat);
const outputMessages = native ? result.messages : fromOpenAI(result.messages, inputFormat);
const finalResult: CompressResult = {
...result,
+2
View File
@@ -54,6 +54,8 @@ export function detectFormat(messages: any[]): MessageFormat {
if (part.type === "tool_use" || part.type === "tool_result") return "anthropic";
// Anthropic: image / document with source.type
if ((part.type === "image" || part.type === "document") && part.source?.type) return "anthropic";
// Anthropic: thinking blocks (the Vercel AI SDK spells its equivalent part `reasoning`)
if (part.type === "thinking" || part.type === "redacted_thinking") return "anthropic";
// Vercel: `file` parts carry data/mediaType (OpenAI's `file` part carries a `file` object);
// `image` parts carry `image` (Anthropic's carry `source`)
if (part && typeof part === "object") {
+137 -223
View File
@@ -21,6 +21,39 @@ function mockCompressSuccess(
);
}
// A thinking-enabled tool-use turn. Anthropic requires the thinking blocks
// back "complete and unmodified" with the tool result, or the request is a 400.
const thinkingToolTurn = [
{
role: "user",
content: [
{ type: "text", text: "Read the gateway log.", cache_control: { type: "ephemeral" } },
],
},
{
role: "assistant",
content: [
{ type: "thinking", thinking: "I should read the log first.", signature: "SIG-abc123" },
{ type: "redacted_thinking", data: "REDACTED-xyz" },
{ type: "text", text: "Reading the log now." },
{ type: "tool_use", id: "toolu_01", name: "fetch_gateway_metrics", input: { path: "/var/log/gateway.json" } },
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "toolu_01",
content: [
{ type: "text", text: "row 1\nrow 2\nrow 3" },
{ type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },
],
},
],
},
];
describe("withHeadroom (Anthropic)", () => {
beforeEach(() => {
mockFetch.mockReset();
@@ -102,7 +135,7 @@ describe("withHeadroom (Anthropic)", () => {
expect(result).toEqual(expectedResponse);
});
it("converts Anthropic content blocks to OpenAI format for compression", async () => {
it("posts Anthropic content blocks to /v1/compress unchanged", async () => {
mockFetch.mockResolvedValueOnce(mockCompressSuccess());
const mockCreate = vi.fn().mockResolvedValue({});
@@ -112,226 +145,47 @@ describe("withHeadroom (Anthropic)", () => {
baseUrl: "http://localhost:8787",
});
const messages = [
{ role: "user", content: [{ type: "text", text: "analyze this data" }] },
];
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: [
{
role: "user",
content: [{ type: "text", text: "analyze this data" }],
},
],
messages,
max_tokens: 1024,
});
// Verify the compress call used OpenAI format
// The proxy compresses Anthropic blocks natively; nothing is converted.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages[0].role).toBe("user");
expect(body.messages[0].content).toBe("analyze this data");
expect(body.messages).toEqual(messages);
expect(body.model).toBe("claude-sonnet-4-5-20250929");
});
it("handles Anthropic tool_use and tool_result blocks", async () => {
// Proxy returns compressed messages
mockFetch.mockResolvedValueOnce(
mockCompressSuccess([
{ role: "user", content: "search for X" },
{
role: "assistant",
content: null,
tool_calls: [
{
id: "tu_1",
type: "function",
function: { name: "search", arguments: '{"q":"X"}' },
},
],
},
{
role: "tool",
content: '{"results":["a","b"]}',
tool_call_id: "tu_1",
},
]),
);
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
const wrapped = withHeadroom(fakeClient as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: [
{ role: "user", content: "search for X" },
{
role: "assistant",
content: [
{ type: "tool_use", id: "tu_1", name: "search", input: { q: "X" } },
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "tu_1",
content: '{"results":["a","b"]}',
},
],
},
],
max_tokens: 1024,
});
expect(mockCreate).toHaveBeenCalledOnce();
// Messages should be converted back to Anthropic format
const createArgs = mockCreate.mock.calls[0][0];
expect(createArgs.messages.length).toBeGreaterThan(0);
});
it("flattens array-content tool_result blocks to text for the compression request", async () => {
mockFetch.mockResolvedValueOnce(mockCompressSuccess());
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
const wrapped = withHeadroom(fakeClient as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: [
{ role: "user", content: "run the query" },
{
role: "assistant",
content: [
{ type: "tool_use", id: "tu_1", name: "query", input: { sql: "select 1" } },
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "tu_1",
content: [
{ type: "text", text: "row 1" },
{ type: "text", text: "row 2" },
],
},
],
},
],
max_tokens: 1024,
});
// The proxy must receive the tool output as text, not a JSON-encoded block array.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages[2]).toEqual({
role: "tool",
content: "row 1\nrow 2",
tool_call_id: "tu_1",
});
});
it("preserves image blocks instead of dropping them during compression", async () => {
mockFetch.mockResolvedValueOnce(mockCompressSuccess());
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
const wrapped = withHeadroom(fakeClient as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: [
{
role: "user",
content: [
{ type: "text", text: "what is this?" },
{
type: "image",
source: { type: "base64", media_type: "image/png", data: "AAAA" },
},
],
},
],
max_tokens: 1024,
});
// The image must not be silently dropped from the compression request.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toHaveLength(1);
expect(body.messages[0].content).toEqual([
{ type: "text", text: "what is this?" },
{ type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
]);
});
it("converts a retained image_url back to an Anthropic image block for the final messages.create call (image-only turn)", async () => {
// Proxy retains the image through compression (successful-compression path).
mockFetch.mockResolvedValueOnce(
mockCompressSuccess([
{
role: "user",
content: [
{ type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
],
},
]),
);
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
const wrapped = withHeadroom(fakeClient as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: [
{
role: "user",
content: [
{
type: "image",
source: { type: "base64", media_type: "image/png", data: "AAAA" },
},
],
},
],
max_tokens: 1024,
});
expect(mockCreate).toHaveBeenCalledOnce();
const createArgs = mockCreate.mock.calls[0][0];
expect(createArgs.messages).toEqual([
it("posts tool_use and array-content tool_result blocks as-is and forwards the proxy's result", async () => {
const turn = [
{ role: "user", content: "run the query" },
{
role: "assistant",
content: [
{ type: "tool_use", id: "tu_1", name: "query", input: { sql: "select 1" } },
],
},
{
role: "user",
content: [
{ type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },
{
type: "tool_result",
tool_use_id: "tu_1",
content: [
{ type: "text", text: "row 1" },
{ type: "text", text: "row 2" },
],
},
],
},
]);
});
it("converts a retained mixed text/image turn back to Anthropic blocks, order preserved, for the final messages.create call", async () => {
mockFetch.mockResolvedValueOnce(
mockCompressSuccess([
{
role: "user",
content: [
{ type: "text", text: "what is this?" },
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,BBBB" } },
],
},
]),
);
];
const compressed = structuredClone(turn);
(compressed[2].content as any[])[0].content = [{ type: "text", text: "[2 rows]" }];
mockFetch.mockResolvedValueOnce(mockCompressSuccess(compressed));
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
@@ -342,32 +196,92 @@ describe("withHeadroom (Anthropic)", () => {
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: [
{
role: "user",
content: [
{ type: "text", text: "what is this?" },
{
type: "image",
source: { type: "base64", media_type: "image/jpeg", data: "BBBB" },
},
],
},
],
messages: turn,
max_tokens: 1024,
});
// The proxy reads Anthropic tool_result blocks natively, list content included.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toEqual(turn);
expect(mockCreate).toHaveBeenCalledOnce();
const createArgs = mockCreate.mock.calls[0][0];
expect(createArgs.messages).toEqual([
expect(createArgs.messages).toEqual(compressed);
});
it("keeps image blocks native in the compression request and in the forwarded messages", async () => {
const turn = [
{
role: "user",
content: [
{ type: "text", text: "what is this?" },
{ type: "image", source: { type: "base64", media_type: "image/jpeg", data: "BBBB" } },
{
type: "image",
source: { type: "base64", media_type: "image/png", data: "AAAA" },
},
],
},
]);
];
mockFetch.mockResolvedValueOnce(mockCompressSuccess(turn));
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
const wrapped = withHeadroom(fakeClient as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: turn,
max_tokens: 1024,
});
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toEqual(turn);
expect(mockCreate).toHaveBeenCalledOnce();
const createArgs = mockCreate.mock.calls[0][0];
expect(createArgs.messages).toEqual(turn);
});
it("posts a thinking-enabled tool-use turn unchanged, thinking blocks and tool_result blocks included", async () => {
mockFetch.mockResolvedValueOnce(mockCompressSuccess(thinkingToolTurn));
const mockCreate = vi.fn().mockResolvedValue({});
const wrapped = withHeadroom({ messages: { create: mockCreate } } as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: thinkingToolTurn,
max_tokens: 1024,
});
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toEqual(thinkingToolTurn);
expect(body.model).toBe("claude-sonnet-4-5-20250929");
});
it("forwards the proxy's returned Anthropic messages to messages.create as-is", async () => {
const compressed = structuredClone(thinkingToolTurn);
(compressed[2].content[0] as any).content[0].text = "[3 rows]";
mockFetch.mockResolvedValueOnce(mockCompressSuccess(compressed));
const mockCreate = vi.fn().mockResolvedValue({});
const wrapped = withHeadroom({ messages: { create: mockCreate } } as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
messages: thinkingToolTurn,
max_tokens: 1024,
});
const createArgs = mockCreate.mock.calls[0][0];
expect(createArgs.messages).toEqual(compressed);
// The thinking blocks reach the provider exactly as the model produced them.
expect(createArgs.messages[1].content.slice(0, 2)).toEqual(thinkingToolTurn[1].content.slice(0, 2));
});
it("returns original messages on compression fallback", async () => {
+37 -4
View File
@@ -184,21 +184,54 @@ describe("compress() keeps Anthropic text documents through the public path", ()
mockFetch.mockReset();
});
it("sends a document-only turn as a media part and restores the document block", async () => {
it("posts a document-only turn as the native document block and returns it unchanged", async () => {
const doc = {
type: "document",
source: { type: "text", media_type: "text/plain", data: "Critical contract: do not delete account 123" },
title: "Contract",
citations: { enabled: true },
};
// The proxy passes non-text parts through unchanged; echo the request's messages back.
// The proxy passes non-text blocks through unchanged; echo the request's messages back.
mockFetch.mockImplementationOnce(async (_url: string, init: any) => okResponse({ messages: JSON.parse(init.body).messages }));
const result = await compress([{ role: "user", content: [doc] }], { baseUrl: "http://localhost:8787" });
// Anthropic input is not converted: the document block itself reaches the proxy.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toHaveLength(1);
expect(body.messages[0].content[0].type).toBe("image_url");
expect(body.messages).toEqual([{ role: "user", content: [doc] }]);
expect(result.messages).toEqual([{ role: "user", content: [doc] }]);
});
});
describe("compress() sends Anthropic-format input to the proxy natively", () => {
beforeEach(() => {
mockFetch.mockReset();
});
const anthropicTurn = [
{ role: "user", content: "search for X" },
{
role: "assistant",
content: [
{ type: "thinking", thinking: "Search first.", signature: "SIG-1" },
{ type: "tool_use", id: "toolu_1", name: "search", input: { q: "X" } },
],
},
{
role: "user",
content: [{ type: "tool_result", tool_use_id: "toolu_1", content: "a\nb\nc" }],
},
];
it("posts the Anthropic blocks as-is and returns the proxy's messages unconverted", async () => {
const returned = structuredClone(anthropicTurn);
(returned[2].content as any[])[0].content = "[3 lines]";
mockFetch.mockResolvedValueOnce(okResponse({ messages: returned }));
const result = await compress(anthropicTurn, { baseUrl: "http://localhost:8787" });
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toEqual(anthropicTurn);
expect(result.messages).toEqual(returned);
});
});
@@ -185,3 +185,26 @@ describe("detectFormat: Anthropic document blocks", () => {
).toBe("anthropic");
});
});
describe("detectFormat: Anthropic thinking blocks", () => {
it("detects a turn whose only Anthropic marker is a thinking block", () => {
const messages = [
{ role: "user", content: "hi" },
{
role: "assistant",
content: [
{ type: "thinking", thinking: "...", signature: "SIG" },
{ type: "text", text: "hello" },
],
},
];
expect(detectFormat(messages)).toBe("anthropic");
});
it("detects a turn whose only Anthropic marker is a redacted_thinking block", () => {
const messages = [
{ role: "assistant", content: [{ type: "redacted_thinking", data: "REDACTED" }, { type: "text", text: "hello" }] },
];
expect(detectFormat(messages)).toBe("anthropic");
});
});