mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(sdk): carry Vercel AI SDK reasoning parts and provider options through compression (#4564)
## Description
`headroomMiddleware` / `withHeadroom` for the Vercel AI SDK convert the
prompt to OpenAI chat shape before `/v1/compress` and back afterwards.
The converters only model `text`, `file`/`image`, `tool-call` and
`tool-result` parts, so everything else the AI SDK puts on a prompt is
dropped: **`reasoning` parts**, and **`providerOptions`** on every
message and part (`providerExecuted` on tool calls too).
That is the state the providers need back on the next step of a tool
loop:
- `@ai-sdk/anthropic` rebuilds `thinking` / `redacted_thinking` blocks
from `reasoning` parts' `providerOptions.anthropic.signature` /
`redactedData`; without them the request reaches Anthropic with
`tool_use` and no thinking block, which its docs say "triggers a 400
error". Message- and part-level `anthropic.cacheControl` breakpoints are
read from `providerOptions` as well.
- `@ai-sdk/google` carries Gemini thought signatures as
`providerOptions.google.thoughtSignature` on text, tool-call, file and
reasoning parts. Its current release injects a sentinel for Gemini 3 "to
keep the request from failing with HTTP 400" and names the cause:
"application code that drops `providerOptions.google.thoughtSignature`
when persisting or serializing assistant tool-call messages" — which is
what this middleware did.
- `@ai-sdk/openai` keeps Responses `itemId` /
`reasoningEncryptedContent` the same way.
Captured on `main` (`a2d4fe72`) by running a LanguageModelV3 step-2
prompt through the public converters, which is the middleware's exact
path — every message changed:
```text
[0] system : providerOptions (anthropic.cacheControl) dropped
[1] user : text part providerOptions (anthropic.cacheControl) dropped
[2] assistant: both reasoning parts dropped (signature + redactedData); text and tool-call
providerOptions (google.thoughtSignature, openai.itemId) dropped
[3] tool : tool-result providerOptions (anthropic.cacheControl) dropped
```
The proxy does not need any of this removed. `/v1/compress` copies
messages with their extra keys, keeps extra keys on text parts even when
it rewrites the text, and passes unknown content parts through (verified
live, below). So the converters now carry the AI SDK state as-is and the
reverse conversion hands it back verbatim.
Closes # (none — follow-up to #4489; same converter sweep)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`sdk/typescript/src/utils/format.ts`**
- `vercelToOpenAI` / `openAIToVercel`: `providerOptions` on a
system/user/assistant message stays on the OpenAI message under the same
key; on text parts it stays on the part; on `image`/`file` parts it
rides inside the existing `image_url` carrier object. Assistant parts
the converter does not model — `reasoning`, `reasoning-file`, `custom`,
provider-executed `tool-result` — keep their place in the OpenAI
`content[]` verbatim (text-only turns still collapse to a string unless
a text part carries options; same for user turns). `tool-call` →
`tool_calls[i]` plus `providerOptions` / `providerExecuted`, and
`position` (how many content parts preceded the call), so
`openAIToVercel` puts each call back in its place among reasoning / text
/ provider-executed parts instead of appending it; a turn whose text
follows a tool call keeps the content array for the same reason. One
OpenAI tool message is one `tool-result` part, so the part's options
take `providerOptions` and the Vercel message's go under
`messageProviderOptions` on the last split message only (a copy on each
would multiply an Anthropic cache breakpoint); a
`tool-approval-response` part (no OpenAI equivalent) is carried on
`part` and restored as-is. All of it restored on the way back.
- `detectFormat`: `reasoning` / `reasoning-file` / `custom` /
`tool-approval-*` parts are Vercel markers (Anthropic spells its block
`thinking`); `image_url` / `input_audio` / `refusal` / `file`-object
parts are OpenAI markers. A message-level `providerOptions` key is
Vercel evidence only once no message carries a structural marker,
because the public OpenAI message types allow the key too.
- `toOpenAI(messages, format?)` accepts a declared format.
- **`sdk/typescript/src/types.ts`, `src/index.ts`** — `ProviderOptions`;
`providerOptions?` on `TextContentPart`, `ToolCall` (plus
`providerExecuted?`) and the four message types; `ReasoningContentPart`
and a literal-typed `PassthroughContentPart` join `ContentPart`, so
`part.type === "text"` narrowing stays exact; `CompressOptions.format`.
- **`sdk/typescript/src/compress.ts`** — `format` (declared input
format) skips autodetection and is passed to `toOpenAI`.
- **`sdk/typescript/src/adapters/vercel-ai.ts`** — the middleware and
`compressVercelMessages` call `compress(prompt, { format: "vercel" })`
and use `result.messages`, instead of converting themselves and then
handing OpenAI-shaped messages to `compress()`, which re-detects the
format. That re-detection is why `format` exists: reasoning parts are
not OpenAI-shaped, so a converted conversation whose first marker is a
reasoning part would have been read as Vercel input again and lost its
tool calls; and a text-only prompt is indistinguishable from OpenAI
messages, which the middleware could never declare before.
- **Tests** — `test/utils/format.test.ts`: the original "skips reasoning
parts" expectation becomes a carry-verbatim one; a step-2 prompt
(system/user/assistant/tool with options everywhere, two reasoning
parts, tool-call options) round-trips exactly; tool-call
`providerOptions`/`providerExecuted`; provider-executed `tool-result`
and `tool-approval-response` parts verbatim; user text with options kept
as a part; text-only assistant turns still a string; file-part options
through the media carrier. `test/adapters/vercel-ai.test.ts`: a typed
`LanguageModelV3Prompt` through `transformParams` with the proxy
rewriting only the tool message. `test/compress.test.ts`: `format:
"vercel"` on a text-only prompt. `test/utils/detect-format.test.ts`: the
new markers, and OpenAI `image_url` / `file` / `input_audio` parts
outranking `providerOptions` (the review repro included).
`test/types.test-d.ts`: the public type changes. Review round 1 added:
`compress()` with no `format` keeping an OpenAI user message with
`providerOptions`, `image_url` and `file`; round trips for `[tool-call,
reasoning, tool-call]`, a provider-executed search turn and `[tool-call,
text]`; the `text, tool-call` wire shape unchanged; message-level
options once per split tool message; the search turn through
`transformParams`.
- **Docs** — `docs/content/docs/vercel-ai-sdk.mdx` (what is carried),
`docs/content/docs/api-reference.mdx` and `sdk/typescript/README.md`
(the `format` option).
**Not in this PR.** Tool-result `output` kinds: every output is still
flattened to a string and restored as `json`/`text`, so `error-text` /
`error-json` lose their error-ness and `content` outputs with media
become JSON text. Same converter, different defect — next PR. The
Gemini-native converter's `thought` parts and `thoughtSignature` are
also a separate PR.
## Testing
- [x] Unit tests pass (`npm test` — vitest, incl. typecheck)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
- [x] Manual testing performed (below)
### Test Output
```text
$ cd sdk/typescript && npm test
Test Files 19 passed | 5 skipped (24)
Tests 435 passed | 33 skipped (468)
Type Errors no errors
$ npm run typecheck # tsc --noEmit — clean
$ npm run build # tsup — ESM, CJS and DTS built
```
## Real Behavior Proof
- Environment: published proxy `headroom-ai 0.42.0` (`uvx --from
'headroom-ai[proxy]' headroom proxy --port 8799`), this branch built
with `npm run build`, Node v22.16.0, macOS.
- Exact command / steps: a LanguageModelV3 step-2 prompt — system and
user with `anthropic.cacheControl`, assistant with two `reasoning` parts
(signature, redactedData), a text part and a tool call carrying
`google.thoughtSignature` / `openai.itemId`, message-level options, and
a `tool-result` carrying a 72 KB JSON log plus `cacheControl` — passed
through `headroomMiddleware().transformParams` against the live proxy;
the returned prompt compared message by message with the input.
- Observed result: every message other than the tool result came back
byte-identical, the tool result kept its options and shrank from 72 432
to 34 916 chars. Full output:
```text
system byte-identical (cacheControl kept): true
user byte-identical (text providerOptions kept): true
assistant byte-identical (2 reasoning parts, text + tool-call providerOptions, message providerOptions): true
tool-result output: 72432 -> 34916 chars, type=text, toolName=fetch_gateway_metrics, providerOptions kept: true
tool-result output head: "[400]{id:int,level:string,msg:string,path:string,service:string,status:int,ts:string}\n0,WA"
```
Review round 1, same proxy: an interleaved provider-executed turn
(`reasoning, tool-call, tool-result, reasoning, text, tool-call`)
through `headroomMiddleware`, then an OpenAI user message with
`providerOptions` + `image_url` + `file` (and a 56 KB tool result)
through `compress()` with no `format`:
```text
assistant byte-identical (order back: reasoning, tool-call, tool-result, reasoning, text, tool-call): true
tool-result output: 56002 -> 31287 chars, type=text, toolName=fetch_gateway_metrics; message providerOptions kept: true
detectFormat: openai
user message byte-identical (image_url and file kept): true
tool message: 56002 -> 31690 chars; compressed=true
```
The same payload through the converters on `main` loses every item
listed in the Description (captured before the change with the same
script).
- Not tested: an end-to-end multi-step `generateText` against Anthropic
/ Gemini with thinking enabled (no provider keys in this environment);
the provider behaviour is taken from the `@ai-sdk/anthropic` and
`@ai-sdk/google` sources and Anthropic's docs quoted above.
## Runtime Rollout Safety
- Rollout-managed feature(s): none (TypeScript SDK only, no proxy
change)
- Minimum rollout channel: n/a
- Stable/default behavior changed: for SDK users, OpenAI-shaped messages
sent to `/v1/compress` now carry the AI SDK's `providerOptions` keys and
reasoning parts, which the proxy already passes through; the public
`ContentPart` union grows (additive).
- Kill switch / disable path: unchanged — `fallback: true` returns the
original prompt when the proxy is unreachable; unwrap the model to
disable compression.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
- The carried keys never reach an OpenAI endpoint through the SDK's own
paths: the universal `compress()` returns messages in the input format,
and the OpenAI adapter never sees Vercel input.
- Enumerated before writing: the V3 and V4 prompt unions
(`@ai-sdk/provider` 4.0.26) and the `ModelMessage` unions
(`@ai-sdk/provider-utils` 5.0.58); every assistant and tool part type in
them is either modelled or carried verbatim.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4564"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
682bf73f48
commit
a26faaeee3
@@ -152,6 +152,7 @@ Top-level function to compress messages via the Headroom proxy.
|
||||
client: { type: 'HeadroomClientInterface', description: 'Pre-configured client instance to use' },
|
||||
tokenBudget: { type: 'number', description: 'Token budget — compress to fit within this limit' },
|
||||
hooks: { type: 'CompressionHooks', description: 'Compression hooks for pre/post processing' },
|
||||
format: { type: "'openai' | 'anthropic' | 'vercel' | 'gemini'", description: 'Format of `messages`, when known. Skips autodetection, which cannot tell a text-only Vercel AI SDK prompt from OpenAI messages' },
|
||||
}} />
|
||||
|
||||
```ts twoslash
|
||||
|
||||
@@ -136,4 +136,4 @@ const { output } = await generateText({
|
||||
3. Compressed messages are converted back to Vercel format
|
||||
4. The original model receives the smaller prompt
|
||||
|
||||
All other model behavior (tool calling, structured output, streaming) is unchanged.
|
||||
All other model behavior (tool calling, structured output, streaming) is unchanged. Provider state the AI SDK attaches to the prompt travels through untouched: `reasoning` parts (with the Anthropic `signature` or `redactedData`, OpenAI `itemId`, Google `thoughtSignature` they carry) stay in place, tool calls return to their place among them, and `providerOptions` on messages, text and file parts, tool calls and tool results come back exactly as sent. Only large tool results change.
|
||||
|
||||
@@ -384,6 +384,7 @@ const result = await compress(messages, {
|
||||
retries: 1, // retry on transient failures (default)
|
||||
tokenBudget: 4000, // compress to fit this limit
|
||||
hooks: new MyHooks(), // pre/post compression hooks
|
||||
format: 'vercel', // declare the input format; skips autodetection
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { createRequire } from "node:module";
|
||||
import { compress } from "../compress.js";
|
||||
import type { CompressOptions, CompressResult } from "../types.js";
|
||||
import { vercelToOpenAI, openAIToVercel } from "../utils/format.js";
|
||||
|
||||
/**
|
||||
* Minimal structural type for Vercel AI SDK language models.
|
||||
@@ -49,22 +48,20 @@ export function headroomMiddleware(options: CompressOptions = {}) {
|
||||
|
||||
const model = options.model ?? params.modelId ?? "gpt-4o";
|
||||
|
||||
// Convert Vercel format → OpenAI format
|
||||
const openaiMessages = vercelToOpenAI(prompt);
|
||||
|
||||
// Compress via Headroom
|
||||
const result = await compress(openaiMessages, {
|
||||
// compress() converts the prompt to OpenAI shape for the proxy and back. The format is
|
||||
// declared rather than detected: a text-only prompt is indistinguishable from OpenAI
|
||||
// messages, and the converted messages carry AI SDK parts (reasoning, providerOptions)
|
||||
// that detection would otherwise read as Vercel input again.
|
||||
const result = await compress(prompt, {
|
||||
stack: "adapter_ts_vercel_ai",
|
||||
...options,
|
||||
model,
|
||||
format: "vercel",
|
||||
});
|
||||
|
||||
if (!result.compressed) return params;
|
||||
|
||||
// Convert back to Vercel format
|
||||
const compressedPrompt = openAIToVercel(result.messages);
|
||||
|
||||
return { ...params, prompt: compressedPrompt };
|
||||
return { ...params, prompt: result.messages };
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -77,16 +74,15 @@ export async function compressVercelMessages(
|
||||
messages: VercelMessage[],
|
||||
options: CompressOptions = {},
|
||||
): Promise<CompressResult & { messages: VercelMessage[] }> {
|
||||
const openaiMessages = vercelToOpenAI(messages);
|
||||
const result = await compress(openaiMessages, {
|
||||
const result = await compress(messages, {
|
||||
stack: "adapter_ts_vercel_ai",
|
||||
...options,
|
||||
format: "vercel",
|
||||
});
|
||||
const vercelMessages = openAIToVercel(result.messages);
|
||||
|
||||
return {
|
||||
...result,
|
||||
messages: vercelMessages,
|
||||
messages: result.messages,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@ export async function compress(
|
||||
tokenBudget,
|
||||
hooks,
|
||||
config,
|
||||
format,
|
||||
...clientOptions
|
||||
} = options;
|
||||
|
||||
@@ -45,8 +46,8 @@ export async function compress(
|
||||
processedMessages = await hooks.preCompress(messages, ctx);
|
||||
}
|
||||
|
||||
// 2. Detect input format
|
||||
const inputFormat = detectFormat(processedMessages);
|
||||
// 2. Input format: declared by the caller, else detected
|
||||
const inputFormat = format ?? detectFormat(processedMessages);
|
||||
|
||||
// 3. Shape the messages for the proxy. Anthropic messages go as-is: /v1/compress
|
||||
// compresses Anthropic content blocks natively, and converting them would drop
|
||||
@@ -54,7 +55,7 @@ export async function compress(
|
||||
// markers and server-tool blocks. Every other format is converted to OpenAI
|
||||
// shape, the proxy's lingua franca.
|
||||
const native = inputFormat === "anthropic";
|
||||
const proxyMessages = native ? processedMessages : toOpenAI(processedMessages);
|
||||
const proxyMessages = native ? processedMessages : toOpenAI(processedMessages, inputFormat);
|
||||
|
||||
// 4. Compute biases
|
||||
let biases: Record<number, number> = {};
|
||||
|
||||
@@ -17,8 +17,11 @@ export { parseSSE, collectStream } from "./utils/stream.js";
|
||||
|
||||
// --- Core types ---
|
||||
export type {
|
||||
ProviderOptions,
|
||||
TextContentPart,
|
||||
ImageContentPart,
|
||||
ReasoningContentPart,
|
||||
PassthroughContentPart,
|
||||
ContentPart,
|
||||
ToolCall,
|
||||
SystemMessage,
|
||||
|
||||
@@ -5,12 +5,22 @@
|
||||
|
||||
import type { CompressionHooks } from "./hooks.js";
|
||||
import type { CompressRequestConfig } from "./types/config.js";
|
||||
import type { MessageFormat } from "./utils/format.js";
|
||||
|
||||
// --- Message types (OpenAI chat format) ---
|
||||
|
||||
/**
|
||||
* Provider-specific options as the Vercel AI SDK attaches them to messages and
|
||||
* parts (`{ anthropic: { cacheControl }, google: { thoughtSignature }, ... }`).
|
||||
* The SDK carries them through `/v1/compress` unchanged so the AI SDK provider
|
||||
* sees exactly what it produced; they are never sent to an OpenAI endpoint.
|
||||
*/
|
||||
export type ProviderOptions = Record<string, Record<string, unknown>>;
|
||||
|
||||
export interface TextContentPart {
|
||||
type: "text";
|
||||
text: string;
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
export interface ImageContentPart {
|
||||
@@ -18,22 +28,53 @@ export interface ImageContentPart {
|
||||
image_url: { url: string; detail?: "auto" | "low" | "high" };
|
||||
}
|
||||
|
||||
export type ContentPart = TextContentPart | ImageContentPart;
|
||||
/**
|
||||
* A Vercel AI SDK `reasoning` part (the assistant's thinking, with the
|
||||
* provider's signature or redacted data in `providerOptions`). Carried verbatim
|
||||
* through `/v1/compress`, which passes unknown parts through, so the provider
|
||||
* gets its reasoning back on the next step of a tool loop.
|
||||
*/
|
||||
export interface ReasoningContentPart {
|
||||
type: "reasoning";
|
||||
text: string;
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
/**
|
||||
* Other Vercel AI SDK assistant parts with no OpenAI equivalent, carried verbatim
|
||||
* the same way: `reasoning-file`, `custom`, provider-executed `tool-result`,
|
||||
* `tool-approval-request`.
|
||||
*/
|
||||
export interface PassthroughContentPart {
|
||||
type: "reasoning-file" | "custom" | "tool-result" | "tool-approval-request";
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export type ContentPart =
|
||||
| TextContentPart
|
||||
| ImageContentPart
|
||||
| ReasoningContentPart
|
||||
| PassthroughContentPart;
|
||||
|
||||
export interface ToolCall {
|
||||
id: string;
|
||||
type: "function";
|
||||
function: { name: string; arguments: string };
|
||||
/** Vercel AI SDK: the call was executed by the provider. Carried, never sent to OpenAI. */
|
||||
providerExecuted?: boolean;
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
export interface SystemMessage {
|
||||
role: "system";
|
||||
content: string;
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
export interface UserMessage {
|
||||
role: "user";
|
||||
content: string | ContentPart[];
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
export interface AssistantMessage {
|
||||
@@ -44,12 +85,14 @@ export interface AssistantMessage {
|
||||
*/
|
||||
content: string | ContentPart[] | null;
|
||||
tool_calls?: ToolCall[];
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
export interface ToolMessage {
|
||||
role: "tool";
|
||||
content: string;
|
||||
tool_call_id: string;
|
||||
providerOptions?: ProviderOptions;
|
||||
}
|
||||
|
||||
export type OpenAIMessage =
|
||||
@@ -77,6 +120,11 @@ export interface CompressOptions {
|
||||
/** Per-call /v1/compress config, merged over the client-level config
|
||||
* (per-call keys override at the top level). */
|
||||
config?: CompressRequestConfig;
|
||||
/**
|
||||
* The format of `messages`, when the caller knows it. Skips autodetection,
|
||||
* which cannot tell a text-only Vercel AI SDK prompt from OpenAI messages.
|
||||
*/
|
||||
format?: MessageFormat;
|
||||
}
|
||||
|
||||
export interface CompressResult {
|
||||
|
||||
@@ -16,6 +16,7 @@ import type {
|
||||
AssistantMessage,
|
||||
ContentPart,
|
||||
ToolCall,
|
||||
ToolMessage,
|
||||
} from "../types.js";
|
||||
|
||||
/* eslint-disable @typescript-eslint/no-explicit-any */
|
||||
@@ -29,9 +30,14 @@ export type MessageFormat = "openai" | "anthropic" | "vercel" | "gemini";
|
||||
* - Gemini: messages have `parts` field (not `content`)
|
||||
* - Vercel: content parts use hyphenated types: `tool-call`, `tool-result`
|
||||
* - Anthropic: content blocks use underscored types: `tool_use`, `tool_result`
|
||||
* - OpenAI: assistant messages have `tool_calls` field; tool messages have `tool_call_id`
|
||||
* - OpenAI: assistant messages have `tool_calls` field; tool messages have `tool_call_id`;
|
||||
* content parts of type `image_url`, `input_audio`, `refusal`, or `file` with a `file` object
|
||||
*
|
||||
* A message-level `providerOptions` key is Vercel evidence too, but the public OpenAI message
|
||||
* types allow it as well, so it only decides once no message carries a structural marker.
|
||||
*/
|
||||
export function detectFormat(messages: any[]): MessageFormat {
|
||||
let providerOptions = false;
|
||||
for (const msg of messages) {
|
||||
// Gemini: uses `parts` instead of `content`
|
||||
if ("parts" in msg && !("content" in msg)) return "gemini";
|
||||
@@ -48,8 +54,17 @@ export function detectFormat(messages: any[]): MessageFormat {
|
||||
// Check content blocks/parts for format-specific markers
|
||||
if (Array.isArray(msg.content)) {
|
||||
for (const part of msg.content) {
|
||||
// Vercel: hyphenated types
|
||||
// OpenAI: image_url / input_audio / refusal parts
|
||||
if (part.type === "image_url" || part.type === "input_audio" || part.type === "refusal") return "openai";
|
||||
// Vercel: hyphenated types, reasoning parts (Anthropic spells its block `thinking`)
|
||||
if (part.type === "tool-call" || part.type === "tool-result") return "vercel";
|
||||
if (
|
||||
part.type === "reasoning" ||
|
||||
part.type === "reasoning-file" ||
|
||||
part.type === "custom" ||
|
||||
part.type === "tool-approval-request" ||
|
||||
part.type === "tool-approval-response"
|
||||
) return "vercel";
|
||||
// Anthropic: underscored types
|
||||
if (part.type === "tool_use" || part.type === "tool_result") return "anthropic";
|
||||
// Anthropic: image / document with source.type
|
||||
@@ -59,15 +74,19 @@ export function detectFormat(messages: any[]): MessageFormat {
|
||||
// Vercel: `file` parts carry data/mediaType (OpenAI's `file` part carries a `file` object);
|
||||
// `image` parts carry `image` (Anthropic's carry `source`)
|
||||
if (part && typeof part === "object") {
|
||||
if (part.type === "file" && "file" in part) return "openai";
|
||||
if (part.type === "file" && ("data" in part || "mediaType" in part)) return "vercel";
|
||||
if (part.type === "image" && "image" in part) return "vercel";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (msg && typeof msg === "object" && "providerOptions" in msg) providerOptions = true;
|
||||
}
|
||||
|
||||
// Default: OpenAI (simple {role, content: string} is valid OpenAI)
|
||||
return "openai";
|
||||
// Default: OpenAI (simple {role, content: string} is valid OpenAI), unless a message
|
||||
// carried providerOptions and nothing marked the conversation as another format.
|
||||
return providerOptions ? "vercel" : "openai";
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
@@ -362,6 +381,7 @@ type VercelMediaImageUrl = {
|
||||
data_tag?: "data" | "url" | "text" | "reference";
|
||||
// A provider-managed file reference, carried verbatim (it has no bytes or url to convert).
|
||||
reference?: Record<string, string>;
|
||||
providerOptions?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
function base64ToText(b64: string): string {
|
||||
@@ -399,6 +419,7 @@ function vercelMediaImageUrl(p: any): VercelMediaImageUrl | null {
|
||||
const out: VercelMediaImageUrl = { url: "" };
|
||||
if (p.type === "file") out.part = "file";
|
||||
if (typeof p.filename === "string") out.filename = p.filename;
|
||||
if (p.providerOptions && typeof p.providerOptions === "object") out.providerOptions = p.providerOptions;
|
||||
|
||||
if (data && typeof data === "object" && typeof data.type === "string") {
|
||||
if (data.type === "data") {
|
||||
@@ -468,14 +489,25 @@ function vercelMediaPart(imageUrl: any): any {
|
||||
if (tag === "url") data = { type: "url", url: parsed ?? url };
|
||||
}
|
||||
const isFile = imageUrl.part === "file" || (mediaType !== undefined && !mediaType.startsWith("image/"));
|
||||
let part: any;
|
||||
if (isFile) {
|
||||
const part: any = { type: "file", mediaType: mediaType ?? "application/octet-stream", data };
|
||||
part = { type: "file", mediaType: mediaType ?? "application/octet-stream", data };
|
||||
if (typeof imageUrl.filename === "string") part.filename = imageUrl.filename;
|
||||
return part;
|
||||
} else {
|
||||
part = { type: "image", image: data };
|
||||
if (mediaType) part.mediaType = mediaType;
|
||||
}
|
||||
const part: any = { type: "image", image: data };
|
||||
if (mediaType) part.mediaType = mediaType;
|
||||
return part;
|
||||
return withProviderOptions(part, imageUrl);
|
||||
}
|
||||
|
||||
// Copy the AI SDK's `providerOptions` from a message or part onto its OpenAI counterpart. The
|
||||
// key has no OpenAI meaning; the proxy copies messages with their extra keys and passes unknown
|
||||
// content parts through, so the reverse conversion hands it back exactly as it arrived.
|
||||
function withProviderOptions<T extends object>(target: T, source: any): T {
|
||||
if (source && typeof source === "object" && source.providerOptions && typeof source.providerOptions === "object") {
|
||||
(target as any).providerOptions = source.providerOptions;
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
// Assistant content parts for the Anthropic / Gemini reverse converters. A text-only array
|
||||
@@ -501,69 +533,86 @@ export function vercelToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
|
||||
for (const msg of messages) {
|
||||
if (msg.role === "system") {
|
||||
result.push({ role: "system", content: typeof msg.content === "string" ? msg.content : String(msg.content) });
|
||||
result.push(withProviderOptions(
|
||||
{ role: "system", content: typeof msg.content === "string" ? msg.content : String(msg.content) },
|
||||
msg,
|
||||
));
|
||||
continue;
|
||||
}
|
||||
|
||||
if (msg.role === "user") {
|
||||
if (typeof msg.content === "string") {
|
||||
result.push({ role: "user", content: msg.content });
|
||||
result.push(withProviderOptions({ role: "user", content: msg.content }, msg));
|
||||
continue;
|
||||
}
|
||||
const parts = Array.isArray(msg.content) ? msg.content : [];
|
||||
const textParts = parts.filter((p: any) => p.type === "text");
|
||||
const mediaParts = parts.filter((p: any) => p.type === "image" || p.type === "file");
|
||||
// Text-only turns collapse to a string unless a text part carries providerOptions.
|
||||
const plainText = mediaParts.length === 0 && textParts.length > 0 && textParts.every((p: any) => !p.providerOptions);
|
||||
|
||||
if (mediaParts.length === 0 && textParts.length > 0) {
|
||||
result.push({ role: "user", content: textParts.map((p: any) => p.text).join("") });
|
||||
if (plainText) {
|
||||
result.push(withProviderOptions({ role: "user", content: textParts.map((p: any) => p.text).join("") }, msg));
|
||||
} else {
|
||||
const openaiParts = parts
|
||||
.filter((p: any) => p.type === "text" || p.type === "image" || p.type === "file")
|
||||
.map((p: any) => {
|
||||
if (p.type === "text") return { type: "text" as const, text: p.text };
|
||||
if (p.type === "text") return withProviderOptions({ type: "text" as const, text: p.text }, p);
|
||||
const imageUrl = vercelMediaImageUrl(p);
|
||||
return imageUrl ? { type: "image_url" as const, image_url: imageUrl } : null;
|
||||
})
|
||||
.filter((p: any): p is NonNullable<typeof p> => p !== null);
|
||||
result.push({ role: "user", content: openaiParts });
|
||||
result.push(withProviderOptions({ role: "user", content: openaiParts }, msg));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (msg.role === "assistant") {
|
||||
if (typeof msg.content === "string") {
|
||||
result.push({ role: "assistant", content: msg.content });
|
||||
result.push(withProviderOptions({ role: "assistant", content: msg.content }, msg));
|
||||
continue;
|
||||
}
|
||||
const parts = Array.isArray(msg.content) ? msg.content : [];
|
||||
const textParts = parts.filter((p: any) => p.type === "text");
|
||||
const toolCallParts = parts.filter((p: any) => p.type === "tool-call");
|
||||
const hasFiles = parts.some((p: any) => p.type === "file");
|
||||
// Anything other than text and tool calls keeps its place in the content array: files
|
||||
// as image_url carriers; reasoning, reasoning-file, custom and provider-executed
|
||||
// tool-result parts verbatim. So does a turn whose text follows a tool call, since a
|
||||
// string cannot say which came first. Text-only content otherwise collapses to a
|
||||
// string unless a text part carries providerOptions.
|
||||
const keepParts =
|
||||
parts.some((p: any) => p.type !== "text" && p.type !== "tool-call") ||
|
||||
textParts.some((p: any) => p.providerOptions) ||
|
||||
parts.some((p: any, i: number) => p.type === "text" && parts.slice(0, i).some((q: any) => q.type === "tool-call"));
|
||||
|
||||
let content: AssistantMessage["content"];
|
||||
if (hasFiles) {
|
||||
// AI SDK assistant content may carry generated files; keep them as ordered parts.
|
||||
content = parts
|
||||
.filter((p: any) => p.type === "text" || p.type === "file")
|
||||
.map((p: any) => {
|
||||
if (p.type === "text") return { type: "text" as const, text: p.text };
|
||||
const imageUrl = vercelMediaImageUrl(p);
|
||||
return imageUrl ? { type: "image_url" as const, image_url: imageUrl } : null;
|
||||
})
|
||||
.filter((p: any): p is NonNullable<typeof p> => p !== null);
|
||||
} else {
|
||||
content = textParts.length > 0 ? textParts.map((p: any) => p.text).join("") : null;
|
||||
}
|
||||
const openaiMsg: AssistantMessage = { role: "assistant", content };
|
||||
|
||||
if (toolCallParts.length > 0) {
|
||||
openaiMsg.tool_calls = toolCallParts.map((p: any): ToolCall => ({
|
||||
id: p.toolCallId,
|
||||
type: "function",
|
||||
// AI SDK v6 uses `input`, earlier versions used `args`
|
||||
function: { name: p.toolName, arguments: JSON.stringify(p.input ?? p.args) },
|
||||
}));
|
||||
const content: any[] = [];
|
||||
const toolCalls: ToolCall[] = [];
|
||||
for (const p of parts) {
|
||||
if (p.type === "tool-call") {
|
||||
const tc: ToolCall = {
|
||||
id: p.toolCallId,
|
||||
type: "function",
|
||||
// AI SDK v6 uses `input`, earlier versions used `args`
|
||||
function: { name: p.toolName, arguments: JSON.stringify(p.input ?? p.args) },
|
||||
};
|
||||
if (p.providerExecuted !== undefined) tc.providerExecuted = p.providerExecuted;
|
||||
// OpenAI lists tool calls apart from the content, so each one carries how many
|
||||
// content parts preceded it; openAIToVercel puts it back there.
|
||||
if (keepParts) (tc as any).position = content.length;
|
||||
toolCalls.push(withProviderOptions(tc, p));
|
||||
} else if (p.type === "text") {
|
||||
content.push(withProviderOptions({ type: "text", text: p.text }, p));
|
||||
} else if (p.type === "file") {
|
||||
const imageUrl = vercelMediaImageUrl(p);
|
||||
if (imageUrl) content.push({ type: "image_url", image_url: imageUrl });
|
||||
} else {
|
||||
content.push(p);
|
||||
}
|
||||
}
|
||||
const openaiMsg: AssistantMessage = withProviderOptions({
|
||||
role: "assistant",
|
||||
content: keepParts ? content : textParts.length > 0 ? textParts.map((p: any) => p.text).join("") : null,
|
||||
}, msg);
|
||||
if (toolCalls.length > 0) openaiMsg.tool_calls = toolCalls;
|
||||
result.push(openaiMsg);
|
||||
continue;
|
||||
}
|
||||
@@ -571,6 +620,7 @@ export function vercelToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
if (msg.role === "tool") {
|
||||
const parts = Array.isArray(msg.content) ? msg.content : [];
|
||||
for (const part of parts) {
|
||||
let toolMsg: ToolMessage;
|
||||
if (part.type === "tool-result") {
|
||||
// AI SDK v6 uses `output: { type, value }`, earlier versions used `result`
|
||||
let contentStr: string;
|
||||
@@ -584,12 +634,22 @@ export function vercelToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
} else {
|
||||
contentStr = "";
|
||||
}
|
||||
result.push({
|
||||
role: "tool",
|
||||
content: contentStr,
|
||||
tool_call_id: part.toolCallId,
|
||||
});
|
||||
toolMsg = withProviderOptions({ role: "tool", content: contentStr, tool_call_id: part.toolCallId }, part);
|
||||
} else {
|
||||
// tool-approval-response (or any other AI SDK tool part): no OpenAI equivalent,
|
||||
// carried verbatim on `part` and restored as-is.
|
||||
toolMsg = { role: "tool", content: "", tool_call_id: part.toolCallId ?? part.approvalId ?? "" };
|
||||
(toolMsg as any).part = part;
|
||||
}
|
||||
// One OpenAI tool message is one tool-result part, so the part's providerOptions
|
||||
// take the `providerOptions` key and the message's go under `messageProviderOptions`
|
||||
// on the last part only: the AI SDK's Anthropic provider applies message-level
|
||||
// options (a cache breakpoint) to a message's last part, and a copy on every split
|
||||
// message would multiply them.
|
||||
if (part === parts[parts.length - 1] && msg.providerOptions && typeof msg.providerOptions === "object") {
|
||||
(toolMsg as any).messageProviderOptions = msg.providerOptions;
|
||||
}
|
||||
result.push(toolMsg);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
@@ -605,20 +665,20 @@ export function openAIToVercel(messages: OpenAIMessage[]): any[] {
|
||||
|
||||
for (const msg of messages) {
|
||||
if (msg.role === "system") {
|
||||
result.push({ role: "system", content: msg.content });
|
||||
result.push(withProviderOptions({ role: "system", content: msg.content }, msg));
|
||||
continue;
|
||||
}
|
||||
|
||||
if (msg.role === "user") {
|
||||
if (typeof msg.content === "string") {
|
||||
result.push({ role: "user", content: [{ type: "text", text: msg.content }] });
|
||||
result.push(withProviderOptions({ role: "user", content: [{ type: "text", text: msg.content }] }, msg));
|
||||
} else if (Array.isArray(msg.content)) {
|
||||
const parts = msg.content.map((p) => {
|
||||
if (p.type === "text") return { type: "text", text: p.text };
|
||||
if (p.type === "text") return withProviderOptions({ type: "text", text: p.text }, p);
|
||||
if (p.type === "image_url") return vercelMediaPart(p.image_url);
|
||||
return { type: "text", text: "" };
|
||||
return p;
|
||||
});
|
||||
result.push({ role: "user", content: parts });
|
||||
result.push(withProviderOptions({ role: "user", content: parts }, msg));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
@@ -629,43 +689,59 @@ export function openAIToVercel(messages: OpenAIMessage[]): any[] {
|
||||
if (msg.content) parts.push({ type: "text", text: msg.content });
|
||||
} else if (Array.isArray(msg.content)) {
|
||||
for (const p of msg.content) {
|
||||
if (p.type === "text") parts.push({ type: "text", text: p.text });
|
||||
if (p.type === "text") parts.push(withProviderOptions({ type: "text", text: p.text }, p));
|
||||
else if (p.type === "image_url") parts.push(vercelMediaPart(p.image_url));
|
||||
else parts.push(p); // reasoning and other AI SDK parts, carried verbatim
|
||||
}
|
||||
}
|
||||
if (msg.tool_calls) {
|
||||
// A tool call vercelToOpenAI took out of the content array carries `position`, the
|
||||
// number of content parts that preceded it, and goes back there; others go last.
|
||||
let inserted = 0;
|
||||
for (const tc of msg.tool_calls) {
|
||||
toolNames.set(tc.id, tc.function.name);
|
||||
let input: any;
|
||||
try { input = JSON.parse(tc.function.arguments); } catch { input = tc.function.arguments ?? {}; }
|
||||
parts.push({
|
||||
const part: any = {
|
||||
type: "tool-call",
|
||||
toolCallId: tc.id,
|
||||
toolName: tc.function.name,
|
||||
input, // AI SDK v6 uses `input`, not `args`
|
||||
});
|
||||
};
|
||||
if (tc.providerExecuted !== undefined) part.providerExecuted = tc.providerExecuted;
|
||||
withProviderOptions(part, tc);
|
||||
const at = (tc as any).position;
|
||||
if (typeof at === "number") parts.splice(at + inserted++, 0, part);
|
||||
else parts.push(part);
|
||||
}
|
||||
}
|
||||
result.push({ role: "assistant", content: parts });
|
||||
result.push(withProviderOptions({ role: "assistant", content: parts }, msg));
|
||||
continue;
|
||||
}
|
||||
|
||||
if (msg.role === "tool") {
|
||||
let parsed: any;
|
||||
try { parsed = JSON.parse(msg.content); } catch { parsed = msg.content; }
|
||||
// AI SDK v6 requires output: { type: 'json' | 'text', value }
|
||||
const output = typeof parsed === "string"
|
||||
? { type: "text" as const, value: parsed }
|
||||
: { type: "json" as const, value: parsed };
|
||||
result.push({
|
||||
role: "tool",
|
||||
content: [{
|
||||
const carried = (msg as any).part;
|
||||
let part: any;
|
||||
if (carried && typeof carried === "object") {
|
||||
part = carried;
|
||||
} else {
|
||||
let parsed: any;
|
||||
try { parsed = JSON.parse(msg.content); } catch { parsed = msg.content; }
|
||||
// AI SDK v6 requires output: { type: 'json' | 'text', value }
|
||||
const output = typeof parsed === "string"
|
||||
? { type: "text" as const, value: parsed }
|
||||
: { type: "json" as const, value: parsed };
|
||||
part = withProviderOptions({
|
||||
type: "tool-result",
|
||||
toolCallId: msg.tool_call_id,
|
||||
toolName: toolNames.get(msg.tool_call_id) ?? "unknown",
|
||||
output,
|
||||
}],
|
||||
});
|
||||
}, msg);
|
||||
}
|
||||
const toolMsg: any = { role: "tool", content: [part] };
|
||||
const messageOptions = (msg as any).messageProviderOptions;
|
||||
if (messageOptions && typeof messageOptions === "object") toolMsg.providerOptions = messageOptions;
|
||||
result.push(toolMsg);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
@@ -863,8 +939,7 @@ export function openAIToGemini(messages: OpenAIMessage[]): any[] {
|
||||
// Universal: any format → OpenAI, and OpenAI → original format
|
||||
// ============================================================
|
||||
|
||||
export function toOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
const format = detectFormat(messages);
|
||||
export function toOpenAI(messages: any[], format: MessageFormat = detectFormat(messages)): OpenAIMessage[] {
|
||||
switch (format) {
|
||||
case "openai": return messages as OpenAIMessage[];
|
||||
case "anthropic": return anthropicToOpenAI(messages);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import type { LanguageModelV3, LanguageModelV4FilePart, LanguageModelV4Prompt } from "@ai-sdk/provider";
|
||||
import type { LanguageModelV3, LanguageModelV3Prompt, LanguageModelV4FilePart, LanguageModelV4Prompt } from "@ai-sdk/provider";
|
||||
import {
|
||||
headroomMiddleware,
|
||||
compressVercelMessages,
|
||||
@@ -512,3 +512,87 @@ describe("withHeadroom", () => {
|
||||
expect(fakeModel.doGenerate).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("headroomMiddleware carries provider reasoning state", () => {
|
||||
beforeEach(() => {
|
||||
mockFetch.mockReset();
|
||||
});
|
||||
|
||||
it("restores reasoning parts and providerOptions after a compressed round trip", async () => {
|
||||
// Step two of a thinking-enabled tool loop, as the AI SDK hands it to transformParams.
|
||||
const prompt: LanguageModelV3Prompt = [
|
||||
{ role: "system", content: "Be brief.", providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } } },
|
||||
{ role: "user", content: [{ type: "text", text: "Read the gateway log." }] },
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "reasoning", text: "I should read the log first.", providerOptions: { anthropic: { signature: "SIG-abc123" } } },
|
||||
{ type: "text", text: "Reading the log now.", providerOptions: { google: { thoughtSignature: "TS-text" } } },
|
||||
{
|
||||
type: "tool-call",
|
||||
toolCallId: "call_01",
|
||||
toolName: "fetch_gateway_metrics",
|
||||
input: { path: "/var/log/gateway.json" },
|
||||
providerOptions: { google: { thoughtSignature: "TS-999" } },
|
||||
},
|
||||
],
|
||||
providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } },
|
||||
},
|
||||
{
|
||||
role: "tool",
|
||||
content: [
|
||||
{
|
||||
type: "tool-result",
|
||||
toolCallId: "call_01",
|
||||
toolName: "fetch_gateway_metrics",
|
||||
output: { type: "text", value: "row 1\nrow 2\nrow 3\nrow 4" },
|
||||
providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } },
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
// The proxy rewrites only the tool message's content and passes everything else through.
|
||||
mockFetch.mockImplementationOnce(async (_url: string, init: any) => {
|
||||
const messages = JSON.parse(init.body).messages;
|
||||
messages[3] = { ...messages[3], content: "[4 rows]" };
|
||||
return mockCompressResponse(messages);
|
||||
});
|
||||
|
||||
const middleware = headroomMiddleware({ baseUrl: "http://localhost:8787" });
|
||||
const out = await middleware.transformParams({ params: { prompt }, model: { specificationVersion: "v3" }, type: "generate" });
|
||||
|
||||
const expected = structuredClone(prompt) as any[];
|
||||
expected[3].content[0].output = { type: "text", value: "[4 rows]" };
|
||||
expect(out.prompt).toEqual(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe("headroomMiddleware keeps provider-executed tool calls in place", () => {
|
||||
beforeEach(() => {
|
||||
mockFetch.mockReset();
|
||||
});
|
||||
|
||||
it("returns an interleaved web-search turn in its original order after the round trip", async () => {
|
||||
const prompt: LanguageModelV3Prompt = [
|
||||
{ role: "user", content: [{ type: "text", text: "What is Headroom?" }] },
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "reasoning", text: "Search first.", providerOptions: { openai: { itemId: "rs_1" } } },
|
||||
{ type: "tool-call", toolCallId: "ws_1", toolName: "web_search", input: { query: "Headroom" }, providerExecuted: true },
|
||||
{ type: "tool-result", toolCallId: "ws_1", toolName: "web_search", output: { type: "json", value: { hits: 3 } } },
|
||||
{ type: "reasoning", text: "Three hits.", providerOptions: { openai: { itemId: "rs_2" } } },
|
||||
{ type: "text", text: "Headroom is a context-compression proxy." },
|
||||
],
|
||||
},
|
||||
{ role: "user", content: [{ type: "text", text: "Now read the gateway log." }] },
|
||||
];
|
||||
// The proxy passes this prompt through unchanged.
|
||||
mockFetch.mockImplementationOnce(async (_url: string, init: any) => mockCompressResponse(JSON.parse(init.body).messages));
|
||||
|
||||
const middleware = headroomMiddleware({ baseUrl: "http://localhost:8787" });
|
||||
const out = await middleware.transformParams({ params: { prompt }, model: { specificationVersion: "v3" }, type: "generate" });
|
||||
|
||||
expect(out.prompt).toEqual(prompt);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -235,3 +235,49 @@ describe("compress() sends Anthropic-format input to the proxy natively", () =>
|
||||
expect(result.messages).toEqual(returned);
|
||||
});
|
||||
});
|
||||
|
||||
describe("compress() honors a declared input format", () => {
|
||||
beforeEach(() => {
|
||||
mockFetch.mockReset();
|
||||
});
|
||||
|
||||
it("converts a text-only Vercel prompt both ways when format is declared, instead of autodetecting OpenAI", async () => {
|
||||
// Nothing in this prompt distinguishes it from OpenAI, so autodetection would hand it
|
||||
// to the proxy unconverted and return OpenAI shape (a string) to an AI SDK caller.
|
||||
const prompt = [{ role: "user", content: [{ type: "text", text: "hi" }] }];
|
||||
mockFetch.mockResolvedValueOnce(okResponse({ messages: [{ role: "user", content: "compressed" }] }));
|
||||
|
||||
const result = await compress(prompt, { baseUrl: "http://localhost:8787", format: "vercel" });
|
||||
|
||||
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(body.messages).toEqual([{ role: "user", content: "hi" }]);
|
||||
expect(result.messages).toEqual([{ role: "user", content: [{ type: "text", text: "compressed" }] }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("compress() keeps OpenAI attachments on the autodetection path when messages carry providerOptions", () => {
|
||||
beforeEach(() => {
|
||||
mockFetch.mockReset();
|
||||
});
|
||||
|
||||
it("passes an OpenAI user message with providerOptions, an image_url and a file part through unconverted", async () => {
|
||||
const messages = [
|
||||
{
|
||||
role: "user",
|
||||
providerOptions: { openai: {} },
|
||||
content: [
|
||||
{ type: "text", text: "Describe this" },
|
||||
{ type: "image_url", image_url: { url: "https://example.com/cat.png" } },
|
||||
{ type: "file", file: { file_id: "file-abc", filename: "notes.pdf" } },
|
||||
],
|
||||
},
|
||||
];
|
||||
mockFetch.mockResolvedValueOnce(okResponse({ messages }));
|
||||
|
||||
const result = await compress(messages, { baseUrl: "http://localhost:8787" });
|
||||
|
||||
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(body.messages).toEqual(messages);
|
||||
expect(result.messages).toEqual(messages);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -15,6 +15,9 @@ import type {
|
||||
HeadroomClientInterface,
|
||||
ProxyCompressResponse,
|
||||
ProxyErrorResponse,
|
||||
ProviderOptions,
|
||||
ReasoningContentPart,
|
||||
PassthroughContentPart,
|
||||
} from "../src/types.js";
|
||||
import type { CompressRequestConfig } from "../src/types/config.js";
|
||||
import {
|
||||
@@ -249,3 +252,36 @@ describe("Proxy response types (internal)", () => {
|
||||
}>();
|
||||
});
|
||||
});
|
||||
|
||||
describe("Provider carriers (Vercel reasoning state)", () => {
|
||||
it("ProviderOptions is a two-level record", () => {
|
||||
expectTypeOf<ProviderOptions>().toEqualTypeOf<Record<string, Record<string, unknown>>>();
|
||||
});
|
||||
|
||||
it("TextContentPart carries optional providerOptions", () => {
|
||||
expectTypeOf<TextContentPart["providerOptions"]>().toEqualTypeOf<ProviderOptions | undefined>();
|
||||
});
|
||||
|
||||
it("ReasoningContentPart is a ContentPart", () => {
|
||||
expectTypeOf<ReasoningContentPart["type"]>().toEqualTypeOf<"reasoning">();
|
||||
expectTypeOf<ReasoningContentPart["text"]>().toBeString();
|
||||
expectTypeOf<ReasoningContentPart>().toMatchTypeOf<ContentPart>();
|
||||
});
|
||||
|
||||
it("PassthroughContentPart is a ContentPart with literal types", () => {
|
||||
expectTypeOf<PassthroughContentPart["type"]>().toEqualTypeOf<"reasoning-file" | "custom" | "tool-result" | "tool-approval-request">();
|
||||
expectTypeOf<PassthroughContentPart>().toMatchTypeOf<ContentPart>();
|
||||
});
|
||||
|
||||
it("ToolCall carries optional providerOptions and providerExecuted", () => {
|
||||
expectTypeOf<ToolCall["providerOptions"]>().toEqualTypeOf<ProviderOptions | undefined>();
|
||||
expectTypeOf<ToolCall["providerExecuted"]>().toEqualTypeOf<boolean | undefined>();
|
||||
});
|
||||
|
||||
it("every message type carries optional providerOptions", () => {
|
||||
expectTypeOf<SystemMessage["providerOptions"]>().toEqualTypeOf<ProviderOptions | undefined>();
|
||||
expectTypeOf<UserMessage["providerOptions"]>().toEqualTypeOf<ProviderOptions | undefined>();
|
||||
expectTypeOf<AssistantMessage["providerOptions"]>().toEqualTypeOf<ProviderOptions | undefined>();
|
||||
expectTypeOf<ToolMessage["providerOptions"]>().toEqualTypeOf<ProviderOptions | undefined>();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -208,3 +208,62 @@ describe("detectFormat: Anthropic thinking blocks", () => {
|
||||
expect(detectFormat(messages)).toBe("anthropic");
|
||||
});
|
||||
});
|
||||
|
||||
describe("detectFormat: Vercel reasoning parts and providerOptions", () => {
|
||||
it("detects a turn whose only Vercel marker is a reasoning part", () => {
|
||||
const messages = [
|
||||
{ role: "user", content: "hi" },
|
||||
{ role: "assistant", content: [{ type: "reasoning", text: "..." }, { type: "text", text: "hello" }] },
|
||||
];
|
||||
expect(detectFormat(messages)).toBe("vercel");
|
||||
});
|
||||
|
||||
it("detects a message-level providerOptions key as Vercel", () => {
|
||||
const messages = [{ role: "user", content: "hi", providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } } }];
|
||||
expect(detectFormat(messages)).toBe("vercel");
|
||||
});
|
||||
});
|
||||
|
||||
describe("detectFormat: OpenAI content markers outrank message-level providerOptions", () => {
|
||||
// The public OpenAI message types allow `providerOptions`, so the key alone must not route
|
||||
// an OpenAI conversation through the Vercel converter, which keeps only text/image/file parts.
|
||||
it("keeps an OpenAI user message with providerOptions and an image_url part as openai", () => {
|
||||
const messages = [
|
||||
{
|
||||
role: "user",
|
||||
providerOptions: { openai: {} },
|
||||
content: [
|
||||
{ type: "text", text: "Describe this" },
|
||||
{ type: "image_url", image_url: { url: "https://example.com/cat.png" } },
|
||||
],
|
||||
},
|
||||
];
|
||||
expect(detectFormat(messages)).toBe("openai");
|
||||
});
|
||||
|
||||
it("keeps an OpenAI user message with providerOptions and a file part (file object) as openai", () => {
|
||||
const messages = [
|
||||
{
|
||||
role: "user",
|
||||
providerOptions: { openai: {} },
|
||||
content: [{ type: "text", text: "Summarize" }, { type: "file", file: { file_id: "file-abc" } }],
|
||||
},
|
||||
];
|
||||
expect(detectFormat(messages)).toBe("openai");
|
||||
});
|
||||
|
||||
it("keeps an OpenAI user message with providerOptions and an input_audio part as openai", () => {
|
||||
const messages = [
|
||||
{ role: "user", providerOptions: { openai: {} }, content: [{ type: "input_audio", input_audio: { data: "AAAA", format: "wav" } }] },
|
||||
];
|
||||
expect(detectFormat(messages)).toBe("openai");
|
||||
});
|
||||
|
||||
it("lets an image_url part on a later message outrank providerOptions on an earlier one", () => {
|
||||
const messages = [
|
||||
{ role: "system", content: "You describe images.", providerOptions: { openai: {} } },
|
||||
{ role: "user", content: [{ type: "image_url", image_url: { url: "https://example.com/cat.png" } }] },
|
||||
];
|
||||
expect(detectFormat(messages)).toBe("openai");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -223,17 +223,25 @@ describe("vercelToOpenAI", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("skips reasoning parts in assistant messages", () => {
|
||||
it("carries reasoning parts in assistant messages ahead of the text, verbatim", () => {
|
||||
const result = vercelToOpenAI([
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "reasoning", text: "thinking..." },
|
||||
{ type: "reasoning", text: "thinking...", providerOptions: { anthropic: { signature: "SIG" } } },
|
||||
{ type: "text", text: "answer" },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result).toEqual([
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "reasoning", text: "thinking...", providerOptions: { anthropic: { signature: "SIG" } } },
|
||||
{ type: "text", text: "answer" },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result).toEqual([{ role: "assistant", content: "answer" }]);
|
||||
});
|
||||
|
||||
it("handles full multi-turn conversation", () => {
|
||||
@@ -1459,3 +1467,206 @@ describe("Anthropic document sources (text, content, file) and document fields",
|
||||
expect(fromOpenAI(toOpenAI(input), "anthropic")).toEqual(input);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Vercel converters carry provider reasoning state through the OpenAI round trip", () => {
|
||||
// A LanguageModelV3 prompt as the AI SDK builds it on the second step of a
|
||||
// thinking-enabled tool loop: reasoning parts carry the Anthropic signature /
|
||||
// redacted data, text and tool-call parts carry Google thought signatures,
|
||||
// messages and parts carry Anthropic cache breakpoints.
|
||||
const prompt = [
|
||||
{ role: "system", content: "Be brief.", providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } } },
|
||||
{
|
||||
role: "user",
|
||||
content: [{ type: "text", text: "Read the gateway log.", providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } } }],
|
||||
providerOptions: { openai: { tag: "u1" } },
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "reasoning", text: "I should read the log first.", providerOptions: { anthropic: { signature: "SIG-abc123" } } },
|
||||
{ type: "reasoning", text: "", providerOptions: { anthropic: { redactedData: "REDACTED-xyz" } } },
|
||||
{ type: "text", text: "Reading the log now.", providerOptions: { google: { thoughtSignature: "TS-text" } } },
|
||||
{
|
||||
type: "tool-call",
|
||||
toolCallId: "call_01",
|
||||
toolName: "fetch_gateway_metrics",
|
||||
input: { path: "/var/log/gateway.json" },
|
||||
providerOptions: { google: { thoughtSignature: "TS-999" }, openai: { itemId: "fc_1" } },
|
||||
},
|
||||
],
|
||||
providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } },
|
||||
},
|
||||
{
|
||||
role: "tool",
|
||||
content: [
|
||||
{
|
||||
type: "tool-result",
|
||||
toolCallId: "call_01",
|
||||
toolName: "fetch_gateway_metrics",
|
||||
output: { type: "text", value: "row 1\nrow 2" },
|
||||
providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } },
|
||||
},
|
||||
],
|
||||
providerOptions: { openai: { tag: "t1" } },
|
||||
},
|
||||
];
|
||||
|
||||
it("round-trips the whole prompt exactly", () => {
|
||||
expect(openAIToVercel(vercelToOpenAI(prompt))).toEqual(prompt);
|
||||
});
|
||||
|
||||
it("keeps reasoning parts verbatim and in order inside the OpenAI content array", () => {
|
||||
const [assistant] = vercelToOpenAI([prompt[2]]);
|
||||
expect(assistant.content).toEqual([
|
||||
{ type: "reasoning", text: "I should read the log first.", providerOptions: { anthropic: { signature: "SIG-abc123" } } },
|
||||
{ type: "reasoning", text: "", providerOptions: { anthropic: { redactedData: "REDACTED-xyz" } } },
|
||||
{ type: "text", text: "Reading the log now.", providerOptions: { google: { thoughtSignature: "TS-text" } } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("carries providerOptions and providerExecuted on tool calls", () => {
|
||||
const [assistant] = vercelToOpenAI([
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "tool-call", toolCallId: "ws_1", toolName: "web_search", input: { q: "x" }, providerExecuted: true, providerOptions: { openai: { itemId: "ws_1" } } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(assistant.tool_calls).toEqual([
|
||||
{
|
||||
id: "ws_1",
|
||||
type: "function",
|
||||
function: { name: "web_search", arguments: '{"q":"x"}' },
|
||||
providerExecuted: true,
|
||||
providerOptions: { openai: { itemId: "ws_1" } },
|
||||
},
|
||||
]);
|
||||
const [back] = openAIToVercel([assistant]);
|
||||
expect(back.content).toEqual([
|
||||
{ type: "tool-call", toolCallId: "ws_1", toolName: "web_search", input: { q: "x" }, providerExecuted: true, providerOptions: { openai: { itemId: "ws_1" } } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("carries a provider-executed tool-result part in assistant content verbatim", () => {
|
||||
const part = {
|
||||
type: "tool-result",
|
||||
toolCallId: "ws_1",
|
||||
toolName: "web_search",
|
||||
output: { type: "json", value: { hits: 3 } },
|
||||
providerExecuted: true,
|
||||
};
|
||||
const message = { role: "assistant", content: [{ type: "text", text: "Searching." }, part] };
|
||||
const [converted] = vercelToOpenAI([message]);
|
||||
expect(converted.content).toEqual([{ type: "text", text: "Searching." }, part]);
|
||||
expect(openAIToVercel([converted])).toEqual([message]);
|
||||
});
|
||||
|
||||
it("carries a tool-approval-response part verbatim", () => {
|
||||
const approval = { type: "tool-approval-response", approvalId: "ap_1", approved: true, providerExecuted: true };
|
||||
const converted = vercelToOpenAI([
|
||||
{ role: "assistant", content: [{ type: "tool-call", toolCallId: "call_1", toolName: "deploy", input: {} }] },
|
||||
{ role: "tool", content: [{ type: "tool-result", toolCallId: "call_1", toolName: "deploy", output: { type: "text", value: "done" } }, approval] },
|
||||
]);
|
||||
const back = openAIToVercel(converted);
|
||||
expect(back.flatMap((m: any) => m.content)).toContainEqual(approval);
|
||||
expect(back.flatMap((m: any) => m.content)).toContainEqual({ type: "tool-result", toolCallId: "call_1", toolName: "deploy", output: { type: "text", value: "done" } });
|
||||
});
|
||||
|
||||
it("keeps a user text part with providerOptions as a part, and a plain one as a string", () => {
|
||||
const [withOptions] = vercelToOpenAI([prompt[1]]);
|
||||
expect(withOptions.content).toEqual([{ type: "text", text: "Read the gateway log.", providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } } }]);
|
||||
const [plain] = vercelToOpenAI([{ role: "user", content: [{ type: "text", text: "hi" }] }]);
|
||||
expect(plain.content).toBe("hi");
|
||||
});
|
||||
|
||||
it("still collapses a text-only assistant turn without providerOptions to a string", () => {
|
||||
expect(vercelToOpenAI([{ role: "assistant", content: [{ type: "text", text: "a" }, { type: "text", text: "b" }] }])).toEqual([
|
||||
{ role: "assistant", content: "ab" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Vercel media parts carry providerOptions through the image_url carrier", () => {
|
||||
it("round-trips providerOptions on a file part", () => {
|
||||
const message = {
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "see attached" },
|
||||
{ type: "file", mediaType: "application/pdf", data: "JVBERi0=", providerOptions: { anthropic: { citations: { enabled: true } } } },
|
||||
],
|
||||
};
|
||||
const [converted] = vercelToOpenAI([message]);
|
||||
expect((converted.content as any[])[1].image_url.providerOptions).toEqual({ anthropic: { citations: { enabled: true } } });
|
||||
expect(openAIToVercel([converted])).toEqual([message]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Vercel converters keep tool calls in their original place among the content parts", () => {
|
||||
it("round-trips a tool call ahead of a reasoning part (tool-call, reasoning, tool-call)", () => {
|
||||
const message = {
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "tool-call", toolCallId: "call_a", toolName: "lookup", input: { id: 1 } },
|
||||
{ type: "reasoning", text: "Now the second one.", providerOptions: { openai: { itemId: "rs_1" } } },
|
||||
{ type: "tool-call", toolCallId: "call_b", toolName: "lookup", input: { id: 2 } },
|
||||
],
|
||||
};
|
||||
expect(openAIToVercel(vercelToOpenAI([message]))).toEqual([message]);
|
||||
});
|
||||
|
||||
it("round-trips a provider-executed search turn (reasoning, tool-call, tool-result, reasoning, text) exactly", () => {
|
||||
const message = {
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "reasoning", text: "Search first.", providerOptions: { openai: { itemId: "rs_1" } } },
|
||||
{ type: "tool-call", toolCallId: "ws_1", toolName: "web_search", input: { q: "headroom" }, providerExecuted: true, providerOptions: { openai: { itemId: "ws_1" } } },
|
||||
{ type: "tool-result", toolCallId: "ws_1", toolName: "web_search", output: { type: "json", value: { hits: 3 } } },
|
||||
{ type: "reasoning", text: "Three hits.", providerOptions: { openai: { itemId: "rs_2" } } },
|
||||
{ type: "text", text: "Found three results." },
|
||||
],
|
||||
};
|
||||
const [converted] = vercelToOpenAI([message]);
|
||||
// OpenAI keeps tool calls in a separate list, so the call carries where it sat: after one content part.
|
||||
expect(converted.content).toHaveLength(4);
|
||||
expect((converted.tool_calls as any[])[0].position).toBe(1);
|
||||
expect(openAIToVercel([converted])).toEqual([message]);
|
||||
});
|
||||
|
||||
it("round-trips a tool call ahead of a text part", () => {
|
||||
const message = {
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "tool-call", toolCallId: "call_1", toolName: "search", input: { q: "x" } },
|
||||
{ type: "text", text: "Searching." },
|
||||
],
|
||||
};
|
||||
expect(openAIToVercel(vercelToOpenAI([message]))).toEqual([message]);
|
||||
});
|
||||
|
||||
it("still sends text followed by tool calls as a string plus tool_calls, with no position", () => {
|
||||
const [converted] = vercelToOpenAI([
|
||||
{ role: "assistant", content: [{ type: "text", text: "Searching." }, { type: "tool-call", toolCallId: "call_1", toolName: "search", input: { q: "x" } }] },
|
||||
]);
|
||||
expect(converted.content).toBe("Searching.");
|
||||
expect(converted.tool_calls).toEqual([{ id: "call_1", type: "function", function: { name: "search", arguments: '{"q":"x"}' } }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Vercel tool messages with several results carry message-level providerOptions once", () => {
|
||||
it("puts the message's providerOptions on the last of the split OpenAI tool messages only", () => {
|
||||
const message = {
|
||||
role: "tool",
|
||||
content: [
|
||||
{ type: "tool-result", toolCallId: "call_1", toolName: "a", output: { type: "text", value: "A" } },
|
||||
{ type: "tool-result", toolCallId: "call_2", toolName: "b", output: { type: "text", value: "B" } },
|
||||
],
|
||||
providerOptions: { anthropic: { cacheControl: { type: "ephemeral" } } },
|
||||
};
|
||||
const converted = vercelToOpenAI([message]);
|
||||
expect((converted[0] as any).messageProviderOptions).toBeUndefined();
|
||||
expect((converted[1] as any).messageProviderOptions).toEqual({ anthropic: { cacheControl: { type: "ephemeral" } } });
|
||||
const back = openAIToVercel(converted);
|
||||
expect(back.map((m: any) => m.providerOptions)).toEqual([undefined, { anthropic: { cacheControl: { type: "ephemeral" } } }]);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user