mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(sdk): preserve Anthropic image blocks in message conversion instead of dropping the turn (#3439)
## Description
The TypeScript SDK's `anthropicToOpenAI()` converter drops image content
when translating Anthropic-format messages to OpenAI format for
compression. For a `user` message with array `content`, it only extracts
`text` and `tool_result` blocks — there is no branch for `image` blocks.
So:
- An **image-only** user turn (a normal vision request) produces
`textBlocks.length === 0` with no tool_results, and **nothing is
pushed** — the entire conversation turn is silently deleted, breaking
role alternation and losing the image.
- A **mixed text+image** turn keeps the text but silently drops the
image.
`detectFormat()` already recognizes Anthropic images (`{ type: "image",
source: { type } }`), and the sibling `vercelToOpenAI()` already
converts images to OpenAI `image_url` parts — so images are a known,
intended input; the Anthropic converter simply had no branch for them.
The bug exists in two copies, both on real user paths:
- `sdk/typescript/src/utils/format.ts` — exported `anthropicToOpenAI`,
used by `toOpenAI()` on the main `compress()` path.
- `sdk/typescript/src/adapters/anthropic.ts` — the private helper used
by `withHeadroom()` for the Anthropic SDK.
This adds image handling to both: Anthropic image blocks are converted
to OpenAI `image_url` content parts (base64 source → a `data:` URI, url
source → the url), a turn containing images is emitted as a multimodal
`content` array with original block order preserved, and unconvertible
image blocks are skipped without deleting the turn. Text-only turns are
unchanged (still a joined string), so no regression on the common path.
**Round trip:** the reverse conversion (`openAIToAnthropic`, in both
files) also mapped every non-text part to empty text, so on a successful
compression the newly-preserved `image_url` was turned into an empty
text block before reaching the wrapped Anthropic client — the image was
lost anyway. This PR also fixes the reverse mapping (`image_url` →
Anthropic image block: a `data:<media_type>;base64,<data>` URI → a
base64 source, any other url → a url source), so images survive the full
`withHeadroom()` round trip through to `messages.create`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`sdk/typescript/src/utils/format.ts`** —
- `anthropicToOpenAI` user-branch (forward): handle `image` blocks.
Added `anthropicImageUrl(source)` (base64 →
`data:<media_type>;base64,<data>`, url → `source.url`, else `null`).
When a user turn has image blocks, emit one `{ role: "user", content:
ContentPart[] }` preserving text/image order; unconvertible images are
skipped. Text-only and tool_result handling unchanged.
- `openAIToAnthropic` user-branch (reverse): added
`openAIImageBlock(url)` and map `image_url` parts back to Anthropic
image blocks (data URI → base64 source, other url → url source).
- **`sdk/typescript/src/adapters/anthropic.ts`** — same forward and
reverse fixes applied to the duplicated private helpers (kept
self-contained; existing `tool_result`/assistant handling preserved
unchanged).
- **Tests** — forward, reverse, round-trip, and adapter end-to-end cases
(see Testing).
## Testing
- [x] Unit tests pass (`npm test` — vitest)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ cd sdk/typescript && npm test
Test Files 18 passed | 5 skipped (23)
Tests 308 passed | 33 skipped (340)
$ npm run typecheck # tsc --noEmit
(clean, no errors)
$ npm run build # tsup
(ESM + CJS + DTS built successfully)
```
New tests (`test/utils/format.test.ts`,
`test/adapters/anthropic.test.ts`):
- **forward** (`anthropicToOpenAI`): image-only user message → returns a
message (not `[]`) with an `image_url` part; base64 → `data:` URI; url →
passthrough; text+image → array, order preserved; text-only → unchanged
(string); image+tool_result → user message plus separate tool message;
- **reverse** (`openAIToAnthropic`): `image_url` → Anthropic image block
(data URI → base64 source; http → url source); mixed order preserved;
- **round trip**: `openAIToAnthropic(anthropicToOpenAI(input))`
reproduces image-only and mixed turns exactly;
- **adapter end-to-end** (`withHeadroom`): with a mocked compression
response that retains the image, the original `messages.create` receives
the Anthropic image block (image-only and mixed turns) — not empty text.
## Real Behavior Proof
- Environment: Node v22.16.0, Windows 11; TypeScript SDK from this
branch (`cd sdk/typescript && npm install`).
- Exact command / steps: `cd sdk/typescript && npm test` (vitest), `npm
run typecheck` (tsc --noEmit), `npm run build` (tsup). The new tests
drive the real exported `anthropicToOpenAI` and assert the returned
OpenAI messages.
- Observed result: Before the fix, `anthropicToOpenAI([{ role: "user",
content: [{ type: "image", source: {...} }] }])` returns `[]` (the turn
is dropped); and even a preserved image was flattened to empty text by
the reverse conversion before reaching the wrapped client. After: the
forward pass returns a `user` message carrying the image as an
`image_url` part (base64 → `data:` URI, url → passthrough, order
preserved, text-only still a string), and the reverse pass restores it
to an Anthropic image block — the adapter test asserts the original
`messages.create` receives the image block (image-only and mixed turns)
on a successful compression. `npm test`: 308 passed | 33 skipped,
typecheck + build clean.
- Not tested: a live proxy round-trip. The conversion is pure and fully
covered by unit tests; the proxy consumes standard OpenAI-format
messages.
## Runtime Rollout Safety
- Rollout-managed feature(s): None. Client-side TypeScript SDK
message-conversion fix; no server/runtime feature flag or rollout
channel involved.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Only the buggy path changes — image
content that was previously dropped is now preserved. Text-only and
tool_result conversions are byte-identical to before (no regression on
the common path).
- Kill switch / disable path: N/A — this is a correctness fix, not a
gated feature.
- Unsafe override required: No.
- Qualification impact: None — additive image handling; existing
conversion tests unchanged and green.
- Rollback path: Revert this PR; no persisted state, migrations, or
config to unwind.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
- Both directions of the conversion are fixed so the `withHeadroom()`
round trip preserves images. One related cleanup is left as a follow-up
rather than widening this PR: the image conversion (and the converters
generally) are duplicated across `utils/format.ts` and
`adapters/anthropic.ts` (with slightly different `tool_result` handling)
— a candidate for later de-duplication into the shared util. Happy to
file/fix that separately.
- CHANGELOG left untouched — the `no-manual-changelog` CI guard owns it;
the Conventional Commit title drives the entry.
---------
Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Jerrett Davis
parent
95cbbb8158
commit
63676e6cff
@@ -11,6 +11,23 @@ interface AnthropicLike {
|
||||
[key: string]: any;
|
||||
}
|
||||
|
||||
// Anthropic image block `source` -> OpenAI image_url url string, or null if unconvertible.
|
||||
function anthropicImageUrl(source: any): string | null {
|
||||
if (!source || typeof source !== "object") return null;
|
||||
if (source.type === "base64" && source.media_type && source.data) {
|
||||
return `data:${source.media_type};base64,${source.data}`;
|
||||
}
|
||||
if (source.type === "url" && typeof source.url === "string") return source.url;
|
||||
return null;
|
||||
}
|
||||
|
||||
// OpenAI image_url url -> Anthropic image block. data: URIs become a base64 source; other urls a url source.
|
||||
function openAIImageBlock(url: string): any {
|
||||
const m = /^data:([^;,]+);base64,(.*)$/s.exec(url);
|
||||
if (m) return { type: "image", source: { type: "base64", media_type: m[1], data: m[2] } };
|
||||
return { type: "image", source: { type: "url", url } };
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert Anthropic messages to OpenAI format for compression.
|
||||
*
|
||||
@@ -29,13 +46,29 @@ function anthropicToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
const toolResults = msg.content.filter(
|
||||
(b: any) => b.type === "tool_result",
|
||||
);
|
||||
const textBlocks = msg.content.filter((b: any) => b.type === "text");
|
||||
const hasImages = msg.content.some((b: any) => b.type === "image");
|
||||
|
||||
if (textBlocks.length > 0) {
|
||||
result.push({
|
||||
role: "user",
|
||||
content: textBlocks.map((b: any) => b.text).join("\n"),
|
||||
});
|
||||
if (hasImages) {
|
||||
const parts = msg.content
|
||||
.filter((b: any) => b.type === "text" || b.type === "image")
|
||||
.map((b: any) => {
|
||||
if (b.type === "text") return { type: "text" as const, text: b.text };
|
||||
const url = anthropicImageUrl(b.source);
|
||||
return url ? { type: "image_url" as const, image_url: { url } } : null;
|
||||
})
|
||||
.filter((p: any): p is NonNullable<typeof p> => p !== null);
|
||||
|
||||
if (parts.length > 0) {
|
||||
result.push({ role: "user", content: parts });
|
||||
}
|
||||
} else {
|
||||
const textBlocks = msg.content.filter((b: any) => b.type === "text");
|
||||
if (textBlocks.length > 0) {
|
||||
result.push({
|
||||
role: "user",
|
||||
content: textBlocks.map((b: any) => b.text).join("\n"),
|
||||
});
|
||||
}
|
||||
}
|
||||
for (const tr of toolResults) {
|
||||
result.push({
|
||||
@@ -102,6 +135,7 @@ function openAIToAnthropic(messages: OpenAIMessage[]): any[] {
|
||||
role: "user",
|
||||
content: msg.content.map((p) => {
|
||||
if (p.type === "text") return { type: "text", text: p.text };
|
||||
if (p.type === "image_url") return openAIImageBlock(p.image_url.url);
|
||||
return { type: "text", text: "" };
|
||||
}),
|
||||
});
|
||||
|
||||
@@ -71,6 +71,23 @@ export function detectFormat(messages: any[]): MessageFormat {
|
||||
// Anthropic → OpenAI
|
||||
// ============================================================
|
||||
|
||||
// Anthropic image block `source` -> OpenAI image_url url string, or null if unconvertible.
|
||||
function anthropicImageUrl(source: any): string | null {
|
||||
if (!source || typeof source !== "object") return null;
|
||||
if (source.type === "base64" && source.media_type && source.data) {
|
||||
return `data:${source.media_type};base64,${source.data}`;
|
||||
}
|
||||
if (source.type === "url" && typeof source.url === "string") return source.url;
|
||||
return null;
|
||||
}
|
||||
|
||||
// OpenAI image_url url -> Anthropic image block. data: URIs become a base64 source; other urls a url source.
|
||||
function openAIImageBlock(url: string): any {
|
||||
const m = /^data:([^;,]+);base64,(.*)$/s.exec(url);
|
||||
if (m) return { type: "image", source: { type: "base64", media_type: m[1], data: m[2] } };
|
||||
return { type: "image", source: { type: "url", url } };
|
||||
}
|
||||
|
||||
export function anthropicToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
const result: OpenAIMessage[] = [];
|
||||
|
||||
@@ -81,14 +98,30 @@ export function anthropicToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
continue;
|
||||
}
|
||||
if (Array.isArray(msg.content)) {
|
||||
const textBlocks = msg.content.filter((b: any) => b.type === "text");
|
||||
const toolResults = msg.content.filter((b: any) => b.type === "tool_result");
|
||||
const hasImages = msg.content.some((b: any) => b.type === "image");
|
||||
|
||||
if (textBlocks.length > 0) {
|
||||
result.push({
|
||||
role: "user",
|
||||
content: textBlocks.map((b: any) => b.text).join("\n"),
|
||||
});
|
||||
if (hasImages) {
|
||||
const parts = msg.content
|
||||
.filter((b: any) => b.type === "text" || b.type === "image")
|
||||
.map((b: any) => {
|
||||
if (b.type === "text") return { type: "text" as const, text: b.text };
|
||||
const url = anthropicImageUrl(b.source);
|
||||
return url ? { type: "image_url" as const, image_url: { url } } : null;
|
||||
})
|
||||
.filter((p: any): p is NonNullable<typeof p> => p !== null);
|
||||
|
||||
if (parts.length > 0) {
|
||||
result.push({ role: "user", content: parts });
|
||||
}
|
||||
} else {
|
||||
const textBlocks = msg.content.filter((b: any) => b.type === "text");
|
||||
if (textBlocks.length > 0) {
|
||||
result.push({
|
||||
role: "user",
|
||||
content: textBlocks.map((b: any) => b.text).join("\n"),
|
||||
});
|
||||
}
|
||||
}
|
||||
for (const tr of toolResults) {
|
||||
const content = typeof tr.content === "string"
|
||||
@@ -155,9 +188,11 @@ export function openAIToAnthropic(messages: OpenAIMessage[]): any[] {
|
||||
} else if (Array.isArray(msg.content)) {
|
||||
result.push({
|
||||
role: "user",
|
||||
content: msg.content.map((p) =>
|
||||
p.type === "text" ? { type: "text", text: p.text } : { type: "text", text: "" },
|
||||
),
|
||||
content: msg.content.map((p) => {
|
||||
if (p.type === "text") return { type: "text", text: p.text };
|
||||
if (p.type === "image_url") return openAIImageBlock(p.image_url.url);
|
||||
return { type: "text", text: "" };
|
||||
}),
|
||||
});
|
||||
}
|
||||
continue;
|
||||
|
||||
@@ -190,6 +190,140 @@ describe("withHeadroom (Anthropic)", () => {
|
||||
expect(createArgs.messages.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("preserves image blocks instead of dropping them during compression", async () => {
|
||||
mockFetch.mockResolvedValueOnce(mockCompressSuccess());
|
||||
|
||||
const mockCreate = vi.fn().mockResolvedValue({});
|
||||
const fakeClient = { messages: { create: mockCreate } };
|
||||
|
||||
const wrapped = withHeadroom(fakeClient as any, {
|
||||
baseUrl: "http://localhost:8787",
|
||||
});
|
||||
|
||||
await wrapped.messages.create({
|
||||
model: "claude-sonnet-4-5-20250929",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{
|
||||
type: "image",
|
||||
source: { type: "base64", media_type: "image/png", data: "AAAA" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
max_tokens: 1024,
|
||||
});
|
||||
|
||||
// The image must not be silently dropped from the compression request.
|
||||
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(body.messages).toHaveLength(1);
|
||||
expect(body.messages[0].content).toEqual([
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("converts a retained image_url back to an Anthropic image block for the final messages.create call (image-only turn)", async () => {
|
||||
// Proxy retains the image through compression (successful-compression path).
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
mockCompressSuccess([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const mockCreate = vi.fn().mockResolvedValue({});
|
||||
const fakeClient = { messages: { create: mockCreate } };
|
||||
|
||||
const wrapped = withHeadroom(fakeClient as any, {
|
||||
baseUrl: "http://localhost:8787",
|
||||
});
|
||||
|
||||
await wrapped.messages.create({
|
||||
model: "claude-sonnet-4-5-20250929",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "image",
|
||||
source: { type: "base64", media_type: "image/png", data: "AAAA" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
max_tokens: 1024,
|
||||
});
|
||||
|
||||
expect(mockCreate).toHaveBeenCalledOnce();
|
||||
const createArgs = mockCreate.mock.calls[0][0];
|
||||
expect(createArgs.messages).toEqual([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("converts a retained mixed text/image turn back to Anthropic blocks, order preserved, for the final messages.create call", async () => {
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
mockCompressSuccess([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,BBBB" } },
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const mockCreate = vi.fn().mockResolvedValue({});
|
||||
const fakeClient = { messages: { create: mockCreate } };
|
||||
|
||||
const wrapped = withHeadroom(fakeClient as any, {
|
||||
baseUrl: "http://localhost:8787",
|
||||
});
|
||||
|
||||
await wrapped.messages.create({
|
||||
model: "claude-sonnet-4-5-20250929",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{
|
||||
type: "image",
|
||||
source: { type: "base64", media_type: "image/jpeg", data: "BBBB" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
max_tokens: 1024,
|
||||
});
|
||||
|
||||
expect(mockCreate).toHaveBeenCalledOnce();
|
||||
const createArgs = mockCreate.mock.calls[0][0];
|
||||
expect(createArgs.messages).toEqual([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image", source: { type: "base64", media_type: "image/jpeg", data: "BBBB" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("returns original messages on compression fallback", async () => {
|
||||
// Simulate proxy unreachable
|
||||
mockFetch.mockRejectedValueOnce(new TypeError("fetch failed"));
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { vercelToOpenAI, openAIToVercel } from "../../src/utils/format.js";
|
||||
import { vercelToOpenAI, openAIToVercel, anthropicToOpenAI, openAIToAnthropic } from "../../src/utils/format.js";
|
||||
import type { OpenAIMessage } from "../../src/types.js";
|
||||
|
||||
describe("vercelToOpenAI", () => {
|
||||
@@ -269,6 +269,191 @@ describe("vercelToOpenAI", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("anthropicToOpenAI", () => {
|
||||
it("converts image-only user message to a content-parts array (not dropped)", () => {
|
||||
const result = anthropicToOpenAI([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "image",
|
||||
source: { type: "base64", media_type: "image/png", data: "AAAA" },
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0]).toEqual({
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "data:image/png;base64,AAAA" },
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("converts a url-sourced image as a passthrough url", () => {
|
||||
const result = anthropicToOpenAI([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image", source: { type: "url", url: "https://example.com/cat.png" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result[0].content).toEqual([
|
||||
{ type: "image_url", image_url: { url: "https://example.com/cat.png" } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("preserves order of text + image blocks", () => {
|
||||
const result = anthropicToOpenAI([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{
|
||||
type: "image",
|
||||
source: { type: "base64", media_type: "image/jpeg", data: "BBBB" },
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result[0].content).toEqual([
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,BBBB" } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("leaves text-only user messages as a flat string (backward compat)", () => {
|
||||
const result = anthropicToOpenAI([
|
||||
{ role: "user", content: [{ type: "text", text: "hello" }] },
|
||||
]);
|
||||
expect(result).toEqual([{ role: "user", content: "hello" }]);
|
||||
});
|
||||
|
||||
it("emits a separate tool message alongside an image in the same user turn", () => {
|
||||
const result = anthropicToOpenAI([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "image",
|
||||
source: { type: "base64", media_type: "image/png", data: "CCCC" },
|
||||
},
|
||||
{ type: "tool_result", tool_use_id: "tu_1", content: "ok" },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result).toHaveLength(2);
|
||||
expect(result[0]).toEqual({
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image_url", image_url: { url: "data:image/png;base64,CCCC" } },
|
||||
],
|
||||
});
|
||||
expect(result[1]).toEqual({
|
||||
role: "tool",
|
||||
content: "ok",
|
||||
tool_call_id: "tu_1",
|
||||
});
|
||||
});
|
||||
|
||||
it("drops an unconvertible lone image block (no usable url) rather than pushing a broken part", () => {
|
||||
const result = anthropicToOpenAI([
|
||||
{ role: "user", content: [{ type: "image", source: { type: "unknown" } }] },
|
||||
]);
|
||||
expect(result).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("openAIToAnthropic", () => {
|
||||
it("converts a base64 image_url part back to an Anthropic base64 image block", () => {
|
||||
const result = openAIToAnthropic([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image_url", image_url: { url: "data:image/png;base64,ABC" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result).toEqual([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image", source: { type: "base64", media_type: "image/png", data: "ABC" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("converts an http(s) image_url part back to an Anthropic url image block", () => {
|
||||
const result = openAIToAnthropic([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image_url", image_url: { url: "https://example.com/cat.png" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result).toEqual([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image", source: { type: "url", url: "https://example.com/cat.png" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("preserves order of text + image_url parts", () => {
|
||||
const result = openAIToAnthropic([
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,BBBB" } },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result[0].content).toEqual([
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image", source: { type: "base64", media_type: "image/jpeg", data: "BBBB" } },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("round-trip: anthropicToOpenAI then openAIToAnthropic", () => {
|
||||
it("preserves an image-only turn", () => {
|
||||
const original = [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },
|
||||
],
|
||||
},
|
||||
];
|
||||
const result = openAIToAnthropic(anthropicToOpenAI(original));
|
||||
expect(result).toEqual(original);
|
||||
});
|
||||
|
||||
it("preserves a mixed text + image turn, in order", () => {
|
||||
const original = [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "what is this?" },
|
||||
{ type: "image", source: { type: "base64", media_type: "image/jpeg", data: "BBBB" } },
|
||||
],
|
||||
},
|
||||
];
|
||||
const result = openAIToAnthropic(anthropicToOpenAI(original));
|
||||
expect(result).toEqual(original);
|
||||
});
|
||||
});
|
||||
|
||||
describe("openAIToVercel", () => {
|
||||
it("converts system message (passthrough)", () => {
|
||||
const result = openAIToVercel([
|
||||
|
||||
Reference in New Issue
Block a user