mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 12:05:34 +08:00
fix(sdk): accept string contents and functionCall without args in the Gemini adapter (#4756)
## Description Two inputs made the TypeScript SDK's Gemini path throw instead of compressing or passing through. Both are fixed, and the shorthand-prompt handling was hardened following Devin Review feedback. Closes #4741 1. **Plain-string contents.** `withHeadroom(model).generateContent("text")` (or a part array) reached `detectFormat`, which evaluates `"parts" in msg` on a primitive and threw `TypeError: Cannot use 'in' operator to search for 'parts' in text`. 2. **`functionCall` without `args`.** `args` is optional in the Gemini API. `geminiToOpenAI` wrote `JSON.stringify(undefined)`, and `compress()` always ran `fromOpenAI()` — even when the proxy was unreachable and the fallback returned the input unchanged — so `openAIToGemini` threw `SyntaxError: "undefined" is not valid JSON` on `JSON.parse(tc.function.arguments)`. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `sdk/typescript/src/adapters/gemini.ts` - `toContents()` wraps a shorthand prompt. The Gemini SDK reads a bare string — or a top-level array of strings/Parts — as the **parts of a single user turn**, so it becomes one `{ role: "user", parts: [...] }` Content before format detection. Treating each array element as its own turn would split turns and drop a bare media Part. - The duplicated generate/stream bodies are one `compressParams()`, which returns the original `params` untouched when `result.compressed` is false — the guard `anthropic.ts` and `vercel-ai.ts` already use. - A compressed shorthand prompt returns `{ contents: result.messages }`; handing the bare Contents array to the SDK would make it read them as the parts of one turn. Real request objects keep their other settings. - `sdk/typescript/src/compress.ts` - A request containing a non-object entry now passes through untouched (`compressed: false`, proxy not called) instead of being converted and silently losing those entries — CONTRIBUTING: "malformed content passes through unchanged". - `sdk/typescript/src/utils/format.ts` - `detectFormat()` skips non-object entries instead of throwing on them. - `geminiToOpenAI()` uses `JSON.stringify(args ?? {})`. - `openAIToGemini()` parses `arguments ?? "{}"` in a `try/catch`; a malformed value falls back to the raw string rather than `{}`, so no tool input is discarded (matching the Vercel conversion). - Tests: `test/adapters/gemini.test.ts` (string and `["describe", { inlineData }]` prompts reach the proxy as one user turn with the image intact; a compressed string prompt comes back as a request object; string params pass through on an unreachable proxy) and `test/compress.test.ts` (round-trip of a `functionCall` with no `args` through an unreachable `HeadroomClient`; a mixed non-object array passes through untouched). ## Testing - [ ] Unit tests pass (`pytest`) — N/A, TypeScript SDK only - [ ] Linting passes (`ruff check .`) — N/A, TypeScript SDK only - [ ] Type checking passes (`mypy headroom`) — N/A, TypeScript SDK only - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cd sdk/typescript && npx vitest run RUN v5.1.0 G:/.../sdk/typescript Test Files 19 passed | 5 skipped (24) Tests 459 passed | 33 skipped (492) Type Errors no errors ``` The reported `TypeError` and `SyntaxError` reproduced exactly as in the issue before the fix, and each new test was confirmed to fail first. ## Real Behavior Proof - Environment: Windows 11, Node v26.10.0, `sdk/typescript` on this branch. No Gemini API key, no real proxy. - Exact command / steps: `cd sdk/typescript && npx vitest run test/adapters/gemini.test.ts test/compress.test.ts` - Observed result: `generateContent("text")` now reaches the proxy as `[{ role: "user", content: "text" }]` and returns `{ contents: [...] }` to the model; `generateContent(["describe", { inlineData }])` keeps text and image in one turn. A Gemini `functionCall` with no `args` goes out as `arguments: "{}"` and comes back as `{ functionCall: { name: "f", args: {} } }` instead of throwing. On an unreachable proxy with `fallback: true`, the original params/messages are returned unchanged. - Not tested: a live Gemini API call, and a real proxy request carrying a `functionCall` with omitted `args`. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: no — only previously-throwing inputs change behavior - Kill switch / disable path: N/A - Unsafe override required: no - Qualification impact: none - Rollback path: revert the commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --- ### CI note `npm audit (plugins/opencode)` is red but **pre-existing on `main`**, not caused by this PR: `effect` (`GHSA-chp3-j5qr-2vfr`, cookie attribute injection in `@effect/platform` `makeCookie()`) via `@opencode-ai/plugin`. It reproduces on a clean `8f04221b` checkout, `plugins/opencode/package-lock.json` is untouched here, and `npm audit (sdk/typescript)` passes. Left alone per CONTRIBUTING ("Test/CI-only PR chasing a known `main` failure — **Don't.**"); happy to take it separately. ### Out of scope The issue's "Additional Context" deserves its own issue: `geminiToOpenAI` / `openAIToGemini` still drop `thoughtSignature`, `thought` and `functionCall.id`, and synthesize call ids as `gemini_${name}`, so two calls to the same function in one conversation collide. Fixing it changes the conversion contract, so it is kept separate from these crashes. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -14,6 +14,22 @@ interface GeminiModelLike {
|
||||
[key: string]: any;
|
||||
}
|
||||
|
||||
/**
|
||||
* Gemini accepts a bare string, or an array of strings/Parts, as shorthand for
|
||||
* the parts of a single user turn. Either way that is one Content, not a
|
||||
* conversation, so it has to be wrapped before format detection — which only
|
||||
* understands `{ role, parts }` objects and would throw on a primitive.
|
||||
*/
|
||||
function toContents(params: any): any[] {
|
||||
if (typeof params === "string" || Array.isArray(params)) {
|
||||
const parts = (Array.isArray(params) ? params : [params]).map((p: any) =>
|
||||
typeof p === "string" ? { text: p } : p,
|
||||
);
|
||||
return [{ role: "user", parts }];
|
||||
}
|
||||
return params?.contents ?? [params];
|
||||
}
|
||||
|
||||
/**
|
||||
* Wrap a Google Generative AI model to auto-compress before each request.
|
||||
*
|
||||
@@ -37,39 +53,32 @@ export function withHeadroom<T extends GeminiModelLike>(
|
||||
const originalGenerate = model.generateContent.bind(model);
|
||||
const originalStream = model.generateContentStream?.bind(model);
|
||||
|
||||
const compressedGenerate = async (params: any) => {
|
||||
const contents = params.contents ?? params;
|
||||
const compressParams = async (params: any): Promise<any> => {
|
||||
const modelName = options.model ?? "gemini-2.0-flash";
|
||||
|
||||
// compress() auto-detects Gemini format
|
||||
const result = await compress(
|
||||
Array.isArray(contents) ? contents : [contents],
|
||||
{ stack: "adapter_ts_gemini", ...options, model: modelName },
|
||||
);
|
||||
const result = await compress(toContents(params), {
|
||||
stack: "adapter_ts_gemini",
|
||||
...options,
|
||||
model: modelName,
|
||||
});
|
||||
|
||||
const newParams = Array.isArray(params)
|
||||
? result.messages
|
||||
// Unchanged request (proxy down / nothing to compress): hand the original
|
||||
// params through untouched, as the Anthropic and Vercel adapters do.
|
||||
if (!result.compressed) return params;
|
||||
|
||||
// A bare string or part array is not a request object: rebuild one, or the
|
||||
// SDK would read the returned Contents as the parts of a single turn.
|
||||
return Array.isArray(params) || typeof params === "string"
|
||||
? { contents: result.messages }
|
||||
: { ...params, contents: result.messages };
|
||||
|
||||
return originalGenerate(newParams);
|
||||
};
|
||||
|
||||
const compressedGenerate = async (params: any) =>
|
||||
originalGenerate(await compressParams(params));
|
||||
|
||||
const compressedStream = originalStream
|
||||
? async (params: any) => {
|
||||
const contents = params.contents ?? params;
|
||||
const modelName = options.model ?? "gemini-2.0-flash";
|
||||
|
||||
const result = await compress(
|
||||
Array.isArray(contents) ? contents : [contents],
|
||||
{ stack: "adapter_ts_gemini", ...options, model: modelName },
|
||||
);
|
||||
|
||||
const newParams = Array.isArray(params)
|
||||
? result.messages
|
||||
: { ...params, contents: result.messages };
|
||||
|
||||
return originalStream(newParams);
|
||||
}
|
||||
? async (params: any) => originalStream(await compressParams(params))
|
||||
: undefined;
|
||||
|
||||
return new Proxy(model, {
|
||||
|
||||
@@ -49,6 +49,22 @@ export async function compress(
|
||||
// 2. Input format: declared by the caller, else detected
|
||||
const inputFormat = format ?? detectFormat(processedMessages);
|
||||
|
||||
// Messages that aren't objects can't be converted to OpenAI without losing
|
||||
// content, so leave the request alone rather than silently dropping it. The
|
||||
// proxy is not called, so no compression event is emitted.
|
||||
if (processedMessages.some((m) => !m || typeof m !== "object")) {
|
||||
return {
|
||||
messages: processedMessages,
|
||||
tokensBefore: 0,
|
||||
tokensAfter: 0,
|
||||
tokensSaved: 0,
|
||||
compressionRatio: 1.0,
|
||||
transformsApplied: [],
|
||||
ccrHashes: [],
|
||||
compressed: false,
|
||||
};
|
||||
}
|
||||
|
||||
// 3. Shape the messages for the proxy. Anthropic messages go as-is: /v1/compress
|
||||
// compresses Anthropic content blocks natively, and converting them would drop
|
||||
// the thinking blocks Anthropic requires back unchanged, along with cache_control
|
||||
|
||||
@@ -39,6 +39,9 @@ export type MessageFormat = "openai" | "anthropic" | "vercel" | "gemini";
|
||||
export function detectFormat(messages: any[]): MessageFormat {
|
||||
let providerOptions = false;
|
||||
for (const msg of messages) {
|
||||
// Skip anything that isn't a message object (e.g. a bare string).
|
||||
if (!msg || typeof msg !== "object") continue;
|
||||
|
||||
// Gemini: uses `parts` instead of `content`
|
||||
if ("parts" in msg && !("content" in msg)) return "gemini";
|
||||
|
||||
@@ -880,7 +883,8 @@ export function geminiToOpenAI(messages: any[]): OpenAIMessage[] {
|
||||
type: "function",
|
||||
function: {
|
||||
name: p.functionCall.name,
|
||||
arguments: JSON.stringify(p.functionCall.args),
|
||||
// `args` is optional in the Gemini API; OpenAI requires an object.
|
||||
arguments: JSON.stringify(p.functionCall.args ?? {}),
|
||||
},
|
||||
}));
|
||||
}
|
||||
@@ -933,9 +937,12 @@ export function openAIToGemini(messages: OpenAIMessage[]): any[] {
|
||||
}
|
||||
if (msg.tool_calls) {
|
||||
for (const tc of msg.tool_calls) {
|
||||
parts.push({
|
||||
functionCall: { name: tc.function.name, args: JSON.parse(tc.function.arguments) },
|
||||
});
|
||||
// `arguments` may be missing or malformed; a proxy round-trip drops
|
||||
// `undefined`. Keep whatever is there rather than discarding a tool
|
||||
// input, matching the Vercel conversion.
|
||||
let args: any;
|
||||
try { args = JSON.parse(tc.function.arguments ?? "{}"); } catch { args = tc.function.arguments ?? {}; }
|
||||
parts.push({ functionCall: { name: tc.function.name, args } });
|
||||
}
|
||||
}
|
||||
result.push({ role: "model", parts });
|
||||
|
||||
@@ -194,4 +194,50 @@ describe("Gemini withHeadroom", () => {
|
||||
|
||||
expect(mockModel.generateContent).toHaveBeenCalledWith({ contents });
|
||||
});
|
||||
|
||||
it("accepts a plain string and a part array as a single user turn", async () => {
|
||||
const mockModel = { generateContent: vi.fn().mockResolvedValue({}) };
|
||||
mockFetch.mockResolvedValue(mockCompressResponse([]));
|
||||
|
||||
const wrapped = withHeadroom(mockModel, { baseUrl: "http://test:8787" });
|
||||
|
||||
await wrapped.generateContent("text");
|
||||
const firstBody = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(firstBody.messages).toEqual([{ role: "user", content: "text" }]);
|
||||
|
||||
// A top-level array is the parts of one turn, not several turns: media in
|
||||
// it must reach the proxy alongside the text.
|
||||
await wrapped.generateContent([
|
||||
"describe",
|
||||
{ inlineData: { mimeType: "image/png", data: "AAAA" } },
|
||||
]);
|
||||
const secondBody = JSON.parse(mockFetch.mock.calls[1][1].body);
|
||||
expect(secondBody.messages[0].content).toEqual([
|
||||
{ type: "text", text: "describe" },
|
||||
{ type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("rebuilds a request object when a string prompt is compressed", async () => {
|
||||
const mockModel = { generateContent: vi.fn().mockResolvedValue({}) };
|
||||
mockFetch.mockResolvedValue(mockCompressResponse([]));
|
||||
|
||||
const wrapped = withHeadroom(mockModel, { baseUrl: "http://test:8787" });
|
||||
await wrapped.generateContent("text");
|
||||
|
||||
// The SDK would read a bare Contents array as the parts of one turn.
|
||||
expect(mockModel.generateContent).toHaveBeenCalledWith({
|
||||
contents: [{ role: "user", parts: [{ text: "compressed content" }] }],
|
||||
});
|
||||
});
|
||||
|
||||
it("passes string params through when nothing was compressed", async () => {
|
||||
const mockModel = { generateContent: vi.fn().mockResolvedValue({}) };
|
||||
mockFetch.mockRejectedValue(new Error("connect ECONNREFUSED"));
|
||||
|
||||
const wrapped = withHeadroom(mockModel, { baseUrl: "http://test:8787" });
|
||||
await wrapped.generateContent("text");
|
||||
|
||||
expect(mockModel.generateContent).toHaveBeenCalledWith("text");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -117,6 +117,37 @@ describe("compress()", () => {
|
||||
expect(body.messages).toHaveLength(5);
|
||||
});
|
||||
|
||||
it("round-trips a Gemini functionCall with no args", async () => {
|
||||
const client = new HeadroomClient({ baseUrl: "http://127.0.0.1:9", fallback: true });
|
||||
const messages = [
|
||||
{ role: "user", parts: [{ text: "call f" }] },
|
||||
{ role: "model", parts: [{ functionCall: { name: "f" } }] },
|
||||
{ role: "user", parts: [{ functionResponse: { name: "f", response: { ok: true } } }] },
|
||||
];
|
||||
|
||||
const result = await compress(messages, { client });
|
||||
|
||||
expect(result.compressed).toBe(false);
|
||||
// The proxy sent an empty object for the missing args...
|
||||
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(body.messages[1].tool_calls[0].function.arguments).toBe("{}");
|
||||
// ...and it survives conversion back to Gemini.
|
||||
expect(result.messages[1]).toEqual({
|
||||
role: "model",
|
||||
parts: [{ functionCall: { name: "f", args: {} } }],
|
||||
});
|
||||
});
|
||||
|
||||
it("leaves a non-object entry alone instead of dropping it", async () => {
|
||||
const result = await compress(["Keep this instruction", { role: "user", content: "hi" }], {
|
||||
model: "gpt-4o",
|
||||
});
|
||||
|
||||
expect(result.compressed).toBe(false);
|
||||
expect(result.messages).toEqual(["Keep this instruction", { role: "user", content: "hi" }]);
|
||||
expect(mockFetch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("forwards a per-call config, overriding a provided client's config", async () => {
|
||||
mockFetch.mockResolvedValueOnce(okResponse());
|
||||
const client = new HeadroomClient({
|
||||
|
||||
Reference in New Issue
Block a user