mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat(sdk): support per-call config override in compress() (#3370)
## Description
The TypeScript SDK's `simulate()` accepts a per-call `config` (spread
into the `/v1/compress` request body), but `compress()` did not —
`HeadroomClient.compress()` only used the `config` set once at
construction. There was no way to tune compression per request (e.g. a
different `mode` or `targetRatio`) without building a whole new client,
even though the proxy accepts per-request `config` on that same endpoint
(proven by `simulate()`).
This threads an optional per-call `config` through both
`HeadroomClient.compress()` and the standalone `compress()`, merged over
the client-level config — closing the `simulate()`/`compress()`
inconsistency.
**Config type (addresses review):** the per-call and client-level
`config` are typed as a new `CompressRequestConfig`, not
`HeadroomConfig`. `HeadroomConfig` (with `defaultMode`,
`contentRouterEnabled`, `smartCrusher`, …) is the `wrap`/proxy runtime
config and is **not** the schema `POST /v1/compress` reads.
`handle_compress` in `headroom/proxy/handlers/openai.py` honors only a
flat set of keys — `mode`, `target_ratio`, `compress_user_messages`,
`protect_recent`, `protect_analysis_context`, `frozen_message_count`,
`session_id` — and rejects (400) any `mode` outside `ccr | lossy_inline
| lossless_then_lossy`. `CompressRequestConfig` (camelCase, snake_cased
on the wire) models exactly those keys, so the SDK now offers only knobs
the endpoint actually accepts.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`sdk/typescript/src/types/config.ts`** — add `CompressMode` (`"ccr"
| "lossy_inline" | "lossless_then_lossy"`) and `CompressRequestConfig`
(the flat, all-optional set of keys `/v1/compress` honors: `mode`,
`targetRatio`, `compressUserMessages`, `protectRecent`,
`protectAnalysisContext`, `frozenMessageCount`, `sessionId`).
- **`sdk/typescript/src/index.ts`** — export `CompressMode` and
`CompressRequestConfig`.
- **`sdk/typescript/src/types.ts`** — `config?: CompressRequestConfig`
on `CompressOptions`; widen `HeadroomClientInterface.compress`'s options
to `{ model?; tokenBudget?; config? }`.
- **`sdk/typescript/src/client.ts`** — client-level and per-call
`config` retyped from `HeadroomConfig` to `CompressRequestConfig`;
`compress()` accepts `config` and forwards it to `_doCompress()`;
`_doCompress()` gains a `config?` param and builds the body config as
`const merged = { ...this.config, ...config }; if
(Object.keys(merged).length > 0) body.config = deepSnakeCase(merged);`
(replacing the old `if (this.config)` block).
- **`sdk/typescript/src/compress.ts`** — destructure `config` from
options and forward it to `client.compress(...)`. (Previously `config`
fell into `...clientOptions` and was silently dropped when a pre-built
`client` was supplied — this also fixes that latent drop.)
**Merge semantics:** shallow, per-call keys override client-level keys
at the top level — matching `simulate()`'s existing shallow spread. When
neither client nor call sets `config`, no `config` key is sent
(byte-identical to prior behavior — no regression). `simulate()` is
untouched. Additive, optional, non-breaking; every
`CompressRequestConfig` field is optional so a partial `{ mode: "ccr" }`
is valid.
## Testing
- [x] Unit tests pass (`npm test` — vitest)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
### Test Output
```text
$ cd sdk/typescript && npm test
Test Files 18 passed | 5 skipped (23)
Tests 298 passed | 33 skipped (331)
$ npm run typecheck # tsc --noEmit
(clean, no errors)
$ npm run build # tsup
(ESM + CJS + DTS built successfully)
```
New tests (`test/client.test.ts`, `test/compress.test.ts`):
- per-call `config` is sent snake_cased in the request body (`mode`,
`target_ratio`, `compress_user_messages`);
- per-call `config` overrides client-level `config` (client `mode:
"lossy_inline"` + call `mode: "ccr"` → sends `"ccr"`);
- no `config` key when neither client nor call sets one (no regression);
- standalone `compress()` forwards a per-call `config` (`mode: "ccr"`,
`targetRatio: 0.25`) that overrides a **provided** client's config
(fails pre-fix — `config` was dropped when a `client` was supplied).
## Real Behavior Proof
- Environment: Node v22.16.0, Windows 11; TypeScript SDK built from this
branch (`cd sdk/typescript && npm install`).
- Exact command / steps: `cd sdk/typescript && npm test` (vitest), `npm
run typecheck` (tsc --noEmit), `npm run build` (tsup). The four new
tests drive the real `compress()` / `HeadroomClient.compress()` /
standalone `compress()` paths against a mocked `fetch` and assert the
actual `/v1/compress` request body.
- Observed result: On `main` (implementation reverted, tests kept) the
config tests FAIL — `body.config` is `undefined` because the per-call
config is dropped (`TypeError: Cannot read properties of undefined
(reading 'mode')`, and `expected undefined to be 'ccr'`). After the
change: `298 passed | 33 skipped`, typecheck clean, build clean, and
`body.config.mode` reflects the per-call override (`"ccr"` overriding a
client-level `"lossy_inline"`), with `targetRatio` sent as
`target_ratio`.
- Not tested: a live proxy round-trip. The request-body assertions (via
mocked fetch) cover the SDK's responsibility; the proxy already accepts
per-request `config` on `/v1/compress`, as the existing `simulate()`
path demonstrates.
## Runtime Rollout Safety
- Rollout-managed feature(s): None. This is a client-side TypeScript SDK
API change; no server/runtime feature flag or rollout channel is
involved.
- Minimum rollout channel: N/A (not gated by any rollout channel).
- Stable/default behavior changed: No. The new `config` parameter is
optional; when it (and the client-level config) is unset, the request
body is byte-identical to before (no `config` key sent). Existing
callers are unaffected.
- Kill switch / disable path: N/A — the behavior is opt-in per call
(omit `config` to get prior behavior).
- Unsafe override required: No.
- Qualification impact: None — additive optional parameter; existing SDK
tests unchanged and green.
- Rollback path: Revert this PR; no persisted state, migrations, or
config to unwind.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
- CHANGELOG left untouched — the repo enforces a `no-manual-changelog`
CI check (release-please owns `CHANGELOG.md`); the conventional-commit
`feat(sdk):` message drives the entry.
- Scope kept minimal: no deep/recursive merge (shallow top-level
override matches `simulate()`; a deep-merge could be a future
enhancement), no proxy change, no new dependency.
- Pre-existing, out of scope (happy to file/fix separately if wanted):
`simulate()` calls `compressRaw()` directly and so never merges
client-level `config` (only per-call) — a small asymmetry this PR does
not touch; and `test/types.test.ts`'s
`HeadroomClientInterface["compress"]` param type-assertion was already
stale (it predates this change, missing `tokenBudget`) — it's a dormant
`expectTypeOf` check not enforced by `npm test`/`tsc`.
This commit is contained in:
@@ -19,7 +19,7 @@ import type {
|
||||
import { mapProxyError, HeadroomConnectionError, HeadroomAuthError, HeadroomCompressError } from "./errors.js";
|
||||
import { deepCamelCase, deepSnakeCase } from "./utils/case.js";
|
||||
import { parseSSE } from "./utils/stream.js";
|
||||
import type { HeadroomConfig, HeadroomMode } from "./types/config.js";
|
||||
import type { CompressRequestConfig, HeadroomMode } from "./types/config.js";
|
||||
import type {
|
||||
SimulationResult,
|
||||
RequestMetrics,
|
||||
@@ -191,7 +191,7 @@ class Messages {
|
||||
export interface ExtendedClientOptions extends HeadroomClientOptions {
|
||||
providerApiKey?: string;
|
||||
defaultMode?: HeadroomMode;
|
||||
config?: HeadroomConfig;
|
||||
config?: CompressRequestConfig;
|
||||
}
|
||||
|
||||
export class HeadroomClient implements HeadroomClientInterface {
|
||||
@@ -200,7 +200,7 @@ export class HeadroomClient implements HeadroomClientInterface {
|
||||
private timeout: number;
|
||||
private fallback: boolean;
|
||||
private retries: number;
|
||||
private config: HeadroomConfig | undefined;
|
||||
private config: CompressRequestConfig | undefined;
|
||||
private stack: string | undefined;
|
||||
|
||||
/** @internal */ providerApiKey: string | undefined;
|
||||
@@ -234,7 +234,7 @@ export class HeadroomClient implements HeadroomClientInterface {
|
||||
|
||||
async compress(
|
||||
messages: OpenAIMessage[],
|
||||
options: { model?: string; tokenBudget?: number } = {},
|
||||
options: { model?: string; tokenBudget?: number; config?: CompressRequestConfig } = {},
|
||||
): Promise<CompressResult> {
|
||||
const model = options.model ?? "gpt-4o";
|
||||
|
||||
@@ -243,7 +243,7 @@ export class HeadroomClient implements HeadroomClientInterface {
|
||||
|
||||
for (let attempt = 0; attempt < maxAttempts; attempt++) {
|
||||
try {
|
||||
return await this._doCompress(messages, model, options.tokenBudget);
|
||||
return await this._doCompress(messages, model, options.tokenBudget, options.config);
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
if (error instanceof HeadroomAuthError) throw error;
|
||||
@@ -569,13 +569,15 @@ export class HeadroomClient implements HeadroomClientInterface {
|
||||
messages: OpenAIMessage[],
|
||||
model: string,
|
||||
tokenBudget?: number,
|
||||
config?: CompressRequestConfig,
|
||||
): Promise<CompressResult> {
|
||||
const body: Record<string, unknown> = { messages, model };
|
||||
if (tokenBudget) {
|
||||
body.token_budget = tokenBudget;
|
||||
}
|
||||
if (this.config) {
|
||||
body.config = deepSnakeCase(this.config);
|
||||
const mergedConfig = { ...this.config, ...config };
|
||||
if (Object.keys(mergedConfig).length > 0) {
|
||||
body.config = deepSnakeCase(mergedConfig);
|
||||
}
|
||||
|
||||
const response = await this._fetch("/v1/compress", {
|
||||
|
||||
@@ -25,6 +25,7 @@ export async function compress(
|
||||
model,
|
||||
tokenBudget,
|
||||
hooks,
|
||||
config,
|
||||
...clientOptions
|
||||
} = options;
|
||||
|
||||
@@ -57,7 +58,7 @@ export async function compress(
|
||||
|
||||
// 5. Compress via proxy
|
||||
const client = providedClient ?? new HeadroomClient(clientOptions);
|
||||
const result = await client.compress(openaiMessages, { model, tokenBudget });
|
||||
const result = await client.compress(openaiMessages, { model, tokenBudget, config });
|
||||
|
||||
// 6. Convert compressed messages back to original format
|
||||
const outputMessages = fromOpenAI(result.messages, inputFormat);
|
||||
|
||||
@@ -51,6 +51,8 @@ export {
|
||||
// --- Config types ---
|
||||
export type {
|
||||
HeadroomMode,
|
||||
CompressMode,
|
||||
CompressRequestConfig,
|
||||
RelevanceTier,
|
||||
ContentType,
|
||||
BlockKind,
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
*/
|
||||
|
||||
import type { CompressionHooks } from "./hooks.js";
|
||||
import type { CompressRequestConfig } from "./types/config.js";
|
||||
|
||||
// --- Message types (OpenAI chat format) ---
|
||||
|
||||
@@ -69,6 +70,9 @@ export interface CompressOptions {
|
||||
hooks?: CompressionHooks;
|
||||
/** Integration slug sent as X-Headroom-Stack (e.g. "adapter_ts_openai"). */
|
||||
stack?: string;
|
||||
/** Per-call /v1/compress config, merged over the client-level config
|
||||
* (per-call keys override at the top level). */
|
||||
config?: CompressRequestConfig;
|
||||
}
|
||||
|
||||
export interface CompressResult {
|
||||
@@ -103,7 +107,7 @@ export interface HeadroomClientOptions {
|
||||
export interface HeadroomClientInterface {
|
||||
compress(
|
||||
messages: OpenAIMessage[],
|
||||
options?: { model?: string; tokenBudget?: number },
|
||||
options?: { model?: string; tokenBudget?: number; config?: CompressRequestConfig },
|
||||
): Promise<CompressResult>;
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,30 @@
|
||||
|
||||
export type HeadroomMode = "audit" | "optimize" | "simulate";
|
||||
|
||||
/** Pipeline mode accepted by POST /v1/compress. Unset = default marker-free
|
||||
* pipeline. Any other value is rejected (400) by the proxy. */
|
||||
export type CompressMode = "ccr" | "lossy_inline" | "lossless_then_lossy";
|
||||
|
||||
/** Config accepted by the POST /v1/compress endpoint. camelCase keys are
|
||||
* snake_cased on the wire (e.g. targetRatio -> target_ratio). Every field is
|
||||
* optional; only these keys are honored by the endpoint. */
|
||||
export interface CompressRequestConfig {
|
||||
/** Compression pipeline mode (unset = default marker-free pipeline). */
|
||||
mode?: CompressMode;
|
||||
/** Target keep-ratio (0–1) for lossy compression. -> target_ratio */
|
||||
targetRatio?: number;
|
||||
/** Also compress user-role messages (default false). -> compress_user_messages */
|
||||
compressUserMessages?: boolean;
|
||||
/** Number of most-recent messages to protect from compression. -> protect_recent */
|
||||
protectRecent?: number;
|
||||
/** Protect analysis/reasoning context from compression. -> protect_analysis_context */
|
||||
protectAnalysisContext?: boolean;
|
||||
/** Freeze the first N messages (prefix) from compression. -> frozen_message_count */
|
||||
frozenMessageCount?: number;
|
||||
/** Session id for session-aware compression. -> session_id */
|
||||
sessionId?: string;
|
||||
}
|
||||
|
||||
export type RelevanceTier = "bm25" | "embedding" | "hybrid";
|
||||
|
||||
export type ContentType =
|
||||
|
||||
@@ -263,4 +263,48 @@ describe("HeadroomClient", () => {
|
||||
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(body.model).toBe("gpt-4o");
|
||||
});
|
||||
|
||||
it("sends a per-call config (snake_cased) in the request body", async () => {
|
||||
mockFetch.mockResolvedValueOnce(okResponse(sampleProxyResponse));
|
||||
const client = new HeadroomClient({ baseUrl: "http://localhost:8787" });
|
||||
|
||||
await client.compress(sampleMessages, {
|
||||
model: "gpt-4o",
|
||||
config: { mode: "ccr", targetRatio: 0.3, compressUserMessages: true },
|
||||
});
|
||||
|
||||
const [, opts] = mockFetch.mock.calls[0];
|
||||
const body = JSON.parse(opts.body);
|
||||
expect(body.config.mode).toBe("ccr");
|
||||
expect(body.config.target_ratio).toBe(0.3);
|
||||
expect(body.config.compress_user_messages).toBe(true);
|
||||
});
|
||||
|
||||
it("per-call config overrides client-level config", async () => {
|
||||
mockFetch.mockResolvedValueOnce(okResponse(sampleProxyResponse));
|
||||
const client = new HeadroomClient({
|
||||
baseUrl: "http://localhost:8787",
|
||||
config: { mode: "lossy_inline" },
|
||||
});
|
||||
|
||||
await client.compress(sampleMessages, {
|
||||
model: "gpt-4o",
|
||||
config: { mode: "ccr" },
|
||||
});
|
||||
|
||||
const [, opts] = mockFetch.mock.calls[0];
|
||||
const body = JSON.parse(opts.body);
|
||||
expect(body.config.mode).toBe("ccr");
|
||||
});
|
||||
|
||||
it("omits the config key when neither client nor call sets config", async () => {
|
||||
mockFetch.mockResolvedValueOnce(okResponse(sampleProxyResponse));
|
||||
const client = new HeadroomClient({ baseUrl: "http://localhost:8787" });
|
||||
|
||||
await client.compress(sampleMessages, { model: "gpt-4o" });
|
||||
|
||||
const [, opts] = mockFetch.mock.calls[0];
|
||||
const body = JSON.parse(opts.body);
|
||||
expect(body.config).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import { compress } from "../src/compress.js";
|
||||
import { HeadroomClient } from "../src/client.js";
|
||||
import type { OpenAIMessage } from "../src/types.js";
|
||||
|
||||
const mockFetch = vi.fn();
|
||||
@@ -115,4 +116,22 @@ describe("compress()", () => {
|
||||
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(body.messages).toHaveLength(5);
|
||||
});
|
||||
|
||||
it("forwards a per-call config, overriding a provided client's config", async () => {
|
||||
mockFetch.mockResolvedValueOnce(okResponse());
|
||||
const client = new HeadroomClient({
|
||||
baseUrl: "http://localhost:8787",
|
||||
config: { mode: "lossy_inline" },
|
||||
});
|
||||
|
||||
await compress([{ role: "user", content: "hello" }], {
|
||||
client,
|
||||
config: { mode: "ccr", targetRatio: 0.25 },
|
||||
});
|
||||
|
||||
const [, opts] = mockFetch.mock.calls[0];
|
||||
const body = JSON.parse(opts.body);
|
||||
expect(body.config.mode).toBe("ccr");
|
||||
expect(body.config.target_ratio).toBe(0.25);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user