feat(sdk): support per-call config override in compress() (#3370)

## Description

The TypeScript SDK's `simulate()` accepts a per-call `config` (spread
into the `/v1/compress` request body), but `compress()` did not —
`HeadroomClient.compress()` only used the `config` set once at
construction. There was no way to tune compression per request (e.g. a
different `mode` or `targetRatio`) without building a whole new client,
even though the proxy accepts per-request `config` on that same endpoint
(proven by `simulate()`).

This threads an optional per-call `config` through both
`HeadroomClient.compress()` and the standalone `compress()`, merged over
the client-level config — closing the `simulate()`/`compress()`
inconsistency.

**Config type (addresses review):** the per-call and client-level
`config` are typed as a new `CompressRequestConfig`, not
`HeadroomConfig`. `HeadroomConfig` (with `defaultMode`,
`contentRouterEnabled`, `smartCrusher`, …) is the `wrap`/proxy runtime
config and is **not** the schema `POST /v1/compress` reads.
`handle_compress` in `headroom/proxy/handlers/openai.py` honors only a
flat set of keys — `mode`, `target_ratio`, `compress_user_messages`,
`protect_recent`, `protect_analysis_context`, `frozen_message_count`,
`session_id` — and rejects (400) any `mode` outside `ccr | lossy_inline
| lossless_then_lossy`. `CompressRequestConfig` (camelCase, snake_cased
on the wire) models exactly those keys, so the SDK now offers only knobs
the endpoint actually accepts.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`sdk/typescript/src/types/config.ts`** — add `CompressMode` (`"ccr"
| "lossy_inline" | "lossless_then_lossy"`) and `CompressRequestConfig`
(the flat, all-optional set of keys `/v1/compress` honors: `mode`,
`targetRatio`, `compressUserMessages`, `protectRecent`,
`protectAnalysisContext`, `frozenMessageCount`, `sessionId`).
- **`sdk/typescript/src/index.ts`** — export `CompressMode` and
`CompressRequestConfig`.
- **`sdk/typescript/src/types.ts`** — `config?: CompressRequestConfig`
on `CompressOptions`; widen `HeadroomClientInterface.compress`'s options
to `{ model?; tokenBudget?; config? }`.
- **`sdk/typescript/src/client.ts`** — client-level and per-call
`config` retyped from `HeadroomConfig` to `CompressRequestConfig`;
`compress()` accepts `config` and forwards it to `_doCompress()`;
`_doCompress()` gains a `config?` param and builds the body config as
`const merged = { ...this.config, ...config }; if
(Object.keys(merged).length > 0) body.config = deepSnakeCase(merged);`
(replacing the old `if (this.config)` block).
- **`sdk/typescript/src/compress.ts`** — destructure `config` from
options and forward it to `client.compress(...)`. (Previously `config`
fell into `...clientOptions` and was silently dropped when a pre-built
`client` was supplied — this also fixes that latent drop.)

**Merge semantics:** shallow, per-call keys override client-level keys
at the top level — matching `simulate()`'s existing shallow spread. When
neither client nor call sets `config`, no `config` key is sent
(byte-identical to prior behavior — no regression). `simulate()` is
untouched. Additive, optional, non-breaking; every
`CompressRequestConfig` field is optional so a partial `{ mode: "ccr" }`
is valid.

## Testing

- [x] Unit tests pass (`npm test` — vitest)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality

### Test Output

```text
$ cd sdk/typescript && npm test
Test Files  18 passed | 5 skipped (23)
Tests  298 passed | 33 skipped (331)

$ npm run typecheck   # tsc --noEmit
(clean, no errors)

$ npm run build       # tsup
(ESM + CJS + DTS built successfully)
```

New tests (`test/client.test.ts`, `test/compress.test.ts`):
- per-call `config` is sent snake_cased in the request body (`mode`,
`target_ratio`, `compress_user_messages`);
- per-call `config` overrides client-level `config` (client `mode:
"lossy_inline"` + call `mode: "ccr"` → sends `"ccr"`);
- no `config` key when neither client nor call sets one (no regression);
- standalone `compress()` forwards a per-call `config` (`mode: "ccr"`,
`targetRatio: 0.25`) that overrides a **provided** client's config
(fails pre-fix — `config` was dropped when a `client` was supplied).

## Real Behavior Proof

- Environment: Node v22.16.0, Windows 11; TypeScript SDK built from this
branch (`cd sdk/typescript && npm install`).
- Exact command / steps: `cd sdk/typescript && npm test` (vitest), `npm
run typecheck` (tsc --noEmit), `npm run build` (tsup). The four new
tests drive the real `compress()` / `HeadroomClient.compress()` /
standalone `compress()` paths against a mocked `fetch` and assert the
actual `/v1/compress` request body.
- Observed result: On `main` (implementation reverted, tests kept) the
config tests FAIL — `body.config` is `undefined` because the per-call
config is dropped (`TypeError: Cannot read properties of undefined
(reading 'mode')`, and `expected undefined to be 'ccr'`). After the
change: `298 passed | 33 skipped`, typecheck clean, build clean, and
`body.config.mode` reflects the per-call override (`"ccr"` overriding a
client-level `"lossy_inline"`), with `targetRatio` sent as
`target_ratio`.
- Not tested: a live proxy round-trip. The request-body assertions (via
mocked fetch) cover the SDK's responsibility; the proxy already accepts
per-request `config` on `/v1/compress`, as the existing `simulate()`
path demonstrates.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This is a client-side TypeScript SDK
API change; no server/runtime feature flag or rollout channel is
involved.
- Minimum rollout channel: N/A (not gated by any rollout channel).
- Stable/default behavior changed: No. The new `config` parameter is
optional; when it (and the client-level config) is unset, the request
body is byte-identical to before (no `config` key sent). Existing
callers are unaffected.
- Kill switch / disable path: N/A — the behavior is opt-in per call
(omit `config` to get prior behavior).
- Unsafe override required: No.
- Qualification impact: None — additive optional parameter; existing SDK
tests unchanged and green.
- Rollback path: Revert this PR; no persisted state, migrations, or
config to unwind.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- CHANGELOG left untouched — the repo enforces a `no-manual-changelog`
CI check (release-please owns `CHANGELOG.md`); the conventional-commit
`feat(sdk):` message drives the entry.
- Scope kept minimal: no deep/recursive merge (shallow top-level
override matches `simulate()`; a deep-merge could be a future
enhancement), no proxy change, no new dependency.
- Pre-existing, out of scope (happy to file/fix separately if wanted):
`simulate()` calls `compressRaw()` directly and so never merges
client-level `config` (only per-call) — a small asymmetry this PR does
not touch; and `test/types.test.ts`'s
`HeadroomClientInterface["compress"]` param type-assertion was already
stale (it predates this change, missing `tokenBudget`) — it's a dormant
`expectTypeOf` check not enforced by `npm test`/`tsc`.
This commit is contained in:
Shubham Srivastava
2026-09-24 17:06:01 -05:00
committed by GitHub
parent 63676e6cff
commit c3e8a1ee63
7 changed files with 105 additions and 9 deletions
+9 -7
View File
@@ -19,7 +19,7 @@ import type {
import { mapProxyError, HeadroomConnectionError, HeadroomAuthError, HeadroomCompressError } from "./errors.js";
import { deepCamelCase, deepSnakeCase } from "./utils/case.js";
import { parseSSE } from "./utils/stream.js";
import type { HeadroomConfig, HeadroomMode } from "./types/config.js";
import type { CompressRequestConfig, HeadroomMode } from "./types/config.js";
import type {
SimulationResult,
RequestMetrics,
@@ -191,7 +191,7 @@ class Messages {
export interface ExtendedClientOptions extends HeadroomClientOptions {
providerApiKey?: string;
defaultMode?: HeadroomMode;
config?: HeadroomConfig;
config?: CompressRequestConfig;
}
export class HeadroomClient implements HeadroomClientInterface {
@@ -200,7 +200,7 @@ export class HeadroomClient implements HeadroomClientInterface {
private timeout: number;
private fallback: boolean;
private retries: number;
private config: HeadroomConfig | undefined;
private config: CompressRequestConfig | undefined;
private stack: string | undefined;
/** @internal */ providerApiKey: string | undefined;
@@ -234,7 +234,7 @@ export class HeadroomClient implements HeadroomClientInterface {
async compress(
messages: OpenAIMessage[],
options: { model?: string; tokenBudget?: number } = {},
options: { model?: string; tokenBudget?: number; config?: CompressRequestConfig } = {},
): Promise<CompressResult> {
const model = options.model ?? "gpt-4o";
@@ -243,7 +243,7 @@ export class HeadroomClient implements HeadroomClientInterface {
for (let attempt = 0; attempt < maxAttempts; attempt++) {
try {
return await this._doCompress(messages, model, options.tokenBudget);
return await this._doCompress(messages, model, options.tokenBudget, options.config);
} catch (error) {
lastError = error;
if (error instanceof HeadroomAuthError) throw error;
@@ -569,13 +569,15 @@ export class HeadroomClient implements HeadroomClientInterface {
messages: OpenAIMessage[],
model: string,
tokenBudget?: number,
config?: CompressRequestConfig,
): Promise<CompressResult> {
const body: Record<string, unknown> = { messages, model };
if (tokenBudget) {
body.token_budget = tokenBudget;
}
if (this.config) {
body.config = deepSnakeCase(this.config);
const mergedConfig = { ...this.config, ...config };
if (Object.keys(mergedConfig).length > 0) {
body.config = deepSnakeCase(mergedConfig);
}
const response = await this._fetch("/v1/compress", {
+2 -1
View File
@@ -25,6 +25,7 @@ export async function compress(
model,
tokenBudget,
hooks,
config,
...clientOptions
} = options;
@@ -57,7 +58,7 @@ export async function compress(
// 5. Compress via proxy
const client = providedClient ?? new HeadroomClient(clientOptions);
const result = await client.compress(openaiMessages, { model, tokenBudget });
const result = await client.compress(openaiMessages, { model, tokenBudget, config });
// 6. Convert compressed messages back to original format
const outputMessages = fromOpenAI(result.messages, inputFormat);
+2
View File
@@ -51,6 +51,8 @@ export {
// --- Config types ---
export type {
HeadroomMode,
CompressMode,
CompressRequestConfig,
RelevanceTier,
ContentType,
BlockKind,
+5 -1
View File
@@ -4,6 +4,7 @@
*/
import type { CompressionHooks } from "./hooks.js";
import type { CompressRequestConfig } from "./types/config.js";
// --- Message types (OpenAI chat format) ---
@@ -69,6 +70,9 @@ export interface CompressOptions {
hooks?: CompressionHooks;
/** Integration slug sent as X-Headroom-Stack (e.g. "adapter_ts_openai"). */
stack?: string;
/** Per-call /v1/compress config, merged over the client-level config
* (per-call keys override at the top level). */
config?: CompressRequestConfig;
}
export interface CompressResult {
@@ -103,7 +107,7 @@ export interface HeadroomClientOptions {
export interface HeadroomClientInterface {
compress(
messages: OpenAIMessage[],
options?: { model?: string; tokenBudget?: number },
options?: { model?: string; tokenBudget?: number; config?: CompressRequestConfig },
): Promise<CompressResult>;
}
+24
View File
@@ -7,6 +7,30 @@
export type HeadroomMode = "audit" | "optimize" | "simulate";
/** Pipeline mode accepted by POST /v1/compress. Unset = default marker-free
* pipeline. Any other value is rejected (400) by the proxy. */
export type CompressMode = "ccr" | "lossy_inline" | "lossless_then_lossy";
/** Config accepted by the POST /v1/compress endpoint. camelCase keys are
* snake_cased on the wire (e.g. targetRatio -> target_ratio). Every field is
* optional; only these keys are honored by the endpoint. */
export interface CompressRequestConfig {
/** Compression pipeline mode (unset = default marker-free pipeline). */
mode?: CompressMode;
/** Target keep-ratio (0–1) for lossy compression. -> target_ratio */
targetRatio?: number;
/** Also compress user-role messages (default false). -> compress_user_messages */
compressUserMessages?: boolean;
/** Number of most-recent messages to protect from compression. -> protect_recent */
protectRecent?: number;
/** Protect analysis/reasoning context from compression. -> protect_analysis_context */
protectAnalysisContext?: boolean;
/** Freeze the first N messages (prefix) from compression. -> frozen_message_count */
frozenMessageCount?: number;
/** Session id for session-aware compression. -> session_id */
sessionId?: string;
}
export type RelevanceTier = "bm25" | "embedding" | "hybrid";
export type ContentType =
+44
View File
@@ -263,4 +263,48 @@ describe("HeadroomClient", () => {
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.model).toBe("gpt-4o");
});
it("sends a per-call config (snake_cased) in the request body", async () => {
mockFetch.mockResolvedValueOnce(okResponse(sampleProxyResponse));
const client = new HeadroomClient({ baseUrl: "http://localhost:8787" });
await client.compress(sampleMessages, {
model: "gpt-4o",
config: { mode: "ccr", targetRatio: 0.3, compressUserMessages: true },
});
const [, opts] = mockFetch.mock.calls[0];
const body = JSON.parse(opts.body);
expect(body.config.mode).toBe("ccr");
expect(body.config.target_ratio).toBe(0.3);
expect(body.config.compress_user_messages).toBe(true);
});
it("per-call config overrides client-level config", async () => {
mockFetch.mockResolvedValueOnce(okResponse(sampleProxyResponse));
const client = new HeadroomClient({
baseUrl: "http://localhost:8787",
config: { mode: "lossy_inline" },
});
await client.compress(sampleMessages, {
model: "gpt-4o",
config: { mode: "ccr" },
});
const [, opts] = mockFetch.mock.calls[0];
const body = JSON.parse(opts.body);
expect(body.config.mode).toBe("ccr");
});
it("omits the config key when neither client nor call sets config", async () => {
mockFetch.mockResolvedValueOnce(okResponse(sampleProxyResponse));
const client = new HeadroomClient({ baseUrl: "http://localhost:8787" });
await client.compress(sampleMessages, { model: "gpt-4o" });
const [, opts] = mockFetch.mock.calls[0];
const body = JSON.parse(opts.body);
expect(body.config).toBeUndefined();
});
});
+19
View File
@@ -1,5 +1,6 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { compress } from "../src/compress.js";
import { HeadroomClient } from "../src/client.js";
import type { OpenAIMessage } from "../src/types.js";
const mockFetch = vi.fn();
@@ -115,4 +116,22 @@ describe("compress()", () => {
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
expect(body.messages).toHaveLength(5);
});
it("forwards a per-call config, overriding a provided client's config", async () => {
mockFetch.mockResolvedValueOnce(okResponse());
const client = new HeadroomClient({
baseUrl: "http://localhost:8787",
config: { mode: "lossy_inline" },
});
await compress([{ role: "user", content: "hello" }], {
client,
config: { mode: "ccr", targetRatio: 0.25 },
});
const [, opts] = mockFetch.mock.calls[0];
const body = JSON.parse(opts.body);
expect(body.config.mode).toBe("ccr");
expect(body.config.target_ratio).toBe(0.25);
});
});