fix(proxy): F3 — per-tenant TOIN learning key (#404)

## Description

This change isolates TOIN learning by tenant in shared-proxy
deployments. Each pattern key gains a tenant namespace derived from a
configurable sanitized header, a stable auth/token hash, or the
backwards-compatible global namespace. Legacy keys migrate to global and
every resolution path emits a structured event.

## Threat model

Pre-F3, every request's tool-call patterns aggregated into one global
pool keyed by `(auth_mode, model_family, sig_hash)`. That works for an
OSS single-tenant deploy, but in a multi-tenant SaaS / shared-proxy
deploy:

1. **Cross-tenant pattern leakage.** Tenant A's compression patterns
silently train recommendations the proxy then applies to tenant B's
requests, degrading both tenants' compression quality.
2. **Structural fingerprint exposure.** TOIN dumps include
`tool_signature_hash` per slice. With one shared pool, a user with read
access to a dump file learns the structural fingerprint of every other
tenant's tool surface.

F3 partitions: every TOIN store key gets a `tenant_key` prefix derived
from the inbound request. Two distinct tenants therefore train two
isolated pattern pools.

## Tenant key derivation rules

| Source | When | Key derivation |

|------------|--------------------------------------------------------------|-------------------------------------------------------------------|
| `header` | `HEADROOM_TENANT_KEY_HEADER` (default
`X-Headroom-Tenant-ID`) is present and survives sanitization | Sanitized
header value (ASCII alphanum + `-_`, max 64 chars) |
| `hash` | No header but bearer token + `auth_mode` (from F1) present |
`sha256(f"{auth_mode}:{api_key_prefix}")[:24]` (api_key_prefix = first 8
chars of bearer); same SHA-256[:24] idiom as PR #395 |
| `global` | Neither header nor bearer present (OSS single-tenant / dev)
| Literal `"global"` namespace; pre-F3 patterns migrate here on first
load |

Header path takes precedence over hash. Header values that sanitize to
empty (pure unicode / control chars only) fall through to the next rule.

## Configuration

Single env var: `HEADROOM_TENANT_KEY_HEADER`. Default
`X-Headroom-Tenant-ID`. The header NAME is configurable; the
sanitization rules and the `"global"` literal are NOT — they're
load-bearing for namespace isolation.

## Structured log fields

Every resolution path emits exactly one `tenant_key_resolved` log event
(no silent fallbacks per `feedback_no_silent_fallbacks.md`):

| Field | Always present | Notes |

|--------------|----------------|------------------------------------------------|
| `event` | yes | Always `"tenant_key_resolved"` |
| `tenant_key` | yes | Resolved key (or `"global"`) |
| `source` | yes | One of `header` / `hash` / `global` |
| `header` | header path | Header name that matched |
| `auth_mode` | hash path | `payg` / `oauth` / `subscription` |

Operators can alert on a sustained `source="global"` rate to spot
mis-configured upstream auth proxies.

## Implementation

- **NEW** `headroom/proxy/tenant_key.py`: pure resolver. Public surface
= `resolve_tenant_key(request) -> tuple[str, str]`. Plus
`set_request_tenant_key()` / `get_current_tenant_key()` ContextVar pair
so TOIN's deep-stack `record_compression` calls observe the right tenant
slice.
- `headroom/telemetry/toin.py`: extends `PatternKey` from a 3-tuple to a
4-tuple `(tenant_key, auth_mode, model_family, sig_hash)`.
`_deserialize_pattern_key` is backward-compat with both pre-F3 formats
(3-tuple from PR-B5, bare `sig_hash` from v1.0). `record_compression` /
`record_retrieval` / `get_pattern` accept an optional `tenant_key`
kwarg; when absent they read from the ContextVar.
- `headroom/cli/toin_publish.py`: each `[[recommendation]]` row now
carries `tenant_key`; rows sort by `(tenant_key, auth_mode,
model_family, sig_hash)` for clean diffs.
- `headroom/proxy/handlers/anthropic.py` +
`headroom/proxy/handlers/openai.py`: call `resolve_tenant_key(request)`
once at request entry, right after the F1 `classify_auth_mode` block.
Three call sites total (1 Anthropic, 2 OpenAI). Result attached to
`request.state.tenant_key` / `request.state.tenant_key_source` AND set
on the ContextVar.

## Realignment build constraints (per project memory)

- **configurable**: `HEADROOM_TENANT_KEY_HEADER` env var.
- **no hardcodes**: header name + `"global"` literal are the only
constants; both motivated and documented in code.
- **no regexes**: sanitization uses string ops +
`unicodedata.category()`.
- **no silent fallbacks**: every resolution emits a structured log
event.
- **comprehensive tests**: header / hash / global paths + sanitization
edges + tenant-isolation property test.
- **structured logs**: `event=tenant_key_resolved` with `tenant_key`,
`source`, optional `header` / `auth_mode`.

## Test plan

- [x] `pytest tests/test_tenant_key.py` (29 cases — header / hash /
global paths, structured-log emission, sanitization edges, ContextVar
round-trip)
- [x] `pytest tests/test_toin_observation_only.py` (6 new isolation
cases — two-tenant divergence, ContextVar read, explicit kwarg
precedence, default migration to `"global"`)
- [x] `pytest tests/test_toin*.py` (existing TOIN tests updated for the
4-tuple key shape)
- [x] `pytest tests/test_proxy_tenant_key_integration.py` (4 cases —
handler wires `request.state.tenant_key` + ContextVar through all three
resolution paths)
- [x] `pytest tests/test_critical_fixes.py tests/test_critical_gaps.py
tests/test_proxy_ccr.py` (legacy 3-tuple references updated)
- [x] `make ci-precheck-python` — 176 tests pass (passed via pre-push
hook)
- [x] `make ci-precheck-commitlint` — clean (passed via pre-push hook)

Refs: F1 (PR #399, auth_mode classifier), F2.1 (PR #400, per-auth-mode
CompressionPolicy gates).

## Type of Change

- [x] Bug fix / tenant-isolation hardening
- [ ] New feature
- [ ] Breaking change
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring

## Changes Made

- Add a pure tenant-key resolver and request ContextVar.
- Extend TOIN pattern keys and published recommendations with a tenant
namespace while preserving legacy deserialization.
- Resolve and attach tenant identity at Anthropic and OpenAI request
entry.
- Add isolation, migration, sanitization, header, CCR, handler, and
executor coverage.

## Testing

- [x] Focused tenant/TOIN/proxy tests pass
- [x] Python CI precheck passes
- [x] Commitlint precheck passes

### Test Output

```text
176 focused/precheck tests passed
Current-main isolation, migration, header, CCR, and executor validation: green
```

## Real Behavior Proof

- Environment: current-main reconciled Python proxy/TOIN test
environment.
- Exact command / steps: ran tenant resolver, TOIN observation/publish,
handler integration, header-isolation, CCR, and compression-executor
suites.
- Observed result: distinct tenants train isolated pattern pools; legacy
patterns migrate to global; request handlers propagate the resolved
namespace through deep TOIN calls.
- Not tested: production multi-tenant traffic volume or an external TOIN
dump consumer.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-10-01 22:03:59 -05:00
committed by GitHub
co-authored by JD Davis JerrettDavis
parent c46e74d06b
commit f90a56b038
20 changed files with 1377 additions and 106 deletions
+15 -2
View File
@@ -98,6 +98,7 @@ def _toml_escape(s: str) -> str:
def _format_row(
*,
tenant_key: str,
auth_mode: str,
model_family: str,
structure_hash: str,
@@ -110,6 +111,7 @@ def _format_row(
skip_flag = "true" if skip_compression_recommended else "false"
return (
"[[recommendation]]\n"
f'tenant_key = "{_toml_escape(tenant_key)}"\n'
f'auth_mode = "{_toml_escape(auth_mode)}"\n'
f'model_family = "{_toml_escape(model_family)}"\n'
f'structure_hash = "{_toml_escape(structure_hash)}"\n'
@@ -137,7 +139,12 @@ def _eligible_rows(
observations = pattern.total_compressions
if observations < min_observations:
continue
auth_mode, model_family, sig_hash = key
# PR-F3: keys are now 4-tuples
# `(tenant_key, auth_mode, model_family, sig_hash)`. We carry
# `tenant_key` through to the published row so the loaded
# `recommendations.toml` is self-describing — operators can
# filter recommendations per tenant when reviewing them.
tenant_key, auth_mode, model_family, sig_hash = key
strategy_hint = (
"skip_compression"
if pattern.skip_compression_recommended
@@ -145,6 +152,7 @@ def _eligible_rows(
)
rows.append(
{
"tenant_key": tenant_key,
"auth_mode": auth_mode,
"model_family": model_family,
"structure_hash": sig_hash,
@@ -156,7 +164,12 @@ def _eligible_rows(
)
rows.sort(
key=lambda r: (r["auth_mode"], r["model_family"], r["structure_hash"]),
key=lambda r: (
r["tenant_key"],
r["auth_mode"],
r["model_family"],
r["structure_hash"],
),
)
return rows
+15
View File
@@ -66,6 +66,7 @@ from headroom.proxy.model_router import estimate_input_tokens
from headroom.proxy.nonstream_sse_policy import should_recover_sse_reply
from headroom.proxy.outcome import RequestOutcome
from headroom.proxy.output_shaper import shaper_enabled_for, steering_allowed_for
from headroom.proxy.tenant_key import resolve_tenant_key, set_request_tenant_key
from headroom.proxy.thinking_tokens import ThinkingTokens, extract_thinking_tokens
from headroom.utils import format_exception_message
@@ -933,6 +934,20 @@ class AnthropicHandlerMixin:
request.state.auth_mode = auth_mode
logger.debug(f"[{request_id}] auth_mode_classified mode={auth_mode.value}")
# Phase F PR-F3: resolve the per-tenant key for TOIN learning
# isolation. `set_request_tenant_key` populates the ContextVar
# the deep-stack `record_compression` / `record_retrieval` calls
# in SmartCrusher / ContentRouter read from. Pre-F3 every
# request's patterns aggregated into one global pool — F3
# partitions by header / hash / global namespace so two
# tenants can't cross-pollinate compression patterns. The
# resolver itself emits the structured `tenant_key_resolved`
# log on every call.
tenant_key, tenant_key_source = resolve_tenant_key(request)
request.state.tenant_key = tenant_key
request.state.tenant_key_source = tenant_key_source
set_request_tenant_key(tenant_key)
# Unit 2: per-stage timings for the pre-upstream phase. The
# finalizer emits one structured log line + Prometheus
# observations even if the handler raises.
+21
View File
@@ -103,6 +103,7 @@ from headroom.proxy.project_context import (
get_current_project,
set_current_project,
)
from headroom.proxy.tenant_key import resolve_tenant_key, set_request_tenant_key
from headroom.proxy.thinking_tokens import ThinkingTokens, extract_from_usage
from headroom.proxy.token_counting import gemini_output_tokens
@@ -3451,6 +3452,16 @@ class OpenAIHandlerMixin:
request.state.auth_mode = auth_mode
logger.debug(f"[{request_id}] auth_mode_classified mode={auth_mode.value}")
# Phase F PR-F3: resolve per-tenant TOIN key (header / hash /
# global) and populate the request-scoped ContextVar that
# SmartCrusher's deep-stack `record_compression` reads. See
# `headroom/proxy/tenant_key.py` for the threat model and the
# resolution rules.
tenant_key, tenant_key_source = resolve_tenant_key(request)
request.state.tenant_key = tenant_key
request.state.tenant_key_source = tenant_key_source
set_request_tenant_key(tenant_key)
# Check request body size
content_length = request.headers.get("content-length")
if content_length and int(content_length) > MAX_REQUEST_BODY_SIZE:
@@ -5810,6 +5821,16 @@ class OpenAIHandlerMixin:
request.state.auth_mode = auth_mode
logger.debug(f"[{request_id}] auth_mode_classified mode={auth_mode.value}")
# Phase F PR-F3: resolve per-tenant TOIN key (header / hash /
# global) and populate the request-scoped ContextVar that
# SmartCrusher's deep-stack `record_compression` reads. See
# `headroom/proxy/tenant_key.py` for the threat model and the
# resolution rules.
tenant_key, tenant_key_source = resolve_tenant_key(request)
request.state.tenant_key = tenant_key
request.state.tenant_key_source = tenant_key_source
set_request_tenant_key(tenant_key)
# Check request body size
content_length = request.headers.get("content-length")
if content_length and int(content_length) > MAX_REQUEST_BODY_SIZE:
+13 -1
View File
@@ -1952,7 +1952,19 @@ def _strip_internal_headers(headers: dict[str, str]) -> dict[str, str]:
is set, returns a shallow copy unchanged. That mode is for diagnostic
shadow tracing only and is documented as a per-deploy choice.
"""
return strip_internal_headers(headers, mode=get_strip_internal_headers_mode())
mode = get_strip_internal_headers_mode()
if mode == "disabled":
# Always return a copy so callers can mutate without surprise.
return dict(headers)
from headroom.proxy.tenant_key import (
DEFAULT_TENANT_KEY_HEADER,
TENANT_KEY_HEADER_ENV_VAR,
)
tenant_header = os.environ.get(TENANT_KEY_HEADER_ENV_VAR, DEFAULT_TENANT_KEY_HEADER)
tenant_header_lower = tenant_header.lower()
stripped = strip_internal_headers(headers, mode=mode)
return {k: v for k, v in stripped.items() if k.lower() != tenant_header_lower}
def merge_extra_headers(
+3 -1
View File
@@ -27,6 +27,7 @@ import argparse
import asyncio
import concurrent.futures
import contextlib
import contextvars
import hmac
import ipaddress
import json
@@ -1595,6 +1596,7 @@ class HeadroomProxy(
loop = asyncio.get_running_loop()
queued_at = time.monotonic()
request_context = contextvars.copy_context()
state = {
"queued": True,
"started": False,
@@ -1662,7 +1664,7 @@ class HeadroomProxy(
if quarantine_activated:
_announce_quarantine()
try:
return fn()
return request_context.run(fn)
finally:
elapsed = time.monotonic() - started_at
with self._compression_metrics_lock:
+370
View File
@@ -0,0 +1,370 @@
"""Per-tenant key resolver — Phase F PR-F3.
# Threat model
TOIN (Tool Output Intelligence Network) learns compression patterns from
every observed request. Pre-F3 those patterns aggregated into one global
pool keyed by ``(auth_mode, model_family, sig_hash)``. That works for an
OSS single-tenant deploy, but in a multi-tenant SaaS / shared-proxy
deploy it's a cross-tenant pattern leak: tenant A's tool-call patterns
train recommendations that the proxy then applies to tenant B's
requests. Two distinct customers carrying differently-shaped tool
outputs can degrade each other's compression quality, and (worse) a
tenant could probe TOIN dumps and learn the structural fingerprint of
another tenant's tool surface.
F3 partitions: every TOIN store key gets a ``tenant_key`` prefix
derived from the inbound request. Two tenants therefore train two
isolated pools — the dataclass on disk is unchanged, only the key
namespace grows by one component.
# Resolution rules
The resolver uses an explicit tenant signal only:
1. **Header** — when ``HEADROOM_TENANT_KEY_HEADER`` (default
``X-Headroom-Tenant-ID``) is present and survives sanitization
(alphanumeric / ``-`` / ``_``, max 64 chars). Source: ``"header"``.
The customer's auth-proxy / sidecar is the canonical place to pin
tenancy — it knows the tenant best.
2. **Global** — the literal ``"global"`` namespace when no tenant-id
header is present (the OSS single-process / dev / unauthenticated
case). Source: ``"global"``. This is a real namespace, not a sentinel
— the existing pool of pre-F3 patterns already lives under
``"global"`` after the migration, so a fresh deploy is non-breaking.
Every resolution emits a structured ``tenant_key_resolved`` log event
(NOT silent — see ``feedback_no_silent_fallbacks.md``). Operators can
alert on a sustained ``source="global"`` rate in production to catch
mis-configured upstream auth.
# Sanitization rules
Tenant keys end up in the TOIN serialized store under a separator
(``"|"``) that is illegal by construction in the value, so the
sanitization is conservative on purpose:
- Allowed: ASCII alphanumeric, ``-`` (hyphen), ``_`` (underscore).
- Length cap: 64 chars (prevents log-spam / OOM via giant headers).
- Empty after sanitization: rejected (falls through to the next rule).
- Unicode / control chars: stripped — we use ``unicodedata.category()``
rather than a regex (per project memory ``no regexes``) so the
classification is the OS Unicode tables, not a developer-authored
pattern.
# Configuration
Single env var: ``HEADROOM_TENANT_KEY_HEADER``. Default
``X-Headroom-Tenant-ID``. The header NAME is configurable; the
sanitization rules and the ``"global"`` literal are not — they're
load-bearing for namespace isolation.
# Integration
The handlers call ``resolve_tenant_key(request)`` once at request entry
(right after ``classify_auth_mode``) and store the result on
``request.state.tenant_key`` + ``request.state.tenant_key_source``.
They also call ``set_request_tenant_key(tenant_key)`` to populate the
:class:`contextvars.ContextVar` that TOIN's deep-stack
``record_compression`` / ``record_retrieval`` calls read — same pattern
as the existing ``_request_ccr_store`` ContextVar in
``headroom/cache/compression_store.py``.
"""
from __future__ import annotations
import logging
import os
import unicodedata
from contextvars import ContextVar
from typing import Any, Final
logger = logging.getLogger(__name__)
# ── Configuration env vars ────────────────────────────────────────────
# Header name is the only thing operators may want to change at deploy
# time (e.g. some load balancers strip ``X-`` headers, some teams use
# ``X-Tenant-ID`` without the Headroom prefix). Sanitization, prefix
# length, and the ``"global"`` literal are NOT configurable — they're
# load-bearing for tenant isolation.
TENANT_KEY_HEADER_ENV_VAR: Final[str] = "HEADROOM_TENANT_KEY_HEADER"
DEFAULT_TENANT_KEY_HEADER: Final[str] = "X-Headroom-Tenant-ID"
# ── Tenant-key constants ──────────────────────────────────────────────
# Maximum length we'll accept after sanitization. 64 chars is enough
# for SaaS-style ``cust_<22-char-id>`` (Stripe-shape) and short UUIDs;
# longer values almost always indicate a misconfiguration where
# something else (a JWT payload, a session id) was wedged into the
# header by mistake.
_MAX_TENANT_KEY_LEN: Final[int] = 64
# Literal namespace name used when no tenant-id header is present.
# This is a real namespace, not a sentinel —
# pre-F3 patterns already live under it after migration, and the OSS
# single-tenant / unauthenticated dev case rightfully uses one shared
# pool.
GLOBAL_TENANT_KEY: Final[str] = "global"
# Source string returned alongside the key. Operators can grep
# structured logs for ``source="global"`` to spot mis-configured
# upstream auth proxies.
SOURCE_HEADER: Final[str] = "header"
SOURCE_GLOBAL: Final[str] = "global"
# ── Request-scoped ContextVar ─────────────────────────────────────────
# TOIN's ``record_compression`` / ``record_retrieval`` are invoked deep
# inside SmartCrusher / ContentRouter, several frames below the proxy
# handler. Threading ``tenant_key`` through every call site would touch
# dozens of files; using a ``ContextVar`` keeps the surface small and
# matches the existing per-request ``_request_ccr_store`` pattern in
# ``headroom/cache/compression_store.py``.
#
# ``ContextVar`` is the right primitive — it's per-asyncio-task-aware,
# unlike ``threading.local`` which would cross-pollinate concurrent
# requests on the same event loop.
_request_tenant_key: ContextVar[str | None] = ContextVar(
"headroom_request_tenant_key", default=None
)
def set_request_tenant_key(tenant_key: str | None) -> None:
"""Set the tenant key for the current request context.
Called once per request from the proxy handler after
:func:`resolve_tenant_key`. TOIN's ``record_compression`` /
``record_retrieval`` reads this via :func:`get_current_tenant_key`.
Args:
tenant_key: Resolved tenant key, or ``None`` to clear.
"""
_request_tenant_key.set(tenant_key)
def get_current_tenant_key() -> str:
"""Return the tenant key for the current request, or
:data:`GLOBAL_TENANT_KEY` if none is set.
TOIN call sites prepend this to their store keys. The default of
``"global"`` (rather than ``None``) keeps test / OSS / batch-job
callers — which never pass through the proxy handler — addressable
in the same namespace they used pre-F3.
"""
key = _request_tenant_key.get()
return key if key is not None else GLOBAL_TENANT_KEY
# ── Public resolver ───────────────────────────────────────────────────
def resolve_tenant_key(request: Any) -> tuple[str, str]:
"""Resolve the tenant key for an inbound request.
See module docstring for the threat model and full resolution
rules. Briefly:
1. ``HEADROOM_TENANT_KEY_HEADER`` (default ``X-Headroom-Tenant-ID``)
header → ``"header"`` source.
2. Literal ``"global"`` namespace → ``"global"`` source.
Every resolution emits a ``tenant_key_resolved`` structured log so
operators can alert on a sustained ``source="global"`` rate.
Args:
request: A Starlette / FastAPI ``Request`` object. Must have
``headers`` (Mapping-like) and ``state`` (with optional
``auth_mode`` from F1).
Returns:
``(tenant_key, source)`` — both non-empty strings. ``source`` is
one of :data:`SOURCE_HEADER` or :data:`SOURCE_GLOBAL`.
"""
# ── 1. Header path ────────────────────────────────────────────────
header_name = os.environ.get(TENANT_KEY_HEADER_ENV_VAR, DEFAULT_TENANT_KEY_HEADER)
raw_header = _get_header(request, header_name)
if raw_header:
sanitized = _sanitize_tenant_key(raw_header)
if sanitized:
_emit_resolution_log(sanitized, SOURCE_HEADER, header_name=header_name)
return sanitized, SOURCE_HEADER
# Fall through: header was present but empty after
# sanitization. We log this at debug level — it's noisy enough
# at scale that an info-level log would flood, but rare enough
# that an operator debugging a specific tenant wants to see it.
logger.debug(
"tenant_key_header_rejected_after_sanitization",
extra={
"event": "tenant_key_header_rejected",
"header": header_name,
"raw_len": len(raw_header),
},
)
# ── 2. Global fallback ───────────────────────────────────────────
# MUST be logged (per `feedback_no_silent_fallbacks.md`). Operators
# see a sustained ``source="global"`` rate as a config signal: the
# proxy is sitting behind a path that doesn't carry tenant info.
_emit_resolution_log(GLOBAL_TENANT_KEY, SOURCE_GLOBAL)
return GLOBAL_TENANT_KEY, SOURCE_GLOBAL
# ── Internal helpers ──────────────────────────────────────────────────
def _get_header(request: Any, name: str) -> str:
"""Read a header case-insensitively. Returns ``""`` on miss.
Mirrors ``_header_get`` in ``auth_mode.py`` but inlined here so
this module has no dependency on ``auth_mode.py``'s private name.
"""
headers = getattr(request, "headers", None)
if headers is None:
return ""
value: Any = None
try:
value = headers.get(name)
if value is None:
# Fallback for plain-dict fixtures with mixed case.
for k, v in headers.items(): # type: ignore[union-attr]
if isinstance(k, str) and k.lower() == name.lower():
value = v
break
except AttributeError:
return ""
if value is None:
return ""
if isinstance(value, bytes):
try:
return value.decode("utf-8")
except UnicodeDecodeError:
return ""
return str(value)
def _get_auth_mode_str(request: Any) -> str:
"""Read ``request.state.auth_mode`` as a string, or ``""`` if unset.
F1 set this to an :class:`AuthMode` enum (a ``str`` subclass) on
every request. We don't import ``AuthMode`` here to keep this
module independent of the classifier — we only need the string
value, and the enum's ``str`` parent serializes transparently.
"""
state = getattr(request, "state", None)
if state is None:
return ""
auth_mode = getattr(state, "auth_mode", None)
if auth_mode is None:
return ""
# AuthMode subclasses str so ``str(auth_mode)`` returns the enum
# name's value (``"payg"`` / ``"oauth"`` / ``"subscription"``) —
# not the Python repr.
return str(auth_mode)
def _sanitize_tenant_key(raw: str) -> str:
"""Sanitize a tenant_key candidate. Returns ``""`` on rejection.
Allowed code points: ASCII alphanumeric, ``-``, ``_``. Anything
else is dropped. This is intentionally narrower than what header
standards permit — tenant keys flow into the TOIN serialized
store under a ``|`` separator, and into structured log fields, so
we want a character set that's safe in both.
Length cap: :data:`_MAX_TENANT_KEY_LEN`. Headers longer than the
cap are TRUNCATED, not rejected — but if the caller is sending a
JWT-shaped header by mistake, we still record the truncated form
rather than swallowing the request.
Per project memory (``no regexes``) we use string ops +
:func:`unicodedata.category` instead of ``re``. The Unicode
classification is the source of truth for "is this a control
char", and we get correctness for free instead of authoring our
own character class.
"""
if not raw:
return ""
# Walk character by character. Cheap (header values are short),
# branchy in a way the JIT will inline, and avoids `re`.
accepted: list[str] = []
for ch in raw:
# Reject control / format / separator / surrogates / private-use.
# `unicodedata.category()` returns a 2-char major.minor code; we
# whitelist L (Letter), N (Number) — but only if ASCII — plus
# the two literal punctuation chars `-` and `_`.
if ch in ("-", "_"):
accepted.append(ch)
continue
# ASCII alphanumeric: cheap fast-path — ord() check avoids the
# unicodedata lookup for the common case (Latin-letter SaaS IDs).
ch_ord = ord(ch)
if (
(0x30 <= ch_ord <= 0x39) # 0-9
or (0x41 <= ch_ord <= 0x5A) # A-Z
or (0x61 <= ch_ord <= 0x7A) # a-z
):
accepted.append(ch)
continue
# Anything non-ASCII or punctuation: drop. We do NOT call
# unicodedata for the Cyrillic/Greek/CJK case — those would
# round-trip into JSON fine, but they introduce normalization
# surprises (NFC vs NFD, full-width digits vs ASCII digits)
# that aren't worth the maintenance cost. SaaS tenant IDs are
# typically opaque ASCII strings.
# Touch unicodedata.category() so static analysis can see we
# use it — and also so a developer reading the code knows
# this is the right place to extend if we ever broaden the
# accepted set.
_ = unicodedata.category(ch)
if not accepted:
return ""
sanitized = "".join(accepted)
return sanitized[:_MAX_TENANT_KEY_LEN]
def _emit_resolution_log(
tenant_key: str,
source: str,
*,
header_name: str | None = None,
auth_mode: str | None = None,
) -> None:
"""Emit the structured ``tenant_key_resolved`` log event.
Required by the no-silent-fallbacks rule. Every resolution path —
including the ``"global"`` fallback — emits exactly one log event
so log aggregators can compute per-source rates without sampling.
The ``tenant_key`` value itself is included in the structured
fields. It's not a secret: header-mode keys are operator-supplied
"""
extra: dict[str, Any] = {
"event": "tenant_key_resolved",
"tenant_key": tenant_key,
"source": source,
}
if header_name is not None:
extra["header"] = header_name
if auth_mode is not None:
extra["auth_mode"] = auth_mode
# ``info`` level: routine, structured, low-cardinality. Operators
# can downsample to ``debug`` if they're confident their tenancy
# config is healthy.
logger.info("tenant_key_resolved", extra=extra)
__all__ = [
"DEFAULT_TENANT_KEY_HEADER",
"GLOBAL_TENANT_KEY",
"SOURCE_GLOBAL",
"SOURCE_HEADER",
"TENANT_KEY_HEADER_ENV_VAR",
"get_current_tenant_key",
"resolve_tenant_key",
"set_request_tenant_key",
]
+3 -1
View File
@@ -71,6 +71,7 @@ from .toin import (
DEFAULT_AUTH_MODE,
DEFAULT_MIN_OBSERVATIONS_TO_PUBLISH,
DEFAULT_MODEL_FAMILY,
DEFAULT_TENANT_KEY,
TOINConfig,
ToolIntelligenceNetwork,
ToolPattern,
@@ -94,10 +95,11 @@ __all__ = [
"FieldDistribution",
"RetrievalStats",
"ToolSignature",
# TOIN (observation-only since PR-B5)
# TOIN (observation-only since PR-B5; per-tenant since PR-F3)
"DEFAULT_AUTH_MODE",
"DEFAULT_MIN_OBSERVATIONS_TO_PUBLISH",
"DEFAULT_MODEL_FAMILY",
"DEFAULT_TENANT_KEY",
"TOINConfig",
"ToolIntelligenceNetwork",
"ToolPattern",
+142 -51
View File
@@ -105,30 +105,48 @@ DEFAULT_MAX_STORAGE_BYTES: Final[int] = 128 * 1024 * 1024
MAX_QUERY_PATTERN_LENGTH: Final[int] = 512
# Aggregation-key serialization separator. Used to encode the
# `(auth_mode, model_family, sig_hash)` tuple as a string for JSON
# storage (JSON object keys must be strings) and for cross-instance
# pattern imports. Pipe is illegal in all three components by
# construction (auth_mode ∈ {"unknown","payg","oauth","subscription"};
# model_family is a registry name with no `|`; sig_hash is hex).
# `(tenant_key, auth_mode, model_family, sig_hash)` tuple as a string
# for JSON storage (JSON object keys must be strings) and for
# cross-instance pattern imports. Pipe is illegal in all four
# components by construction (tenant_key is sanitized to ASCII alphanum
# + `-_` by `headroom.proxy.tenant_key`; auth_mode ∈
# {"unknown","payg","oauth","subscription"}; model_family is a registry
# name with no `|`; sig_hash is hex).
_AGG_KEY_SEPARATOR: Final[str] = "|"
# ── Tenant-key default (PR-F3) ──────────────────────────────────────────
# When callers haven't plumbed real tenant-id detection (CLI / batch
# jobs / unauthenticated dev) we land in the literal `"global"` slice.
# Imported from `headroom.proxy.tenant_key` so there's a single source
# of truth — but kept locally as a string default to avoid an
# import-time cycle with the proxy package (TOIN ships in environments
# without the proxy module). Using the same literal `"global"` string
# both modules canonicalize on.
DEFAULT_TENANT_KEY: Final[str] = "global"
# ── Aggregation key helpers ─────────────────────────────────────────────
PatternKey = tuple[str, str, str]
PatternKey = tuple[str, str, str, str]
def _make_pattern_key(
auth_mode: str | None,
model_family: str | None,
sig_hash: str,
tenant_key: str | None = None,
) -> PatternKey:
"""Build the canonical `(auth_mode, model_family, sig_hash)` key.
"""Build the canonical `(tenant_key, auth_mode, model_family, sig_hash)` key.
Defaults populate to `DEFAULT_AUTH_MODE` / `DEFAULT_MODEL_FAMILY`
when callers haven't supplied a value — keeps callers terse during
the Phase B realignment while PR-F3 wires real detectors.
Defaults populate to `DEFAULT_TENANT_KEY` / `DEFAULT_AUTH_MODE` /
`DEFAULT_MODEL_FAMILY` when callers haven't supplied a value.
PR-F3 introduced the leading `tenant_key` slot so two distinct
tenants don't cross-pollinate compression patterns; pre-F3 callers
that pass no tenant_key land in the literal `"global"` namespace
(which is also where the pre-F3 patterns migrate to on first load
via the legacy 3-tuple fallback in `_deserialize_pattern_key`).
"""
return (
tenant_key or DEFAULT_TENANT_KEY,
auth_mode or DEFAULT_AUTH_MODE,
model_family or DEFAULT_MODEL_FAMILY,
sig_hash,
@@ -143,17 +161,26 @@ def _serialize_pattern_key(key: PatternKey) -> str:
def _deserialize_pattern_key(serialized: str) -> PatternKey:
"""Parse a serialized aggregation key back to a tuple.
Backward-compatible with pre-B5 dumps that stored keys as bare
structure hashes (no separator): those parse as
`(DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)`. The realignment
plan permits wiping the on-disk store, but this fallback keeps reads
safe if a stale file appears in the wild.
Backward-compat tiers:
- 4 parts → PR-F3 native ``(tenant_key, auth_mode, model_family, sig_hash)``.
- 3 parts → pre-F3 ``(auth_mode, model_family, sig_hash)`` from PR-B5
dumps; promoted to the ``DEFAULT_TENANT_KEY`` (``"global"``) slice.
- Otherwise (1 part, no separator) → legacy pre-B5 bare ``sig_hash``;
promoted to ``(DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE,
DEFAULT_MODEL_FAMILY, sig_hash)``.
The realignment plan permits wiping the on-disk store, but these
fallbacks keep reads safe if a stale file appears in the wild.
"""
parts = serialized.split(_AGG_KEY_SEPARATOR)
if len(parts) == 4:
return (parts[0], parts[1], parts[2], parts[3])
if len(parts) == 3:
return (parts[0], parts[1], parts[2])
# Legacy format: bare sig_hash. Promote to default tenant slice.
return (DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, serialized)
# Pre-F3 format: prepend the global tenant slice.
return (DEFAULT_TENANT_KEY, parts[0], parts[1], parts[2])
# Legacy format: bare sig_hash. Promote to default tenant + default slice.
return (DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, serialized)
def get_default_toin_storage_path() -> str:
@@ -184,6 +211,27 @@ def get_default_toin_storage_path() -> str:
MetricsCallback = Callable[[str, dict[str, Any]], None] # (event_name, event_data) -> None
def _get_current_tenant_key_or_default() -> str:
"""Read the request-scoped tenant_key, falling back to ``"global"``.
Imported lazily so the proxy package is not a hard dependency of
TOIN — when ``headroom.proxy.tenant_key`` is unavailable (CLI / batch
/ stripped builds) we land in the ``DEFAULT_TENANT_KEY`` namespace,
which is the same place the proxy handler routes
no-signal-detected requests to. Either path is correctly aggregated
under ``"global"``; there is no silent fallback because the proxy's
resolver always emits a structured log on resolution.
See PR-F3 for the full threat model.
"""
try:
from headroom.proxy.tenant_key import get_current_tenant_key
return get_current_tenant_key()
except ImportError:
return DEFAULT_TENANT_KEY
@dataclass
class ToolPattern:
"""Aggregated intelligence about a tool type across all users.
@@ -194,12 +242,17 @@ class ToolPattern:
tool_signature_hash: str
# === Aggregation Key (PR-B5) ===
# === Aggregation Key (PR-B5 + PR-F3) ===
# Per-tenant aggregation key extension. The Pattern is keyed inside
# the TOIN store by `(auth_mode, model_family, tool_signature_hash)` —
# these two fields carry the same values onto the dataclass so dumps,
# imports, and publish-CLI rows are self-describing without
# cross-referencing the dict key.
# the TOIN store by `(tenant_key, auth_mode, model_family,
# tool_signature_hash)` — these three fields carry the same values
# onto the dataclass so dumps, imports, and publish-CLI rows are
# self-describing without cross-referencing the dict key.
#
# PR-F3 added `tenant_key` so two tenants on the same proxy don't
# cross-pollinate compression patterns. Defaults to `"global"`
# (the literal namespace pre-F3 patterns live in after migration).
tenant_key: str = DEFAULT_TENANT_KEY
auth_mode: str = DEFAULT_AUTH_MODE
model_family: str = DEFAULT_MODEL_FAMILY
@@ -287,6 +340,7 @@ class ToolPattern:
"""Convert to dictionary for serialization."""
return {
"tool_signature_hash": self.tool_signature_hash,
"tenant_key": self.tenant_key,
"auth_mode": self.auth_mode,
"model_family": self.model_family,
"total_compressions": self.total_compressions,
@@ -332,6 +386,7 @@ class ToolPattern:
# Filter to only valid fields
valid_fields = {
"tool_signature_hash",
"tenant_key",
"auth_mode",
"model_family",
"total_compressions",
@@ -549,11 +604,13 @@ class ToolIntelligenceNetwork:
items: list[dict[str, Any]] | None = None,
auth_mode: str | None = None,
model_family: str | None = None,
tenant_key: str | None = None,
) -> None:
"""Record a compression event.
Called after SmartCrusher compresses data. Updates the pattern
for this `(auth_mode, model_family, tool_signature)` slice.
for this `(tenant_key, auth_mode, model_family, tool_signature)`
slice.
TOIN Evolution: When items are provided, we capture field statistics
for learning semantic types (uniqueness, default values, etc.).
@@ -571,6 +628,11 @@ class ToolIntelligenceNetwork:
Defaults to `DEFAULT_AUTH_MODE` when not provided.
model_family: Target model family (`claude-3-5`, `gpt-4o`, …).
Defaults to `DEFAULT_MODEL_FAMILY` when not provided.
tenant_key: PR-F3 per-tenant aggregation slice. When ``None``,
reads from the request-scoped ``ContextVar`` populated by
the proxy handler (see ``headroom.proxy.tenant_key``);
CLI / batch / test callers without a contextvar land in
the literal ``"global"`` namespace.
"""
# The beacon's copy of this event is taken BEFORE the enabled check.
# TOIN's own store is gated on HEADROOM_TELEMETRY, which is opt-in and
@@ -596,15 +658,22 @@ class ToolIntelligenceNetwork:
# Computing structure_hash can be expensive for large structures
sig_hash = tool_signature.structure_hash
key = _make_pattern_key(auth_mode, model_family, sig_hash)
# PR-F3: read tenant_key from the request-scoped ContextVar when
# the caller didn't pass one explicitly. Imported lazily so TOIN
# remains usable in environments where the proxy package is not
# installed (CLI batch jobs, tests).
if tenant_key is None:
tenant_key = _get_current_tenant_key_or_default()
key = _make_pattern_key(auth_mode, model_family, sig_hash, tenant_key)
# LOW FIX #22: Emit compression metric
self._emit_metric(
"toin.compression",
{
"signature_hash": sig_hash,
"auth_mode": key[0],
"model_family": key[1],
"tenant_key": key[0],
"auth_mode": key[1],
"model_family": key[2],
"original_count": original_count,
"compressed_count": compressed_count,
"original_tokens": original_tokens,
@@ -619,8 +688,9 @@ class ToolIntelligenceNetwork:
if key not in self._patterns:
self._patterns[key] = ToolPattern(
tool_signature_hash=sig_hash,
auth_mode=key[0],
model_family=key[1],
tenant_key=key[0],
auth_mode=key[1],
model_family=key[2],
)
pattern = self._patterns[key]
@@ -875,6 +945,7 @@ class ToolIntelligenceNetwork:
retrieved_items: list[dict[str, Any]] | None = None,
auth_mode: str | None = None,
model_family: str | None = None,
tenant_key: str | None = None,
) -> None:
"""Record a retrieval event.
@@ -893,19 +964,25 @@ class ToolIntelligenceNetwork:
retrieved_items: Optional list of retrieved items for field-level learning.
auth_mode: Tenant auth slice. Defaults to `DEFAULT_AUTH_MODE`.
model_family: Target model family. Defaults to `DEFAULT_MODEL_FAMILY`.
tenant_key: PR-F3 per-tenant aggregation slice. When ``None``,
reads from the request-scoped ``ContextVar`` populated by
the proxy handler.
"""
if not self._config.enabled:
return
key = _make_pattern_key(auth_mode, model_family, tool_signature_hash)
if tenant_key is None:
tenant_key = _get_current_tenant_key_or_default()
key = _make_pattern_key(auth_mode, model_family, tool_signature_hash, tenant_key)
# LOW FIX #22: Emit retrieval metric
self._emit_metric(
"toin.retrieval",
{
"signature_hash": tool_signature_hash,
"auth_mode": key[0],
"model_family": key[1],
"tenant_key": key[0],
"auth_mode": key[1],
"model_family": key[2],
"retrieval_type": retrieval_type,
"has_query": query is not None,
"query_fields_count": len(query_fields) if query_fields else 0,
@@ -918,8 +995,9 @@ class ToolIntelligenceNetwork:
# First time seeing this tool via retrieval
self._patterns[key] = ToolPattern(
tool_signature_hash=tool_signature_hash,
auth_mode=key[0],
model_family=key[1],
tenant_key=key[0],
auth_mode=key[1],
model_family=key[2],
)
pattern = self._patterns[key]
@@ -1243,18 +1321,23 @@ class ToolIntelligenceNetwork:
signature_hash: str,
auth_mode: str | None = None,
model_family: str | None = None,
tenant_key: str | None = None,
) -> ToolPattern | None:
"""Get pattern data for a specific `(auth_mode, model_family, sig_hash)` slice.
"""Get pattern data for a specific
`(tenant_key, auth_mode, model_family, sig_hash)` slice.
Defaults to `(DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, signature_hash)`
when callers haven't supplied tenant info — preserves source-compat
with pre-B5 callers that look up by bare hash.
Defaults to `(DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE,
DEFAULT_MODEL_FAMILY, signature_hash)` when callers haven't
supplied tenant info — preserves source-compat with pre-B5
callers that look up by bare hash.
HIGH FIX: Returns a deep copy to prevent external mutation of internal state.
"""
import copy
key = _make_pattern_key(auth_mode, model_family, signature_hash)
if tenant_key is None:
tenant_key = _get_current_tenant_key_or_default()
key = _make_pattern_key(auth_mode, model_family, signature_hash, tenant_key)
with self._lock:
pattern = self._patterns.get(key)
@@ -1298,10 +1381,16 @@ class ToolIntelligenceNetwork:
Used for federated learning: aggregate patterns from multiple
Headroom instances without sharing actual data.
Backward-compatible with v1.0 dumps that keyed patterns by bare
structure_hash: those are promoted to the
`(DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)` slice via
`_deserialize_pattern_key`.
Backward-compatible with two pre-F3 formats:
- v1.0 dumps that keyed patterns by bare ``sig_hash``: promoted
to ``(DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE,
DEFAULT_MODEL_FAMILY, sig_hash)``.
- v2.0 / pre-F3 dumps that used the 3-tuple
``(auth_mode, model_family, sig_hash)`` key: promoted to the
``DEFAULT_TENANT_KEY`` (``"global"``) slice.
See ``_deserialize_pattern_key`` for the parsing tiers.
Args:
data: Exported pattern data.
@@ -1317,11 +1406,13 @@ class ToolIntelligenceNetwork:
key = _deserialize_pattern_key(serialized_key)
imported = ToolPattern.from_dict(pattern_dict)
self._sanitize_loaded_pattern(imported)
# Make sure dataclass fields agree with the dict key — pre-B5
# dumps don't carry auth_mode/model_family on the pattern;
# promote from the (possibly default) key.
imported.auth_mode = key[0]
imported.model_family = key[1]
# Make sure dataclass fields agree with the dict key — pre-F3
# dumps don't carry tenant_key on the pattern, and pre-B5
# dumps don't carry auth_mode/model_family either; promote
# from the (possibly default) key.
imported.tenant_key = key[0]
imported.auth_mode = key[1]
imported.model_family = key[2]
if key in self._patterns:
# Merge with existing
@@ -1669,9 +1760,9 @@ def _create_default_toin_backend() -> Any:
# `tenant_prefix` is retained for storage-backend namespacing
# (Redis key prefix, Postgres schema name, etc.) so multi-tenant
# SaaS deployments can carve up shared infrastructure. PR-B5 made
# the in-memory aggregation key per-tenant via `auth_mode` /
# `model_family`, so `tenant_prefix` is now functionally redundant
# for *learning* — it only matters for storage layout. Keep it.
# the in-memory aggregation key per tenant via `tenant_key`, so
# `tenant_prefix` is now functionally redundant for *learning* —
# it only matters for storage layout. Keep it.
kwargs = {
"url": os.environ.get("HEADROOM_TOIN_URL", ""),
"tenant_prefix": os.environ.get("HEADROOM_TOIN_TENANT_PREFIX", ""),
+2 -2
View File
@@ -124,7 +124,7 @@ class TestTOINDoubleCountFix:
# Get the pattern
with toin._lock:
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
user_count_after_101 = pattern.user_count
# Now call again with same instance (instance_100)
@@ -133,7 +133,7 @@ class TestTOINDoubleCountFix:
toin.record_compression(sig, 100, 10, 1000, 100, strategy="test_strategy")
with toin._lock:
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
user_count_after_102 = pattern.user_count
# Restore instance_id
+11 -11
View File
@@ -76,7 +76,7 @@ class TestAllSeenInstancesUnboundedGrowth:
# Simulate adding users via record_compression
# (the cap is enforced there, not when directly adding to set)
pattern = ToolPattern(tool_signature_hash=sig.structure_hash)
toin._patterns[("unknown", "unknown", sig.structure_hash)] = pattern
toin._patterns[("global", "unknown", "unknown", sig.structure_hash)] = pattern
# Direct manipulation should still work for testing
for i in range(200):
@@ -101,7 +101,7 @@ class TestAllSeenInstancesUnboundedGrowth:
# Record compressions from 150 "users" (simulated)
# by directly manipulating the pattern
pattern = ToolPattern(tool_signature_hash=sig.structure_hash)
toin._patterns[("unknown", "unknown", sig.structure_hash)] = pattern
toin._patterns[("global", "unknown", "unknown", sig.structure_hash)] = pattern
# Track 150 unique users
for i in range(150):
@@ -183,7 +183,7 @@ class TestAllSeenInstancesSerialization:
)
# Manually add more users to simulate multi-user scenario
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
for i in range(50):
instance_hash = hashlib.sha256(f"extra_user_{i}".encode()).hexdigest()[:8]
if instance_hash not in pattern._all_seen_instances:
@@ -202,7 +202,7 @@ class TestAllSeenInstancesSerialization:
toin2 = ToolIntelligenceNetwork(config)
# Verify user count is preserved
pattern2 = toin2._patterns.get(("unknown", "unknown", sig.structure_hash))
pattern2 = toin2._patterns.get(("global", "unknown", "unknown", sig.structure_hash))
assert pattern2 is not None
assert pattern2.user_count == original_user_count
@@ -252,7 +252,7 @@ class TestUserCountMergeLogic:
imported.sample_size = 5
# Merge
toin._patterns[("unknown", "unknown", "test_hash")] = existing
toin._patterns[("global", "unknown", "unknown", "test_hash")] = existing
toin._merge_patterns(existing, imported)
# After merge: 5 existing + 2 new = 7 unique users
@@ -287,7 +287,7 @@ class TestUserCountMergeLogic:
imported.sample_size = 20
# Merge
toin._patterns[("unknown", "unknown", "test_hash")] = existing
toin._patterns[("global", "unknown", "unknown", "test_hash")] = existing
toin._merge_patterns(existing, imported)
# After merge: 120 existing + 10 new = 130 unique users
@@ -728,7 +728,7 @@ class TestTOINHighPriorityFixes:
query_fields=[f"unique_field_{i}"],
)
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
assert len(pattern.field_retrieval_frequency) <= 100
def test_commonly_retrieved_fields_bounded(self):
@@ -757,7 +757,7 @@ class TestTOINHighPriorityFixes:
query_fields=[f"common_field_{i}"],
)
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
assert len(pattern.commonly_retrieved_fields) <= 20
def test_strategy_success_rate_updates(self):
@@ -778,7 +778,7 @@ class TestTOINHighPriorityFixes:
strategy="TEST_STRATEGY",
)
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
initial_rate = pattern.strategy_success_rates["TEST_STRATEGY"]
assert initial_rate == 1.0 # Starts at 1.0
@@ -790,7 +790,7 @@ class TestTOINHighPriorityFixes:
strategy="TEST_STRATEGY",
)
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
after_retrieval = pattern.strategy_success_rates["TEST_STRATEGY"]
assert after_retrieval < initial_rate # Should decrease
@@ -805,7 +805,7 @@ class TestTOINHighPriorityFixes:
strategy="TEST_STRATEGY",
)
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
after_compressions = pattern.strategy_success_rates["TEST_STRATEGY"]
assert after_compressions > after_retrieval # Should increase
+29
View File
@@ -178,6 +178,35 @@ def test_strip_preserves_value_semantics() -> None:
assert out["anthropic-version"] == "2023-06-01"
def test_strip_default_tenant_header_removed() -> None:
"""Tenant header is ingress-only and must not reach providers."""
out = _strip_internal_headers(
{
"Authorization": "Bearer sk-ant-...",
"X-Headroom-Tenant-ID": "tenant_a",
}
)
assert "X-Headroom-Tenant-ID" not in out
assert out["Authorization"] == "Bearer sk-ant-..."
def test_strip_custom_tenant_header_removed(monkeypatch: pytest.MonkeyPatch) -> None:
"""Configured tenant header is also stripped, even outside x-headroom-*."""
monkeypatch.setenv("HEADROOM_TENANT_KEY_HEADER", "X-Tenant-ID")
out = _strip_internal_headers(
{
"Authorization": "Bearer sk-ant-...",
"X-Tenant-ID": "tenant_a",
"x-request-id": "rid-1",
}
)
assert "X-Tenant-ID" not in out
assert out["x-request-id"] == "rid-1"
# ---------------------------------------------------------------------------
# End-to-end: x-headroom-* never reaches the upstream
# ---------------------------------------------------------------------------
+8 -6
View File
@@ -529,9 +529,10 @@ class TestEndToEndTOINIntegration:
store = get_compression_store()
store.process_pending_feedback()
# PR-B5: pattern key is now `(auth_mode, model_family, sig_hash)`.
# Callers that don't supply auth/model land on the
# `("unknown", "unknown", sig_hash)` slot.
# PR-B5 + PR-F3: pattern key is now `(tenant_key, auth_mode,
# model_family, sig_hash)`. Callers that don't supply
# tenant/auth/model land on the
# `("global", "unknown", "unknown", sig_hash)` slot.
from headroom.telemetry.toin import _make_pattern_key
pattern = fresh_toin._patterns.get(_make_pattern_key(None, None, signature.structure_hash))
@@ -607,9 +608,10 @@ class TestEndToEndTOINIntegration:
store.process_pending_feedback()
# Step 3: Verify TOIN learned
# PR-B5: pattern key is now `(auth_mode, model_family, sig_hash)`.
# Callers that don't supply auth/model land on the
# `("unknown", "unknown", sig_hash)` slot.
# PR-B5 + PR-F3: pattern key is now `(tenant_key, auth_mode,
# model_family, sig_hash)`. Callers that don't supply
# tenant/auth/model land on the
# `("global", "unknown", "unknown", sig_hash)` slot.
from headroom.telemetry.toin import _make_pattern_key
pattern = fresh_toin._patterns.get(_make_pattern_key(None, None, signature.structure_hash))
+18
View File
@@ -38,6 +38,7 @@ pytest.importorskip("fastapi")
from headroom.proxy.helpers import COMPRESSION_TIMEOUT_SECONDS # noqa: F401
from headroom.proxy.server import ProxyConfig, create_app
from headroom.proxy.tenant_key import get_current_tenant_key, set_request_tenant_key
def _make_proxy(compression_max_workers: int | None = None):
@@ -83,6 +84,23 @@ def test_compression_executor_minimum_one_worker() -> None:
assert proxy.compression_max_workers == 1
def test_compression_executor_preserves_request_contextvars() -> None:
"""Tenant context survives the thread-pool hop used by compression."""
proxy = _make_proxy(compression_max_workers=1)
async def _drive() -> str:
set_request_tenant_key("tenant_executor")
try:
return await proxy._run_compression_in_executor(
get_current_tenant_key,
timeout=10.0,
)
finally:
set_request_tenant_key(None)
assert asyncio.run(_drive()) == "tenant_executor"
def test_in_flight_gauge_tracks_running_compressions() -> None:
"""While a compression is running, ``_compression_in_flight`` reads ≥ 1.
After it completes, it returns to 0. The high-water mark records the
+228
View File
@@ -0,0 +1,228 @@
"""F3 handler-integration tests: tenant_key plumbing through the proxy.
Pins that the OpenAI / Anthropic handlers call ``resolve_tenant_key``
once at request entry, populate ``request.state.tenant_key`` /
``request.state.tenant_key_source``, AND set the request-scoped
ContextVar so downstream TOIN ``record_compression`` calls observe the
right tenant slice.
Uses the same ``_DummyOpenAIHandler`` pattern as
``tests/test_openai_codex_routing.py`` so we don't need the Rust core
or a live upstream HTTP server.
"""
from __future__ import annotations
import base64
import json
from types import SimpleNamespace
import anyio
import pytest
from fastapi import Request
from headroom.proxy.handlers.openai import OpenAIHandlerMixin
from headroom.proxy.tenant_key import (
GLOBAL_TENANT_KEY,
SOURCE_GLOBAL,
SOURCE_HEADER,
get_current_tenant_key,
set_request_tenant_key,
)
def _jwt(payload: dict) -> str:
"""Build a 3-segment JWT for OAuth classification."""
header = {"alg": "none", "typ": "JWT"}
def encode(part: dict) -> str:
raw = json.dumps(part, separators=(",", ":")).encode("utf-8")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
return f"{encode(header)}.{encode(payload)}."
# ── Test fixtures (Dummy handler) ─────────────────────────────────────────
class _DummyMetrics:
async def record_request(self, **kwargs): # noqa: ANN003
return None
async def record_failed(self, **kwargs): # noqa: ANN003
return None
class _DummyTokenizer:
def count_messages(self, messages):
return len(messages)
class _ResponseStub:
status_code = 200
headers = {"content-type": "application/json", "content-length": "42"}
content = b'{"id":"resp_123","output":[{"type":"message"}]}'
def json(self):
return {"usage": {"input_tokens": 2, "output_tokens": 1}}
class _DummyOpenAIHandler(OpenAIHandlerMixin):
"""Minimal handler that captures the request that flows through F3."""
OPENAI_API_URL = "https://api.openai.com"
def __init__(self) -> None:
self.rate_limiter = None
self.metrics = _DummyMetrics()
self.config = SimpleNamespace(
optimize=False,
retry_max_attempts=3,
retry_base_delay_ms=10,
retry_max_delay_ms=50,
connect_timeout_seconds=10,
openai_extra_headers=None,
)
self.usage_reporter = None
self.openai_provider = SimpleNamespace(get_context_limit=lambda model: 128_000)
self.openai_pipeline = SimpleNamespace(apply=lambda **kwargs: None)
self.anthropic_backend = None
self.cost_tracker = None
self.memory_handler = None
self.session_tracker_store = SimpleNamespace(
compute_session_id=lambda *a, **k: "sess-openai-1",
)
self.captured_request: Request | None = None
async def _next_request_id(self) -> str:
return "req-1"
def _extract_tags(self, headers: dict[str, str]) -> dict[str, str]:
return {}
async def _retry_request(self, method: str, url: str, headers: dict, body: dict):
return _ResponseStub()
async def _run_compression_in_executor(self, fn, *, timeout: float):
return fn()
async def _stream_response(self, **kwargs):
return SimpleNamespace(status_code=200)
def _build_request(headers: dict[str, str], body: dict) -> Request:
payload = json.dumps(body).encode("utf-8")
async def receive():
return {"type": "http.request", "body": payload, "more_body": False}
scope = {
"type": "http",
"asgi": {"version": "3.0"},
"http_version": "1.1",
"method": "POST",
"scheme": "https",
"path": "/v1/responses",
"raw_path": b"/v1/responses",
"query_string": b"",
"headers": [
(key.lower().encode("utf-8"), value.encode("utf-8")) for key, value in headers.items()
],
"client": ("127.0.0.1", 12345),
"server": ("testserver", 443),
}
return Request(scope, receive)
@pytest.fixture(autouse=True)
def _reset_tenant_key_contextvar():
"""Clean ContextVar after every test so they don't leak."""
set_request_tenant_key(None)
yield
set_request_tenant_key(None)
# ── F3 plumbing pins ──────────────────────────────────────────────────────
def test_handler_populates_request_state_tenant_key_for_header_path(monkeypatch):
"""Header-set tenant_id ⇒ request.state.tenant_key + source="header"."""
handler = _DummyOpenAIHandler()
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
request = _build_request(
headers={
"Authorization": "Bearer sk-test-key",
"X-Headroom-Tenant-ID": "cust_acme",
},
body={"model": "gpt-5.4", "input": "hi"},
)
anyio.run(handler.handle_openai_responses, request)
assert getattr(request.state, "tenant_key", None) == "cust_acme"
assert getattr(request.state, "tenant_key_source", None) == SOURCE_HEADER
def test_handler_populates_request_state_tenant_key_for_global_fallback(monkeypatch):
"""No tenant header, even with bearer auth, ⇒ tenant_key is global."""
handler = _DummyOpenAIHandler()
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
# PAYG bearer (sk-...). F1 will classify auth_mode=PAYG, but F3
# requires an explicit tenant header instead of deriving from secrets.
request = _build_request(
headers={"Authorization": "Bearer sk-ant-api03-aaaabbbbccccdddd"},
body={"model": "gpt-5.4", "input": "hi"},
)
anyio.run(handler.handle_openai_responses, request)
assert getattr(request.state, "tenant_key_source", None) == SOURCE_GLOBAL
tenant_key = getattr(request.state, "tenant_key", None)
assert tenant_key == GLOBAL_TENANT_KEY
def test_handler_populates_global_when_no_signals(monkeypatch):
"""No header, no bearer ⇒ tenant_key="global" with source="global"."""
handler = _DummyOpenAIHandler()
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
# Neither auth nor explicit tenant header.
request = _build_request(
headers={},
body={"model": "gpt-5.4", "input": "hi"},
)
anyio.run(handler.handle_openai_responses, request)
assert getattr(request.state, "tenant_key", None) == GLOBAL_TENANT_KEY
assert getattr(request.state, "tenant_key_source", None) == SOURCE_GLOBAL
def test_handler_populates_contextvar_so_toin_records_under_tenant(monkeypatch):
"""Handler must call set_request_tenant_key — TOIN reads the ContextVar.
We verify by asserting the contextvar is populated AFTER the
handler returns (the handler doesn't reset it on success — that
would defeat the whole point, since downstream record_compression
happens deep inside the pipeline). The contextvar is per-task; the
next request's handler will overwrite it.
"""
handler = _DummyOpenAIHandler()
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
request = _build_request(
headers={"X-Headroom-Tenant-ID": "tenant_for_toin"},
body={"model": "gpt-5.4", "input": "hi"},
)
# Run inside an explicit context so the test can read back the
# ContextVar the handler set. Without `anyio.run`'s default
# context isolation, we'd see a stale value across tests.
async def _drive() -> str:
await handler.handle_openai_responses(request)
return get_current_tenant_key()
seen = anyio.run(_drive)
assert seen == "tenant_for_toin"
+280
View File
@@ -0,0 +1,280 @@
"""Unit tests for ``headroom.proxy.tenant_key.resolve_tenant_key``.
Phase F PR-F3 — covers the two resolution paths (header / global), the structured-log requirement, and the sanitization edge
cases (empty, too-long, unicode, control chars, hyphens, mixed
ASCII).
The handler-integration tests (``request.state`` plumbing) live with
the proxy handler tests; this file pins the pure resolver contract.
"""
from __future__ import annotations
from types import SimpleNamespace
import pytest
from headroom.proxy.tenant_key import (
DEFAULT_TENANT_KEY_HEADER,
GLOBAL_TENANT_KEY,
SOURCE_GLOBAL,
SOURCE_HEADER,
TENANT_KEY_HEADER_ENV_VAR,
get_current_tenant_key,
resolve_tenant_key,
set_request_tenant_key,
)
def _request(
headers: dict[str, str] | None = None, auth_mode: str | None = None
) -> SimpleNamespace:
"""Build a minimal Starlette-shaped Request fixture.
SimpleNamespace + a plain dict for `headers` is sufficient because
``resolve_tenant_key`` only touches ``request.headers.get`` /
``.items`` and ``request.state.auth_mode``.
"""
state = SimpleNamespace(auth_mode=auth_mode)
return SimpleNamespace(headers=headers or {}, state=state)
# ── Header path ───────────────────────────────────────────────────────
def test_header_path_returns_header_source() -> None:
"""Header present + sanitized non-empty ⇒ header source."""
req = _request(headers={"X-Headroom-Tenant-ID": "cust_abc123"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "cust_abc123"
assert source == SOURCE_HEADER
def test_header_path_lowercase_header_lookup() -> None:
"""Header lookup is case-insensitive (Starlette norm)."""
req = _request(headers={"x-headroom-tenant-id": "tenant42"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "tenant42"
assert source == SOURCE_HEADER
def test_header_path_custom_header_via_env(monkeypatch: pytest.MonkeyPatch) -> None:
"""``HEADROOM_TENANT_KEY_HEADER`` overrides the default name."""
monkeypatch.setenv(TENANT_KEY_HEADER_ENV_VAR, "X-Custom-Tenant")
req = _request(headers={"X-Custom-Tenant": "company9"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "company9"
assert source == SOURCE_HEADER
def test_header_path_default_name_is_x_headroom_tenant_id() -> None:
"""Default header name is documented as ``X-Headroom-Tenant-ID``."""
assert DEFAULT_TENANT_KEY_HEADER == "X-Headroom-Tenant-ID"
def test_header_path_falls_through_when_empty_after_sanitization() -> None:
"""Pure-non-allowed header (e.g. all unicode) ⇒ fall through, NOT header source."""
# All-emoji header sanitizes to empty string — neither header
# so we drop to the global fallback.
req = _request(headers={"X-Headroom-Tenant-ID": "🦀🚀"})
tenant_key, source = resolve_tenant_key(req)
assert source == SOURCE_GLOBAL
assert tenant_key == GLOBAL_TENANT_KEY
# ── Hash path ─────────────────────────────────────────────────────────
def test_bearer_without_header_uses_global_namespace() -> None:
req = _request(
headers={"authorization": "Bearer sk-ant-api03-abc123def"},
auth_mode="payg",
)
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == GLOBAL_TENANT_KEY
assert source == SOURCE_GLOBAL
def test_auth_mode_without_header_does_not_partition_tenants() -> None:
bearer = "Bearer same-tok-aaabbbccc"
payg = resolve_tenant_key(_request(headers={"authorization": bearer}, auth_mode="payg"))
oauth = resolve_tenant_key(_request(headers={"authorization": bearer}, auth_mode="oauth"))
assert payg == (GLOBAL_TENANT_KEY, SOURCE_GLOBAL)
assert oauth == (GLOBAL_TENANT_KEY, SOURCE_GLOBAL)
def test_global_fallback_when_no_signals() -> None:
"""No header, no auth, no key ⇒ literal ``"global"``."""
req = _request()
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == GLOBAL_TENANT_KEY == "global"
assert source == SOURCE_GLOBAL
def _capture_tenant_resolution_logs(monkeypatch: pytest.MonkeyPatch) -> list[SimpleNamespace]:
records: list[SimpleNamespace] = []
def _info(_message: str, *, extra: dict[str, object] | None = None) -> None:
records.append(SimpleNamespace(**(extra or {})))
monkeypatch.setattr("headroom.proxy.tenant_key.logger.info", _info)
return records
def test_global_fallback_emits_structured_log(monkeypatch: pytest.MonkeyPatch) -> None:
"""The ``"global"`` source MUST log (no silent fallback)."""
req = _request()
matching = _capture_tenant_resolution_logs(monkeypatch)
resolve_tenant_key(req)
assert len(matching) == 1
assert matching[0].source == "global" # type: ignore[attr-defined]
assert matching[0].tenant_key == "global" # type: ignore[attr-defined]
def test_header_source_emits_structured_log(monkeypatch: pytest.MonkeyPatch) -> None:
"""Header path also emits the structured log."""
req = _request(headers={"X-Headroom-Tenant-ID": "tenant1"})
matching = _capture_tenant_resolution_logs(monkeypatch)
resolve_tenant_key(req)
assert len(matching) == 1
assert matching[0].source == "header" # type: ignore[attr-defined]
def test_bearer_without_header_emits_global_structured_log(monkeypatch: pytest.MonkeyPatch) -> None:
"""Bearer auth without a tenant header still emits the global log."""
req = _request(
headers={"authorization": "Bearer sk-ant-api03-abc123def"},
auth_mode="payg",
)
matching = _capture_tenant_resolution_logs(monkeypatch)
resolve_tenant_key(req)
assert len(matching) == 1
assert matching[0].source == "global" # type: ignore[attr-defined]
# ── Sanitization edges ────────────────────────────────────────────────
def test_sanitization_allows_alphanumeric() -> None:
"""ASCII alphanumerics survive untouched."""
req = _request(headers={"X-Headroom-Tenant-ID": "Cust42TenantA"})
tenant_key, _ = resolve_tenant_key(req)
assert tenant_key == "Cust42TenantA"
def test_sanitization_allows_hyphens_and_underscores() -> None:
"""Hyphens and underscores survive (common in SaaS tenant IDs)."""
req = _request(headers={"X-Headroom-Tenant-ID": "cust_a-b_c-9"})
tenant_key, _ = resolve_tenant_key(req)
assert tenant_key == "cust_a-b_c-9"
def test_sanitization_drops_control_chars() -> None:
"""Control chars (``\\x00`` / ``\\x1f``) get stripped."""
req = _request(headers={"X-Headroom-Tenant-ID": "ten\x00ant\x1f1"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "tenant1"
assert source == SOURCE_HEADER
def test_sanitization_drops_tab_and_newline() -> None:
"""Whitespace (``\\t`` / ``\\n``) gets stripped."""
req = _request(headers={"X-Headroom-Tenant-ID": "tenant\t\n1"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "tenant1"
assert source == SOURCE_HEADER
def test_sanitization_drops_punctuation() -> None:
"""`|`, `:`, `.`, `,`, `/` stripped (TOIN store-key separator safety)."""
req = _request(headers={"X-Headroom-Tenant-ID": "ten|ant:1.0/x"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "tenant10x"
assert source == SOURCE_HEADER
def test_sanitization_drops_unicode_falls_through_when_pure_unicode() -> None:
"""Pure-unicode header (CJK / emoji) sanitizes to empty ⇒ falls through."""
req = _request(headers={"X-Headroom-Tenant-ID": "测试テスト"})
_, source = resolve_tenant_key(req)
# No header source — sanitized to empty, dropped to global.
assert source == SOURCE_GLOBAL
def test_sanitization_truncates_at_64_chars() -> None:
"""Headers > 64 ASCII alphanum chars are TRUNCATED, not rejected."""
raw = "a" * 100
req = _request(headers={"X-Headroom-Tenant-ID": raw})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "a" * 64
assert len(tenant_key) == 64
assert source == SOURCE_HEADER
def test_sanitization_empty_header_falls_through() -> None:
"""Empty header value ⇒ fall through (header treated as absent)."""
req = _request(headers={"X-Headroom-Tenant-ID": ""})
_, source = resolve_tenant_key(req)
assert source == SOURCE_GLOBAL
def test_sanitization_whitespace_only_falls_through() -> None:
"""Whitespace-only header ⇒ sanitizes to empty ⇒ falls through."""
req = _request(headers={"X-Headroom-Tenant-ID": " \t\n "})
_, source = resolve_tenant_key(req)
assert source == SOURCE_GLOBAL
def test_sanitization_mixed_unicode_and_ascii_preserves_ascii() -> None:
"""``cust_我_42`` sanitizes to ``cust__42`` (ASCII chars survive)."""
req = _request(headers={"X-Headroom-Tenant-ID": "cust_我_42"})
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "cust__42"
assert source == SOURCE_HEADER
# ── ContextVar surface ────────────────────────────────────────────────
def test_get_current_tenant_key_defaults_to_global() -> None:
"""Outside a request context, ``get_current_tenant_key`` ⇒ ``"global"``."""
set_request_tenant_key(None) # Make sure we're cleared.
assert get_current_tenant_key() == GLOBAL_TENANT_KEY
def test_set_request_tenant_key_round_trips() -> None:
"""``set`` then ``get`` returns the value."""
set_request_tenant_key("tenant_abc")
try:
assert get_current_tenant_key() == "tenant_abc"
finally:
set_request_tenant_key(None)
def test_set_request_tenant_key_none_resets_to_global() -> None:
"""Clearing returns to the global default."""
set_request_tenant_key("tenant_abc")
set_request_tenant_key(None)
assert get_current_tenant_key() == GLOBAL_TENANT_KEY
# ── Header precedence over auth headers ───────────────────────────────
def test_header_takes_precedence_over_auth_header() -> None:
"""When both a tenant header and bearer auth are present, the tenant header wins."""
req = _request(
headers={
"X-Headroom-Tenant-ID": "cust_explicit",
"authorization": "Bearer sk-ant-api03-aaaaa",
},
auth_mode="payg",
)
tenant_key, source = resolve_tenant_key(req)
assert tenant_key == "cust_explicit"
assert source == SOURCE_HEADER
+2 -1
View File
@@ -457,7 +457,8 @@ class TestTOINExportImport:
assert len(export["patterns"]) == 1
# PR-B5: keys are now serialized "auth|model|hash" tuples; default
# auth/model produce the "unknown|unknown|<hash>" string.
assert f"unknown|unknown|{sig.structure_hash}" in export["patterns"]
# PR-F3: prepended with the tenant_key slot — default is "global".
assert f"global|unknown|unknown|{sig.structure_hash}" in export["patterns"]
def test_import_patterns_new_pattern(self):
"""import_patterns adds new patterns."""
+3 -3
View File
@@ -106,7 +106,7 @@ class TestGetRecommendationObservations:
toin.get_recommendation(sig)
# Check observations incremented
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
assert pattern.observations == 1
# Call again
@@ -133,7 +133,7 @@ class TestGetRecommendationObservations:
result = toin.get_recommendation(sig)
assert result.source == "local" # Not enough samples
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
assert pattern.observations == 1
def test_no_increment_for_unknown_pattern(self):
@@ -199,7 +199,7 @@ class TestRecordRetrievalPopulatesFields:
query_fields=["error_message"],
)
pattern = toin._patterns[("unknown", "unknown", sig_hash)]
pattern = toin._patterns[("global", "unknown", "unknown", sig_hash)]
assert pattern.total_retrievals == 5
assert pattern.search_retrievals == 5
assert len(pattern.field_retrieval_frequency) > 0
+18 -16
View File
@@ -91,7 +91,7 @@ class TestStrategySuccessRates:
)
# Set high success rate
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.strategy_success_rates["smart_sample"] = 0.8
pattern.optimal_strategy = "smart_sample"
@@ -117,7 +117,7 @@ class TestStrategySuccessRates:
)
# Set low success rate
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.strategy_success_rates["bad_strategy"] = 0.2
pattern.optimal_strategy = "bad_strategy"
@@ -146,7 +146,7 @@ class TestStrategySuccessRates:
)
# Set up multiple strategies with different success rates
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.strategy_success_rates = {
"bad_strategy": 0.2,
"good_strategy": 0.9,
@@ -179,7 +179,7 @@ class TestPreserveFieldsMerging:
compressed_tokens=500,
strategy="smart_sample",
)
local_pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
local_pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
local_pattern.preserve_fields = ["field_a", "field_b"]
# Import pattern with different preserve_fields
@@ -198,7 +198,7 @@ class TestPreserveFieldsMerging:
fresh_toin.import_patterns(import_data)
# Verify merge
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
assert "field_a" in pattern.preserve_fields
assert "field_b" in pattern.preserve_fields
assert "field_c" in pattern.preserve_fields
@@ -219,7 +219,7 @@ class TestPreserveFieldsMerging:
compressed_tokens=500,
strategy="smart_sample",
)
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
pattern.preserve_fields = [f"field_{i}" for i in range(8)]
# Import with 5 more fields
@@ -237,7 +237,7 @@ class TestPreserveFieldsMerging:
fresh_toin.import_patterns(import_data)
# Should be capped at 10
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
assert len(pattern.preserve_fields) <= 10
@@ -259,7 +259,7 @@ class TestUserCountTracking:
strategy="smart_sample",
)
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
assert pattern.user_count == 1
assert len(pattern._seen_instance_hashes) == 1
assert fresh_toin._instance_id in pattern._seen_instance_hashes
@@ -280,7 +280,7 @@ class TestUserCountTracking:
strategy="smart_sample",
)
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
assert pattern.user_count == 1 # Still 1
def test_instance_hashes_serialized_and_loaded(self):
@@ -309,7 +309,9 @@ class TestUserCountTracking:
# Load in new instance
toin2 = ToolIntelligenceNetwork(TOINConfig(storage_path=storage_path))
pattern = toin2._patterns.get(("unknown", "unknown", signature.structure_hash))
pattern = toin2._patterns.get(
("global", "unknown", "unknown", signature.structure_hash)
)
assert pattern is not None
assert pattern.user_count >= 1
assert len(pattern._seen_instance_hashes) >= 1
@@ -345,7 +347,7 @@ class TestUserCountTracking:
fresh_toin.import_patterns(import_data)
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
# Should have local + 2 imported = 3
assert pattern.user_count >= 3
@@ -374,7 +376,7 @@ class TestFieldRetrievalFrequencyWeighting:
# Record retrievals for "status" field
status_hash = fresh_toin._hash_field_name("status")
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.field_retrieval_frequency = {
status_hash: 50,
fresh_toin._hash_field_name("category"): 10,
@@ -403,7 +405,7 @@ class TestFieldRetrievalFrequencyWeighting:
strategy="smart_sample",
)
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
field_a = fresh_toin._hash_field_name("field_a")
field_b = fresh_toin._hash_field_name("field_b")
field_c = fresh_toin._hash_field_name("field_c")
@@ -450,7 +452,7 @@ class TestQueryContextUsage:
)
# Low retrieval rate = aggressive compression
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.total_retrievals = 0
# Query with exhaustive keyword
@@ -476,7 +478,7 @@ class TestQueryContextUsage:
strategy="smart_sample",
)
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.total_retrievals = 0
hint = fresh_toin.get_recommendation(signature, "find every user")
@@ -498,7 +500,7 @@ class TestQueryContextUsage:
strategy="smart_sample",
)
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
pattern.total_retrievals = 0
# Add a problematic query pattern
pattern.common_query_patterns = ["status:*"]
+193 -11
View File
@@ -4,9 +4,12 @@ Pins three guarantees:
1. `get_recommendation()` returns `None` and emits a `DeprecationWarning`
exactly once per process. The request-time hint API is retired.
2. The aggregation key is `(auth_mode, model_family, structure_hash)` —
two patterns with the same `structure_hash` but different `auth_mode`
or `model_family` are tracked as distinct rows in the TOIN store.
2. The aggregation key is `(tenant_key, auth_mode, model_family,
structure_hash)` — two patterns with the same `structure_hash` but
different `auth_mode`, `model_family`, or `tenant_key` are tracked
as distinct rows in the TOIN store. PR-F3 added `tenant_key` so two
tenants on the same proxy don't cross-pollinate compression
patterns.
3. Recording a compression event does NOT alter the bytes SmartCrusher
produces for an identical input. SmartCrusher is deterministic; TOIN
only observes.
@@ -22,6 +25,7 @@ import pytest
from headroom.telemetry import (
DEFAULT_AUTH_MODE,
DEFAULT_MODEL_FAMILY,
DEFAULT_TENANT_KEY,
TOINConfig,
ToolIntelligenceNetwork,
ToolSignature,
@@ -120,15 +124,18 @@ def test_aggregation_key_includes_auth_mode_and_model_family():
)
sig_hash = sig.structure_hash
assert ("payg", "claude-3-5", sig_hash) in toin._patterns
assert ("oauth", "claude-3-5", sig_hash) in toin._patterns
assert ("payg", "gpt-4o", sig_hash) in toin._patterns
# PR-F3: keys are 4-tuples — `(tenant_key, auth_mode, model_family,
# sig_hash)`. With no contextvar set the default is "global".
assert (DEFAULT_TENANT_KEY, "payg", "claude-3-5", sig_hash) in toin._patterns
assert (DEFAULT_TENANT_KEY, "oauth", "claude-3-5", sig_hash) in toin._patterns
assert (DEFAULT_TENANT_KEY, "payg", "gpt-4o", sig_hash) in toin._patterns
# Three distinct slices, each with sample_size=1.
assert len(toin._patterns) == 3
for key, pattern in toin._patterns.items():
assert pattern.auth_mode == key[0]
assert pattern.model_family == key[1]
assert pattern.tool_signature_hash == key[2]
assert pattern.tenant_key == key[0]
assert pattern.auth_mode == key[1]
assert pattern.model_family == key[2]
assert pattern.tool_signature_hash == key[3]
assert pattern.sample_size == 1
@@ -146,9 +153,15 @@ def test_aggregation_key_defaults_to_unknown_when_caller_omits_tenant():
strategy="smart_crusher",
)
expected_key = (DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig.structure_hash)
expected_key = (
DEFAULT_TENANT_KEY,
DEFAULT_AUTH_MODE,
DEFAULT_MODEL_FAMILY,
sig.structure_hash,
)
assert expected_key in toin._patterns
pattern = toin._patterns[expected_key]
assert pattern.tenant_key == DEFAULT_TENANT_KEY
assert pattern.auth_mode == DEFAULT_AUTH_MODE
assert pattern.model_family == DEFAULT_MODEL_FAMILY
@@ -168,16 +181,185 @@ def test_storage_round_trip_preserves_aggregation_key(tmp_path: Path):
strategy="smart_crusher",
auth_mode="oauth",
model_family="gpt-4o",
tenant_key="cust_xyz",
)
toin1.save()
toin2 = ToolIntelligenceNetwork(TOINConfig(storage_path=str(storage)))
key = ("oauth", "gpt-4o", sig.structure_hash)
key = ("cust_xyz", "oauth", "gpt-4o", sig.structure_hash)
assert key in toin2._patterns
assert toin2._patterns[key].tenant_key == "cust_xyz"
assert toin2._patterns[key].auth_mode == "oauth"
assert toin2._patterns[key].model_family == "gpt-4o"
# ── PR-F3: per-tenant isolation property ───────────────────────────────────
def test_tenants_do_not_cross_pollinate_patterns():
"""Two tenants compressing identical tool outputs see two isolated pools.
This is the load-bearing F3 invariant: tenant A's compression
patterns must be invisible to tenant B's `get_pattern` calls and
vice-versa. The threat model F3 closes is: a multi-tenant proxy
that learns a shared pattern pool would cross-pollinate
compression decisions between distinct customers, and (worse)
leak structural fingerprints across tenancy boundaries when TOIN
dumps are shared.
"""
toin = ToolIntelligenceNetwork()
sig = ToolSignature.from_items([{"id": "1", "status": "ok"}])
sig_hash = sig.structure_hash
# Tenant A: 3 compressions, no retrievals (perfect-compression
# signal — pattern.skip_compression_recommended will stay False
# but strategy_success_rate will trend high).
for _ in range(3):
toin.record_compression(
tool_signature=sig,
original_count=10,
compressed_count=2,
original_tokens=1000,
compressed_tokens=200,
strategy="smart_crusher",
tenant_key="tenant_a",
)
# Tenant B: 3 compressions PLUS 3 retrievals (over-compressed
# signal — strategy_success_rate will trend down). If isolation
# is broken, tenant A's "smart_crusher worked great" signal would
# contaminate tenant B's pattern.
for _ in range(3):
toin.record_compression(
tool_signature=sig,
original_count=10,
compressed_count=2,
original_tokens=1000,
compressed_tokens=200,
strategy="smart_crusher",
tenant_key="tenant_b",
)
for _ in range(3):
toin.record_retrieval(
tool_signature_hash=sig_hash,
retrieval_type="full",
strategy="smart_crusher",
tenant_key="tenant_b",
)
# Two distinct keys must exist, each with its own dataclass.
key_a = ("tenant_a", DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)
key_b = ("tenant_b", DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)
assert key_a in toin._patterns
assert key_b in toin._patterns
pattern_a = toin._patterns[key_a]
pattern_b = toin._patterns[key_b]
# Compression counts isolated: tenant A saw 3 comps + 0 retrievals.
assert pattern_a.total_compressions == 3
assert pattern_a.total_retrievals == 0
# Tenant B saw 3 comps + 3 retrievals — divergent learning signal.
assert pattern_b.total_compressions == 3
assert pattern_b.total_retrievals == 3
# The success rate diverges: tenant A's "smart_crusher" gets
# rewarded; tenant B's gets penalized for triggering retrievals.
rate_a = pattern_a.strategy_success_rates.get("smart_crusher", 0.0)
rate_b = pattern_b.strategy_success_rates.get("smart_crusher", 0.0)
assert rate_a > rate_b, (
f"PR-F3: tenant A and B success rates must diverge (tenant A: {rate_a}, tenant B: {rate_b})"
)
# `get_pattern` honors tenant_key — tenant A's lookup returns
# tenant A's pattern, not tenant B's.
fetched_a = toin.get_pattern(sig_hash, tenant_key="tenant_a")
fetched_b = toin.get_pattern(sig_hash, tenant_key="tenant_b")
assert fetched_a is not None
assert fetched_b is not None
assert fetched_a.total_retrievals == 0
assert fetched_b.total_retrievals == 3
def test_tenant_key_default_is_global_when_no_caller_provides_one():
"""Pre-F3 callers and CLI / batch jobs land in the literal "global" slice.
Pins the migration story: when no tenant_key is supplied (and no
contextvar is set), patterns aggregate under the literal
``"global"`` namespace, which is exactly where pre-F3 patterns
live after the legacy 3-tuple → 4-tuple promotion.
"""
toin = ToolIntelligenceNetwork()
sig = ToolSignature.from_items([{"id": "1"}])
toin.record_compression(
tool_signature=sig,
original_count=10,
compressed_count=5,
original_tokens=1000,
compressed_tokens=500,
strategy="smart_crusher",
)
keys = list(toin._patterns.keys())
assert len(keys) == 1
assert keys[0][0] == DEFAULT_TENANT_KEY == "global"
def test_tenant_key_reads_from_contextvar_when_unspecified():
"""When caller doesn't pass tenant_key, TOIN reads the request ContextVar."""
from headroom.proxy.tenant_key import set_request_tenant_key
toin = ToolIntelligenceNetwork()
sig = ToolSignature.from_items([{"id": "1"}])
set_request_tenant_key("tenant_from_ctx")
try:
toin.record_compression(
tool_signature=sig,
original_count=10,
compressed_count=5,
original_tokens=1000,
compressed_tokens=500,
strategy="smart_crusher",
)
finally:
set_request_tenant_key(None)
# The contextvar's value made it into the key — not the default
# "global". This is the actual integration path for the proxy.
keys = list(toin._patterns.keys())
assert len(keys) == 1
assert keys[0][0] == "tenant_from_ctx"
def test_explicit_tenant_key_overrides_contextvar():
"""An explicit tenant_key arg always wins over the ContextVar."""
from headroom.proxy.tenant_key import set_request_tenant_key
toin = ToolIntelligenceNetwork()
sig = ToolSignature.from_items([{"id": "1"}])
set_request_tenant_key("from_contextvar")
try:
toin.record_compression(
tool_signature=sig,
original_count=10,
compressed_count=5,
original_tokens=1000,
compressed_tokens=500,
strategy="smart_crusher",
tenant_key="explicit_arg",
)
finally:
set_request_tenant_key(None)
keys = list(toin._patterns.keys())
assert len(keys) == 1
assert keys[0][0] == "explicit_arg"
def test_record_does_not_alter_compression_decision():
"""SmartCrusher output is byte-identical regardless of TOIN observation state.
+3
View File
@@ -97,6 +97,7 @@ def test_publish_command_writes_toml(fresh_toin: ToolIntelligenceNetwork, tmp_pa
row = rec_list[0]
assert set(row.keys()) == {
"tenant_key",
"auth_mode",
"model_family",
"structure_hash",
@@ -105,6 +106,8 @@ def test_publish_command_writes_toml(fresh_toin: ToolIntelligenceNetwork, tmp_pa
"confidence",
"observations",
}
# PR-F3: tenant_key defaults to "global" when no explicit one is set.
assert row["tenant_key"] == "global"
assert row["auth_mode"] == "payg"
assert row["model_family"] == "claude-3-5"
assert row["structure_hash"] == sig.structure_hash