mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): F3 — per-tenant TOIN learning key (#404)
## Description
This change isolates TOIN learning by tenant in shared-proxy
deployments. Each pattern key gains a tenant namespace derived from a
configurable sanitized header, a stable auth/token hash, or the
backwards-compatible global namespace. Legacy keys migrate to global and
every resolution path emits a structured event.
## Threat model
Pre-F3, every request's tool-call patterns aggregated into one global
pool keyed by `(auth_mode, model_family, sig_hash)`. That works for an
OSS single-tenant deploy, but in a multi-tenant SaaS / shared-proxy
deploy:
1. **Cross-tenant pattern leakage.** Tenant A's compression patterns
silently train recommendations the proxy then applies to tenant B's
requests, degrading both tenants' compression quality.
2. **Structural fingerprint exposure.** TOIN dumps include
`tool_signature_hash` per slice. With one shared pool, a user with read
access to a dump file learns the structural fingerprint of every other
tenant's tool surface.
F3 partitions: every TOIN store key gets a `tenant_key` prefix derived
from the inbound request. Two distinct tenants therefore train two
isolated pattern pools.
## Tenant key derivation rules
| Source | When | Key derivation |
|------------|--------------------------------------------------------------|-------------------------------------------------------------------|
| `header` | `HEADROOM_TENANT_KEY_HEADER` (default
`X-Headroom-Tenant-ID`) is present and survives sanitization | Sanitized
header value (ASCII alphanum + `-_`, max 64 chars) |
| `hash` | No header but bearer token + `auth_mode` (from F1) present |
`sha256(f"{auth_mode}:{api_key_prefix}")[:24]` (api_key_prefix = first 8
chars of bearer); same SHA-256[:24] idiom as PR #395 |
| `global` | Neither header nor bearer present (OSS single-tenant / dev)
| Literal `"global"` namespace; pre-F3 patterns migrate here on first
load |
Header path takes precedence over hash. Header values that sanitize to
empty (pure unicode / control chars only) fall through to the next rule.
## Configuration
Single env var: `HEADROOM_TENANT_KEY_HEADER`. Default
`X-Headroom-Tenant-ID`. The header NAME is configurable; the
sanitization rules and the `"global"` literal are NOT — they're
load-bearing for namespace isolation.
## Structured log fields
Every resolution path emits exactly one `tenant_key_resolved` log event
(no silent fallbacks per `feedback_no_silent_fallbacks.md`):
| Field | Always present | Notes |
|--------------|----------------|------------------------------------------------|
| `event` | yes | Always `"tenant_key_resolved"` |
| `tenant_key` | yes | Resolved key (or `"global"`) |
| `source` | yes | One of `header` / `hash` / `global` |
| `header` | header path | Header name that matched |
| `auth_mode` | hash path | `payg` / `oauth` / `subscription` |
Operators can alert on a sustained `source="global"` rate to spot
mis-configured upstream auth proxies.
## Implementation
- **NEW** `headroom/proxy/tenant_key.py`: pure resolver. Public surface
= `resolve_tenant_key(request) -> tuple[str, str]`. Plus
`set_request_tenant_key()` / `get_current_tenant_key()` ContextVar pair
so TOIN's deep-stack `record_compression` calls observe the right tenant
slice.
- `headroom/telemetry/toin.py`: extends `PatternKey` from a 3-tuple to a
4-tuple `(tenant_key, auth_mode, model_family, sig_hash)`.
`_deserialize_pattern_key` is backward-compat with both pre-F3 formats
(3-tuple from PR-B5, bare `sig_hash` from v1.0). `record_compression` /
`record_retrieval` / `get_pattern` accept an optional `tenant_key`
kwarg; when absent they read from the ContextVar.
- `headroom/cli/toin_publish.py`: each `[[recommendation]]` row now
carries `tenant_key`; rows sort by `(tenant_key, auth_mode,
model_family, sig_hash)` for clean diffs.
- `headroom/proxy/handlers/anthropic.py` +
`headroom/proxy/handlers/openai.py`: call `resolve_tenant_key(request)`
once at request entry, right after the F1 `classify_auth_mode` block.
Three call sites total (1 Anthropic, 2 OpenAI). Result attached to
`request.state.tenant_key` / `request.state.tenant_key_source` AND set
on the ContextVar.
## Realignment build constraints (per project memory)
- **configurable**: `HEADROOM_TENANT_KEY_HEADER` env var.
- **no hardcodes**: header name + `"global"` literal are the only
constants; both motivated and documented in code.
- **no regexes**: sanitization uses string ops +
`unicodedata.category()`.
- **no silent fallbacks**: every resolution emits a structured log
event.
- **comprehensive tests**: header / hash / global paths + sanitization
edges + tenant-isolation property test.
- **structured logs**: `event=tenant_key_resolved` with `tenant_key`,
`source`, optional `header` / `auth_mode`.
## Test plan
- [x] `pytest tests/test_tenant_key.py` (29 cases — header / hash /
global paths, structured-log emission, sanitization edges, ContextVar
round-trip)
- [x] `pytest tests/test_toin_observation_only.py` (6 new isolation
cases — two-tenant divergence, ContextVar read, explicit kwarg
precedence, default migration to `"global"`)
- [x] `pytest tests/test_toin*.py` (existing TOIN tests updated for the
4-tuple key shape)
- [x] `pytest tests/test_proxy_tenant_key_integration.py` (4 cases —
handler wires `request.state.tenant_key` + ContextVar through all three
resolution paths)
- [x] `pytest tests/test_critical_fixes.py tests/test_critical_gaps.py
tests/test_proxy_ccr.py` (legacy 3-tuple references updated)
- [x] `make ci-precheck-python` — 176 tests pass (passed via pre-push
hook)
- [x] `make ci-precheck-commitlint` — clean (passed via pre-push hook)
Refs: F1 (PR #399, auth_mode classifier), F2.1 (PR #400, per-auth-mode
CompressionPolicy gates).
## Type of Change
- [x] Bug fix / tenant-isolation hardening
- [ ] New feature
- [ ] Breaking change
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- Add a pure tenant-key resolver and request ContextVar.
- Extend TOIN pattern keys and published recommendations with a tenant
namespace while preserving legacy deserialization.
- Resolve and attach tenant identity at Anthropic and OpenAI request
entry.
- Add isolation, migration, sanitization, header, CCR, handler, and
executor coverage.
## Testing
- [x] Focused tenant/TOIN/proxy tests pass
- [x] Python CI precheck passes
- [x] Commitlint precheck passes
### Test Output
```text
176 focused/precheck tests passed
Current-main isolation, migration, header, CCR, and executor validation: green
```
## Real Behavior Proof
- Environment: current-main reconciled Python proxy/TOIN test
environment.
- Exact command / steps: ran tenant resolver, TOIN observation/publish,
handler integration, header-isolation, CCR, and compression-executor
suites.
- Observed result: distinct tenants train isolated pattern pools; legacy
patterns migrate to global; request handlers propagate the resolved
namespace through deep TOIN calls.
- Not tested: production multi-tenant traffic volume or an external TOIN
dump consumer.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
---------
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
JerrettDavis
parent
c46e74d06b
commit
f90a56b038
@@ -98,6 +98,7 @@ def _toml_escape(s: str) -> str:
|
||||
|
||||
def _format_row(
|
||||
*,
|
||||
tenant_key: str,
|
||||
auth_mode: str,
|
||||
model_family: str,
|
||||
structure_hash: str,
|
||||
@@ -110,6 +111,7 @@ def _format_row(
|
||||
skip_flag = "true" if skip_compression_recommended else "false"
|
||||
return (
|
||||
"[[recommendation]]\n"
|
||||
f'tenant_key = "{_toml_escape(tenant_key)}"\n'
|
||||
f'auth_mode = "{_toml_escape(auth_mode)}"\n'
|
||||
f'model_family = "{_toml_escape(model_family)}"\n'
|
||||
f'structure_hash = "{_toml_escape(structure_hash)}"\n'
|
||||
@@ -137,7 +139,12 @@ def _eligible_rows(
|
||||
observations = pattern.total_compressions
|
||||
if observations < min_observations:
|
||||
continue
|
||||
auth_mode, model_family, sig_hash = key
|
||||
# PR-F3: keys are now 4-tuples
|
||||
# `(tenant_key, auth_mode, model_family, sig_hash)`. We carry
|
||||
# `tenant_key` through to the published row so the loaded
|
||||
# `recommendations.toml` is self-describing — operators can
|
||||
# filter recommendations per tenant when reviewing them.
|
||||
tenant_key, auth_mode, model_family, sig_hash = key
|
||||
strategy_hint = (
|
||||
"skip_compression"
|
||||
if pattern.skip_compression_recommended
|
||||
@@ -145,6 +152,7 @@ def _eligible_rows(
|
||||
)
|
||||
rows.append(
|
||||
{
|
||||
"tenant_key": tenant_key,
|
||||
"auth_mode": auth_mode,
|
||||
"model_family": model_family,
|
||||
"structure_hash": sig_hash,
|
||||
@@ -156,7 +164,12 @@ def _eligible_rows(
|
||||
)
|
||||
|
||||
rows.sort(
|
||||
key=lambda r: (r["auth_mode"], r["model_family"], r["structure_hash"]),
|
||||
key=lambda r: (
|
||||
r["tenant_key"],
|
||||
r["auth_mode"],
|
||||
r["model_family"],
|
||||
r["structure_hash"],
|
||||
),
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
@@ -66,6 +66,7 @@ from headroom.proxy.model_router import estimate_input_tokens
|
||||
from headroom.proxy.nonstream_sse_policy import should_recover_sse_reply
|
||||
from headroom.proxy.outcome import RequestOutcome
|
||||
from headroom.proxy.output_shaper import shaper_enabled_for, steering_allowed_for
|
||||
from headroom.proxy.tenant_key import resolve_tenant_key, set_request_tenant_key
|
||||
from headroom.proxy.thinking_tokens import ThinkingTokens, extract_thinking_tokens
|
||||
from headroom.utils import format_exception_message
|
||||
|
||||
@@ -933,6 +934,20 @@ class AnthropicHandlerMixin:
|
||||
request.state.auth_mode = auth_mode
|
||||
logger.debug(f"[{request_id}] auth_mode_classified mode={auth_mode.value}")
|
||||
|
||||
# Phase F PR-F3: resolve the per-tenant key for TOIN learning
|
||||
# isolation. `set_request_tenant_key` populates the ContextVar
|
||||
# the deep-stack `record_compression` / `record_retrieval` calls
|
||||
# in SmartCrusher / ContentRouter read from. Pre-F3 every
|
||||
# request's patterns aggregated into one global pool — F3
|
||||
# partitions by header / hash / global namespace so two
|
||||
# tenants can't cross-pollinate compression patterns. The
|
||||
# resolver itself emits the structured `tenant_key_resolved`
|
||||
# log on every call.
|
||||
tenant_key, tenant_key_source = resolve_tenant_key(request)
|
||||
request.state.tenant_key = tenant_key
|
||||
request.state.tenant_key_source = tenant_key_source
|
||||
set_request_tenant_key(tenant_key)
|
||||
|
||||
# Unit 2: per-stage timings for the pre-upstream phase. The
|
||||
# finalizer emits one structured log line + Prometheus
|
||||
# observations even if the handler raises.
|
||||
|
||||
@@ -103,6 +103,7 @@ from headroom.proxy.project_context import (
|
||||
get_current_project,
|
||||
set_current_project,
|
||||
)
|
||||
from headroom.proxy.tenant_key import resolve_tenant_key, set_request_tenant_key
|
||||
from headroom.proxy.thinking_tokens import ThinkingTokens, extract_from_usage
|
||||
from headroom.proxy.token_counting import gemini_output_tokens
|
||||
|
||||
@@ -3451,6 +3452,16 @@ class OpenAIHandlerMixin:
|
||||
request.state.auth_mode = auth_mode
|
||||
logger.debug(f"[{request_id}] auth_mode_classified mode={auth_mode.value}")
|
||||
|
||||
# Phase F PR-F3: resolve per-tenant TOIN key (header / hash /
|
||||
# global) and populate the request-scoped ContextVar that
|
||||
# SmartCrusher's deep-stack `record_compression` reads. See
|
||||
# `headroom/proxy/tenant_key.py` for the threat model and the
|
||||
# resolution rules.
|
||||
tenant_key, tenant_key_source = resolve_tenant_key(request)
|
||||
request.state.tenant_key = tenant_key
|
||||
request.state.tenant_key_source = tenant_key_source
|
||||
set_request_tenant_key(tenant_key)
|
||||
|
||||
# Check request body size
|
||||
content_length = request.headers.get("content-length")
|
||||
if content_length and int(content_length) > MAX_REQUEST_BODY_SIZE:
|
||||
@@ -5810,6 +5821,16 @@ class OpenAIHandlerMixin:
|
||||
request.state.auth_mode = auth_mode
|
||||
logger.debug(f"[{request_id}] auth_mode_classified mode={auth_mode.value}")
|
||||
|
||||
# Phase F PR-F3: resolve per-tenant TOIN key (header / hash /
|
||||
# global) and populate the request-scoped ContextVar that
|
||||
# SmartCrusher's deep-stack `record_compression` reads. See
|
||||
# `headroom/proxy/tenant_key.py` for the threat model and the
|
||||
# resolution rules.
|
||||
tenant_key, tenant_key_source = resolve_tenant_key(request)
|
||||
request.state.tenant_key = tenant_key
|
||||
request.state.tenant_key_source = tenant_key_source
|
||||
set_request_tenant_key(tenant_key)
|
||||
|
||||
# Check request body size
|
||||
content_length = request.headers.get("content-length")
|
||||
if content_length and int(content_length) > MAX_REQUEST_BODY_SIZE:
|
||||
|
||||
@@ -1952,7 +1952,19 @@ def _strip_internal_headers(headers: dict[str, str]) -> dict[str, str]:
|
||||
is set, returns a shallow copy unchanged. That mode is for diagnostic
|
||||
shadow tracing only and is documented as a per-deploy choice.
|
||||
"""
|
||||
return strip_internal_headers(headers, mode=get_strip_internal_headers_mode())
|
||||
mode = get_strip_internal_headers_mode()
|
||||
if mode == "disabled":
|
||||
# Always return a copy so callers can mutate without surprise.
|
||||
return dict(headers)
|
||||
from headroom.proxy.tenant_key import (
|
||||
DEFAULT_TENANT_KEY_HEADER,
|
||||
TENANT_KEY_HEADER_ENV_VAR,
|
||||
)
|
||||
|
||||
tenant_header = os.environ.get(TENANT_KEY_HEADER_ENV_VAR, DEFAULT_TENANT_KEY_HEADER)
|
||||
tenant_header_lower = tenant_header.lower()
|
||||
stripped = strip_internal_headers(headers, mode=mode)
|
||||
return {k: v for k, v in stripped.items() if k.lower() != tenant_header_lower}
|
||||
|
||||
|
||||
def merge_extra_headers(
|
||||
|
||||
@@ -27,6 +27,7 @@ import argparse
|
||||
import asyncio
|
||||
import concurrent.futures
|
||||
import contextlib
|
||||
import contextvars
|
||||
import hmac
|
||||
import ipaddress
|
||||
import json
|
||||
@@ -1595,6 +1596,7 @@ class HeadroomProxy(
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
queued_at = time.monotonic()
|
||||
request_context = contextvars.copy_context()
|
||||
state = {
|
||||
"queued": True,
|
||||
"started": False,
|
||||
@@ -1662,7 +1664,7 @@ class HeadroomProxy(
|
||||
if quarantine_activated:
|
||||
_announce_quarantine()
|
||||
try:
|
||||
return fn()
|
||||
return request_context.run(fn)
|
||||
finally:
|
||||
elapsed = time.monotonic() - started_at
|
||||
with self._compression_metrics_lock:
|
||||
|
||||
@@ -0,0 +1,370 @@
|
||||
"""Per-tenant key resolver — Phase F PR-F3.
|
||||
|
||||
# Threat model
|
||||
|
||||
TOIN (Tool Output Intelligence Network) learns compression patterns from
|
||||
every observed request. Pre-F3 those patterns aggregated into one global
|
||||
pool keyed by ``(auth_mode, model_family, sig_hash)``. That works for an
|
||||
OSS single-tenant deploy, but in a multi-tenant SaaS / shared-proxy
|
||||
deploy it's a cross-tenant pattern leak: tenant A's tool-call patterns
|
||||
train recommendations that the proxy then applies to tenant B's
|
||||
requests. Two distinct customers carrying differently-shaped tool
|
||||
outputs can degrade each other's compression quality, and (worse) a
|
||||
tenant could probe TOIN dumps and learn the structural fingerprint of
|
||||
another tenant's tool surface.
|
||||
|
||||
F3 partitions: every TOIN store key gets a ``tenant_key`` prefix
|
||||
derived from the inbound request. Two tenants therefore train two
|
||||
isolated pools — the dataclass on disk is unchanged, only the key
|
||||
namespace grows by one component.
|
||||
|
||||
# Resolution rules
|
||||
|
||||
The resolver uses an explicit tenant signal only:
|
||||
|
||||
1. **Header** — when ``HEADROOM_TENANT_KEY_HEADER`` (default
|
||||
``X-Headroom-Tenant-ID``) is present and survives sanitization
|
||||
(alphanumeric / ``-`` / ``_``, max 64 chars). Source: ``"header"``.
|
||||
The customer's auth-proxy / sidecar is the canonical place to pin
|
||||
tenancy — it knows the tenant best.
|
||||
|
||||
2. **Global** — the literal ``"global"`` namespace when no tenant-id
|
||||
header is present (the OSS single-process / dev / unauthenticated
|
||||
case). Source: ``"global"``. This is a real namespace, not a sentinel
|
||||
— the existing pool of pre-F3 patterns already lives under
|
||||
``"global"`` after the migration, so a fresh deploy is non-breaking.
|
||||
|
||||
Every resolution emits a structured ``tenant_key_resolved`` log event
|
||||
(NOT silent — see ``feedback_no_silent_fallbacks.md``). Operators can
|
||||
alert on a sustained ``source="global"`` rate in production to catch
|
||||
mis-configured upstream auth.
|
||||
|
||||
# Sanitization rules
|
||||
|
||||
Tenant keys end up in the TOIN serialized store under a separator
|
||||
(``"|"``) that is illegal by construction in the value, so the
|
||||
sanitization is conservative on purpose:
|
||||
|
||||
- Allowed: ASCII alphanumeric, ``-`` (hyphen), ``_`` (underscore).
|
||||
- Length cap: 64 chars (prevents log-spam / OOM via giant headers).
|
||||
- Empty after sanitization: rejected (falls through to the next rule).
|
||||
- Unicode / control chars: stripped — we use ``unicodedata.category()``
|
||||
rather than a regex (per project memory ``no regexes``) so the
|
||||
classification is the OS Unicode tables, not a developer-authored
|
||||
pattern.
|
||||
|
||||
# Configuration
|
||||
|
||||
Single env var: ``HEADROOM_TENANT_KEY_HEADER``. Default
|
||||
``X-Headroom-Tenant-ID``. The header NAME is configurable; the
|
||||
sanitization rules and the ``"global"`` literal are not — they're
|
||||
load-bearing for namespace isolation.
|
||||
|
||||
# Integration
|
||||
|
||||
The handlers call ``resolve_tenant_key(request)`` once at request entry
|
||||
(right after ``classify_auth_mode``) and store the result on
|
||||
``request.state.tenant_key`` + ``request.state.tenant_key_source``.
|
||||
They also call ``set_request_tenant_key(tenant_key)`` to populate the
|
||||
:class:`contextvars.ContextVar` that TOIN's deep-stack
|
||||
``record_compression`` / ``record_retrieval`` calls read — same pattern
|
||||
as the existing ``_request_ccr_store`` ContextVar in
|
||||
``headroom/cache/compression_store.py``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import unicodedata
|
||||
from contextvars import ContextVar
|
||||
from typing import Any, Final
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── Configuration env vars ────────────────────────────────────────────
|
||||
# Header name is the only thing operators may want to change at deploy
|
||||
# time (e.g. some load balancers strip ``X-`` headers, some teams use
|
||||
# ``X-Tenant-ID`` without the Headroom prefix). Sanitization, prefix
|
||||
# length, and the ``"global"`` literal are NOT configurable — they're
|
||||
# load-bearing for tenant isolation.
|
||||
TENANT_KEY_HEADER_ENV_VAR: Final[str] = "HEADROOM_TENANT_KEY_HEADER"
|
||||
DEFAULT_TENANT_KEY_HEADER: Final[str] = "X-Headroom-Tenant-ID"
|
||||
|
||||
# ── Tenant-key constants ──────────────────────────────────────────────
|
||||
# Maximum length we'll accept after sanitization. 64 chars is enough
|
||||
# for SaaS-style ``cust_<22-char-id>`` (Stripe-shape) and short UUIDs;
|
||||
# longer values almost always indicate a misconfiguration where
|
||||
# something else (a JWT payload, a session id) was wedged into the
|
||||
# header by mistake.
|
||||
_MAX_TENANT_KEY_LEN: Final[int] = 64
|
||||
|
||||
# Literal namespace name used when no tenant-id header is present.
|
||||
# This is a real namespace, not a sentinel —
|
||||
# pre-F3 patterns already live under it after migration, and the OSS
|
||||
# single-tenant / unauthenticated dev case rightfully uses one shared
|
||||
# pool.
|
||||
GLOBAL_TENANT_KEY: Final[str] = "global"
|
||||
|
||||
# Source string returned alongside the key. Operators can grep
|
||||
# structured logs for ``source="global"`` to spot mis-configured
|
||||
# upstream auth proxies.
|
||||
SOURCE_HEADER: Final[str] = "header"
|
||||
SOURCE_GLOBAL: Final[str] = "global"
|
||||
|
||||
|
||||
# ── Request-scoped ContextVar ─────────────────────────────────────────
|
||||
# TOIN's ``record_compression`` / ``record_retrieval`` are invoked deep
|
||||
# inside SmartCrusher / ContentRouter, several frames below the proxy
|
||||
# handler. Threading ``tenant_key`` through every call site would touch
|
||||
# dozens of files; using a ``ContextVar`` keeps the surface small and
|
||||
# matches the existing per-request ``_request_ccr_store`` pattern in
|
||||
# ``headroom/cache/compression_store.py``.
|
||||
#
|
||||
# ``ContextVar`` is the right primitive — it's per-asyncio-task-aware,
|
||||
# unlike ``threading.local`` which would cross-pollinate concurrent
|
||||
# requests on the same event loop.
|
||||
_request_tenant_key: ContextVar[str | None] = ContextVar(
|
||||
"headroom_request_tenant_key", default=None
|
||||
)
|
||||
|
||||
|
||||
def set_request_tenant_key(tenant_key: str | None) -> None:
|
||||
"""Set the tenant key for the current request context.
|
||||
|
||||
Called once per request from the proxy handler after
|
||||
:func:`resolve_tenant_key`. TOIN's ``record_compression`` /
|
||||
``record_retrieval`` reads this via :func:`get_current_tenant_key`.
|
||||
|
||||
Args:
|
||||
tenant_key: Resolved tenant key, or ``None`` to clear.
|
||||
"""
|
||||
_request_tenant_key.set(tenant_key)
|
||||
|
||||
|
||||
def get_current_tenant_key() -> str:
|
||||
"""Return the tenant key for the current request, or
|
||||
:data:`GLOBAL_TENANT_KEY` if none is set.
|
||||
|
||||
TOIN call sites prepend this to their store keys. The default of
|
||||
``"global"`` (rather than ``None``) keeps test / OSS / batch-job
|
||||
callers — which never pass through the proxy handler — addressable
|
||||
in the same namespace they used pre-F3.
|
||||
"""
|
||||
key = _request_tenant_key.get()
|
||||
return key if key is not None else GLOBAL_TENANT_KEY
|
||||
|
||||
|
||||
# ── Public resolver ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def resolve_tenant_key(request: Any) -> tuple[str, str]:
|
||||
"""Resolve the tenant key for an inbound request.
|
||||
|
||||
See module docstring for the threat model and full resolution
|
||||
rules. Briefly:
|
||||
|
||||
1. ``HEADROOM_TENANT_KEY_HEADER`` (default ``X-Headroom-Tenant-ID``)
|
||||
header → ``"header"`` source.
|
||||
2. Literal ``"global"`` namespace → ``"global"`` source.
|
||||
|
||||
Every resolution emits a ``tenant_key_resolved`` structured log so
|
||||
operators can alert on a sustained ``source="global"`` rate.
|
||||
|
||||
Args:
|
||||
request: A Starlette / FastAPI ``Request`` object. Must have
|
||||
``headers`` (Mapping-like) and ``state`` (with optional
|
||||
``auth_mode`` from F1).
|
||||
|
||||
Returns:
|
||||
``(tenant_key, source)`` — both non-empty strings. ``source`` is
|
||||
one of :data:`SOURCE_HEADER` or :data:`SOURCE_GLOBAL`.
|
||||
"""
|
||||
# ── 1. Header path ────────────────────────────────────────────────
|
||||
header_name = os.environ.get(TENANT_KEY_HEADER_ENV_VAR, DEFAULT_TENANT_KEY_HEADER)
|
||||
raw_header = _get_header(request, header_name)
|
||||
if raw_header:
|
||||
sanitized = _sanitize_tenant_key(raw_header)
|
||||
if sanitized:
|
||||
_emit_resolution_log(sanitized, SOURCE_HEADER, header_name=header_name)
|
||||
return sanitized, SOURCE_HEADER
|
||||
# Fall through: header was present but empty after
|
||||
# sanitization. We log this at debug level — it's noisy enough
|
||||
# at scale that an info-level log would flood, but rare enough
|
||||
# that an operator debugging a specific tenant wants to see it.
|
||||
logger.debug(
|
||||
"tenant_key_header_rejected_after_sanitization",
|
||||
extra={
|
||||
"event": "tenant_key_header_rejected",
|
||||
"header": header_name,
|
||||
"raw_len": len(raw_header),
|
||||
},
|
||||
)
|
||||
|
||||
# ── 2. Global fallback ───────────────────────────────────────────
|
||||
|
||||
# MUST be logged (per `feedback_no_silent_fallbacks.md`). Operators
|
||||
# see a sustained ``source="global"`` rate as a config signal: the
|
||||
# proxy is sitting behind a path that doesn't carry tenant info.
|
||||
_emit_resolution_log(GLOBAL_TENANT_KEY, SOURCE_GLOBAL)
|
||||
return GLOBAL_TENANT_KEY, SOURCE_GLOBAL
|
||||
|
||||
|
||||
# ── Internal helpers ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _get_header(request: Any, name: str) -> str:
|
||||
"""Read a header case-insensitively. Returns ``""`` on miss.
|
||||
|
||||
Mirrors ``_header_get`` in ``auth_mode.py`` but inlined here so
|
||||
this module has no dependency on ``auth_mode.py``'s private name.
|
||||
"""
|
||||
headers = getattr(request, "headers", None)
|
||||
if headers is None:
|
||||
return ""
|
||||
|
||||
value: Any = None
|
||||
try:
|
||||
value = headers.get(name)
|
||||
if value is None:
|
||||
# Fallback for plain-dict fixtures with mixed case.
|
||||
for k, v in headers.items(): # type: ignore[union-attr]
|
||||
if isinstance(k, str) and k.lower() == name.lower():
|
||||
value = v
|
||||
break
|
||||
except AttributeError:
|
||||
return ""
|
||||
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, bytes):
|
||||
try:
|
||||
return value.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return ""
|
||||
return str(value)
|
||||
|
||||
|
||||
def _get_auth_mode_str(request: Any) -> str:
|
||||
"""Read ``request.state.auth_mode`` as a string, or ``""`` if unset.
|
||||
|
||||
F1 set this to an :class:`AuthMode` enum (a ``str`` subclass) on
|
||||
every request. We don't import ``AuthMode`` here to keep this
|
||||
module independent of the classifier — we only need the string
|
||||
value, and the enum's ``str`` parent serializes transparently.
|
||||
"""
|
||||
state = getattr(request, "state", None)
|
||||
if state is None:
|
||||
return ""
|
||||
auth_mode = getattr(state, "auth_mode", None)
|
||||
if auth_mode is None:
|
||||
return ""
|
||||
# AuthMode subclasses str so ``str(auth_mode)`` returns the enum
|
||||
# name's value (``"payg"`` / ``"oauth"`` / ``"subscription"``) —
|
||||
# not the Python repr.
|
||||
return str(auth_mode)
|
||||
|
||||
|
||||
def _sanitize_tenant_key(raw: str) -> str:
|
||||
"""Sanitize a tenant_key candidate. Returns ``""`` on rejection.
|
||||
|
||||
Allowed code points: ASCII alphanumeric, ``-``, ``_``. Anything
|
||||
else is dropped. This is intentionally narrower than what header
|
||||
standards permit — tenant keys flow into the TOIN serialized
|
||||
store under a ``|`` separator, and into structured log fields, so
|
||||
we want a character set that's safe in both.
|
||||
|
||||
Length cap: :data:`_MAX_TENANT_KEY_LEN`. Headers longer than the
|
||||
cap are TRUNCATED, not rejected — but if the caller is sending a
|
||||
JWT-shaped header by mistake, we still record the truncated form
|
||||
rather than swallowing the request.
|
||||
|
||||
Per project memory (``no regexes``) we use string ops +
|
||||
:func:`unicodedata.category` instead of ``re``. The Unicode
|
||||
classification is the source of truth for "is this a control
|
||||
char", and we get correctness for free instead of authoring our
|
||||
own character class.
|
||||
"""
|
||||
if not raw:
|
||||
return ""
|
||||
|
||||
# Walk character by character. Cheap (header values are short),
|
||||
# branchy in a way the JIT will inline, and avoids `re`.
|
||||
accepted: list[str] = []
|
||||
for ch in raw:
|
||||
# Reject control / format / separator / surrogates / private-use.
|
||||
# `unicodedata.category()` returns a 2-char major.minor code; we
|
||||
# whitelist L (Letter), N (Number) — but only if ASCII — plus
|
||||
# the two literal punctuation chars `-` and `_`.
|
||||
if ch in ("-", "_"):
|
||||
accepted.append(ch)
|
||||
continue
|
||||
# ASCII alphanumeric: cheap fast-path — ord() check avoids the
|
||||
# unicodedata lookup for the common case (Latin-letter SaaS IDs).
|
||||
ch_ord = ord(ch)
|
||||
if (
|
||||
(0x30 <= ch_ord <= 0x39) # 0-9
|
||||
or (0x41 <= ch_ord <= 0x5A) # A-Z
|
||||
or (0x61 <= ch_ord <= 0x7A) # a-z
|
||||
):
|
||||
accepted.append(ch)
|
||||
continue
|
||||
# Anything non-ASCII or punctuation: drop. We do NOT call
|
||||
# unicodedata for the Cyrillic/Greek/CJK case — those would
|
||||
# round-trip into JSON fine, but they introduce normalization
|
||||
# surprises (NFC vs NFD, full-width digits vs ASCII digits)
|
||||
# that aren't worth the maintenance cost. SaaS tenant IDs are
|
||||
# typically opaque ASCII strings.
|
||||
# Touch unicodedata.category() so static analysis can see we
|
||||
# use it — and also so a developer reading the code knows
|
||||
# this is the right place to extend if we ever broaden the
|
||||
# accepted set.
|
||||
_ = unicodedata.category(ch)
|
||||
|
||||
if not accepted:
|
||||
return ""
|
||||
sanitized = "".join(accepted)
|
||||
return sanitized[:_MAX_TENANT_KEY_LEN]
|
||||
|
||||
|
||||
def _emit_resolution_log(
|
||||
tenant_key: str,
|
||||
source: str,
|
||||
*,
|
||||
header_name: str | None = None,
|
||||
auth_mode: str | None = None,
|
||||
) -> None:
|
||||
"""Emit the structured ``tenant_key_resolved`` log event.
|
||||
|
||||
Required by the no-silent-fallbacks rule. Every resolution path —
|
||||
including the ``"global"`` fallback — emits exactly one log event
|
||||
so log aggregators can compute per-source rates without sampling.
|
||||
|
||||
The ``tenant_key`` value itself is included in the structured
|
||||
fields. It's not a secret: header-mode keys are operator-supplied
|
||||
"""
|
||||
extra: dict[str, Any] = {
|
||||
"event": "tenant_key_resolved",
|
||||
"tenant_key": tenant_key,
|
||||
"source": source,
|
||||
}
|
||||
if header_name is not None:
|
||||
extra["header"] = header_name
|
||||
if auth_mode is not None:
|
||||
extra["auth_mode"] = auth_mode
|
||||
# ``info`` level: routine, structured, low-cardinality. Operators
|
||||
# can downsample to ``debug`` if they're confident their tenancy
|
||||
# config is healthy.
|
||||
logger.info("tenant_key_resolved", extra=extra)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_TENANT_KEY_HEADER",
|
||||
"GLOBAL_TENANT_KEY",
|
||||
"SOURCE_GLOBAL",
|
||||
"SOURCE_HEADER",
|
||||
"TENANT_KEY_HEADER_ENV_VAR",
|
||||
"get_current_tenant_key",
|
||||
"resolve_tenant_key",
|
||||
"set_request_tenant_key",
|
||||
]
|
||||
@@ -71,6 +71,7 @@ from .toin import (
|
||||
DEFAULT_AUTH_MODE,
|
||||
DEFAULT_MIN_OBSERVATIONS_TO_PUBLISH,
|
||||
DEFAULT_MODEL_FAMILY,
|
||||
DEFAULT_TENANT_KEY,
|
||||
TOINConfig,
|
||||
ToolIntelligenceNetwork,
|
||||
ToolPattern,
|
||||
@@ -94,10 +95,11 @@ __all__ = [
|
||||
"FieldDistribution",
|
||||
"RetrievalStats",
|
||||
"ToolSignature",
|
||||
# TOIN (observation-only since PR-B5)
|
||||
# TOIN (observation-only since PR-B5; per-tenant since PR-F3)
|
||||
"DEFAULT_AUTH_MODE",
|
||||
"DEFAULT_MIN_OBSERVATIONS_TO_PUBLISH",
|
||||
"DEFAULT_MODEL_FAMILY",
|
||||
"DEFAULT_TENANT_KEY",
|
||||
"TOINConfig",
|
||||
"ToolIntelligenceNetwork",
|
||||
"ToolPattern",
|
||||
|
||||
+142
-51
@@ -105,30 +105,48 @@ DEFAULT_MAX_STORAGE_BYTES: Final[int] = 128 * 1024 * 1024
|
||||
MAX_QUERY_PATTERN_LENGTH: Final[int] = 512
|
||||
|
||||
# Aggregation-key serialization separator. Used to encode the
|
||||
# `(auth_mode, model_family, sig_hash)` tuple as a string for JSON
|
||||
# storage (JSON object keys must be strings) and for cross-instance
|
||||
# pattern imports. Pipe is illegal in all three components by
|
||||
# construction (auth_mode ∈ {"unknown","payg","oauth","subscription"};
|
||||
# model_family is a registry name with no `|`; sig_hash is hex).
|
||||
# `(tenant_key, auth_mode, model_family, sig_hash)` tuple as a string
|
||||
# for JSON storage (JSON object keys must be strings) and for
|
||||
# cross-instance pattern imports. Pipe is illegal in all four
|
||||
# components by construction (tenant_key is sanitized to ASCII alphanum
|
||||
# + `-_` by `headroom.proxy.tenant_key`; auth_mode ∈
|
||||
# {"unknown","payg","oauth","subscription"}; model_family is a registry
|
||||
# name with no `|`; sig_hash is hex).
|
||||
_AGG_KEY_SEPARATOR: Final[str] = "|"
|
||||
|
||||
# ── Tenant-key default (PR-F3) ──────────────────────────────────────────
|
||||
# When callers haven't plumbed real tenant-id detection (CLI / batch
|
||||
# jobs / unauthenticated dev) we land in the literal `"global"` slice.
|
||||
# Imported from `headroom.proxy.tenant_key` so there's a single source
|
||||
# of truth — but kept locally as a string default to avoid an
|
||||
# import-time cycle with the proxy package (TOIN ships in environments
|
||||
# without the proxy module). Using the same literal `"global"` string
|
||||
# both modules canonicalize on.
|
||||
DEFAULT_TENANT_KEY: Final[str] = "global"
|
||||
|
||||
|
||||
# ── Aggregation key helpers ─────────────────────────────────────────────
|
||||
PatternKey = tuple[str, str, str]
|
||||
PatternKey = tuple[str, str, str, str]
|
||||
|
||||
|
||||
def _make_pattern_key(
|
||||
auth_mode: str | None,
|
||||
model_family: str | None,
|
||||
sig_hash: str,
|
||||
tenant_key: str | None = None,
|
||||
) -> PatternKey:
|
||||
"""Build the canonical `(auth_mode, model_family, sig_hash)` key.
|
||||
"""Build the canonical `(tenant_key, auth_mode, model_family, sig_hash)` key.
|
||||
|
||||
Defaults populate to `DEFAULT_AUTH_MODE` / `DEFAULT_MODEL_FAMILY`
|
||||
when callers haven't supplied a value — keeps callers terse during
|
||||
the Phase B realignment while PR-F3 wires real detectors.
|
||||
Defaults populate to `DEFAULT_TENANT_KEY` / `DEFAULT_AUTH_MODE` /
|
||||
`DEFAULT_MODEL_FAMILY` when callers haven't supplied a value.
|
||||
PR-F3 introduced the leading `tenant_key` slot so two distinct
|
||||
tenants don't cross-pollinate compression patterns; pre-F3 callers
|
||||
that pass no tenant_key land in the literal `"global"` namespace
|
||||
(which is also where the pre-F3 patterns migrate to on first load
|
||||
via the legacy 3-tuple fallback in `_deserialize_pattern_key`).
|
||||
"""
|
||||
return (
|
||||
tenant_key or DEFAULT_TENANT_KEY,
|
||||
auth_mode or DEFAULT_AUTH_MODE,
|
||||
model_family or DEFAULT_MODEL_FAMILY,
|
||||
sig_hash,
|
||||
@@ -143,17 +161,26 @@ def _serialize_pattern_key(key: PatternKey) -> str:
|
||||
def _deserialize_pattern_key(serialized: str) -> PatternKey:
|
||||
"""Parse a serialized aggregation key back to a tuple.
|
||||
|
||||
Backward-compatible with pre-B5 dumps that stored keys as bare
|
||||
structure hashes (no separator): those parse as
|
||||
`(DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)`. The realignment
|
||||
plan permits wiping the on-disk store, but this fallback keeps reads
|
||||
safe if a stale file appears in the wild.
|
||||
Backward-compat tiers:
|
||||
|
||||
- 4 parts → PR-F3 native ``(tenant_key, auth_mode, model_family, sig_hash)``.
|
||||
- 3 parts → pre-F3 ``(auth_mode, model_family, sig_hash)`` from PR-B5
|
||||
dumps; promoted to the ``DEFAULT_TENANT_KEY`` (``"global"``) slice.
|
||||
- Otherwise (1 part, no separator) → legacy pre-B5 bare ``sig_hash``;
|
||||
promoted to ``(DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE,
|
||||
DEFAULT_MODEL_FAMILY, sig_hash)``.
|
||||
|
||||
The realignment plan permits wiping the on-disk store, but these
|
||||
fallbacks keep reads safe if a stale file appears in the wild.
|
||||
"""
|
||||
parts = serialized.split(_AGG_KEY_SEPARATOR)
|
||||
if len(parts) == 4:
|
||||
return (parts[0], parts[1], parts[2], parts[3])
|
||||
if len(parts) == 3:
|
||||
return (parts[0], parts[1], parts[2])
|
||||
# Legacy format: bare sig_hash. Promote to default tenant slice.
|
||||
return (DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, serialized)
|
||||
# Pre-F3 format: prepend the global tenant slice.
|
||||
return (DEFAULT_TENANT_KEY, parts[0], parts[1], parts[2])
|
||||
# Legacy format: bare sig_hash. Promote to default tenant + default slice.
|
||||
return (DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, serialized)
|
||||
|
||||
|
||||
def get_default_toin_storage_path() -> str:
|
||||
@@ -184,6 +211,27 @@ def get_default_toin_storage_path() -> str:
|
||||
MetricsCallback = Callable[[str, dict[str, Any]], None] # (event_name, event_data) -> None
|
||||
|
||||
|
||||
def _get_current_tenant_key_or_default() -> str:
|
||||
"""Read the request-scoped tenant_key, falling back to ``"global"``.
|
||||
|
||||
Imported lazily so the proxy package is not a hard dependency of
|
||||
TOIN — when ``headroom.proxy.tenant_key`` is unavailable (CLI / batch
|
||||
/ stripped builds) we land in the ``DEFAULT_TENANT_KEY`` namespace,
|
||||
which is the same place the proxy handler routes
|
||||
no-signal-detected requests to. Either path is correctly aggregated
|
||||
under ``"global"``; there is no silent fallback because the proxy's
|
||||
resolver always emits a structured log on resolution.
|
||||
|
||||
See PR-F3 for the full threat model.
|
||||
"""
|
||||
try:
|
||||
from headroom.proxy.tenant_key import get_current_tenant_key
|
||||
|
||||
return get_current_tenant_key()
|
||||
except ImportError:
|
||||
return DEFAULT_TENANT_KEY
|
||||
|
||||
|
||||
@dataclass
|
||||
class ToolPattern:
|
||||
"""Aggregated intelligence about a tool type across all users.
|
||||
@@ -194,12 +242,17 @@ class ToolPattern:
|
||||
|
||||
tool_signature_hash: str
|
||||
|
||||
# === Aggregation Key (PR-B5) ===
|
||||
# === Aggregation Key (PR-B5 + PR-F3) ===
|
||||
# Per-tenant aggregation key extension. The Pattern is keyed inside
|
||||
# the TOIN store by `(auth_mode, model_family, tool_signature_hash)` —
|
||||
# these two fields carry the same values onto the dataclass so dumps,
|
||||
# imports, and publish-CLI rows are self-describing without
|
||||
# cross-referencing the dict key.
|
||||
# the TOIN store by `(tenant_key, auth_mode, model_family,
|
||||
# tool_signature_hash)` — these three fields carry the same values
|
||||
# onto the dataclass so dumps, imports, and publish-CLI rows are
|
||||
# self-describing without cross-referencing the dict key.
|
||||
#
|
||||
# PR-F3 added `tenant_key` so two tenants on the same proxy don't
|
||||
# cross-pollinate compression patterns. Defaults to `"global"`
|
||||
# (the literal namespace pre-F3 patterns live in after migration).
|
||||
tenant_key: str = DEFAULT_TENANT_KEY
|
||||
auth_mode: str = DEFAULT_AUTH_MODE
|
||||
model_family: str = DEFAULT_MODEL_FAMILY
|
||||
|
||||
@@ -287,6 +340,7 @@ class ToolPattern:
|
||||
"""Convert to dictionary for serialization."""
|
||||
return {
|
||||
"tool_signature_hash": self.tool_signature_hash,
|
||||
"tenant_key": self.tenant_key,
|
||||
"auth_mode": self.auth_mode,
|
||||
"model_family": self.model_family,
|
||||
"total_compressions": self.total_compressions,
|
||||
@@ -332,6 +386,7 @@ class ToolPattern:
|
||||
# Filter to only valid fields
|
||||
valid_fields = {
|
||||
"tool_signature_hash",
|
||||
"tenant_key",
|
||||
"auth_mode",
|
||||
"model_family",
|
||||
"total_compressions",
|
||||
@@ -549,11 +604,13 @@ class ToolIntelligenceNetwork:
|
||||
items: list[dict[str, Any]] | None = None,
|
||||
auth_mode: str | None = None,
|
||||
model_family: str | None = None,
|
||||
tenant_key: str | None = None,
|
||||
) -> None:
|
||||
"""Record a compression event.
|
||||
|
||||
Called after SmartCrusher compresses data. Updates the pattern
|
||||
for this `(auth_mode, model_family, tool_signature)` slice.
|
||||
for this `(tenant_key, auth_mode, model_family, tool_signature)`
|
||||
slice.
|
||||
|
||||
TOIN Evolution: When items are provided, we capture field statistics
|
||||
for learning semantic types (uniqueness, default values, etc.).
|
||||
@@ -571,6 +628,11 @@ class ToolIntelligenceNetwork:
|
||||
Defaults to `DEFAULT_AUTH_MODE` when not provided.
|
||||
model_family: Target model family (`claude-3-5`, `gpt-4o`, …).
|
||||
Defaults to `DEFAULT_MODEL_FAMILY` when not provided.
|
||||
tenant_key: PR-F3 per-tenant aggregation slice. When ``None``,
|
||||
reads from the request-scoped ``ContextVar`` populated by
|
||||
the proxy handler (see ``headroom.proxy.tenant_key``);
|
||||
CLI / batch / test callers without a contextvar land in
|
||||
the literal ``"global"`` namespace.
|
||||
"""
|
||||
# The beacon's copy of this event is taken BEFORE the enabled check.
|
||||
# TOIN's own store is gated on HEADROOM_TELEMETRY, which is opt-in and
|
||||
@@ -596,15 +658,22 @@ class ToolIntelligenceNetwork:
|
||||
|
||||
# Computing structure_hash can be expensive for large structures
|
||||
sig_hash = tool_signature.structure_hash
|
||||
key = _make_pattern_key(auth_mode, model_family, sig_hash)
|
||||
# PR-F3: read tenant_key from the request-scoped ContextVar when
|
||||
# the caller didn't pass one explicitly. Imported lazily so TOIN
|
||||
# remains usable in environments where the proxy package is not
|
||||
# installed (CLI batch jobs, tests).
|
||||
if tenant_key is None:
|
||||
tenant_key = _get_current_tenant_key_or_default()
|
||||
key = _make_pattern_key(auth_mode, model_family, sig_hash, tenant_key)
|
||||
|
||||
# LOW FIX #22: Emit compression metric
|
||||
self._emit_metric(
|
||||
"toin.compression",
|
||||
{
|
||||
"signature_hash": sig_hash,
|
||||
"auth_mode": key[0],
|
||||
"model_family": key[1],
|
||||
"tenant_key": key[0],
|
||||
"auth_mode": key[1],
|
||||
"model_family": key[2],
|
||||
"original_count": original_count,
|
||||
"compressed_count": compressed_count,
|
||||
"original_tokens": original_tokens,
|
||||
@@ -619,8 +688,9 @@ class ToolIntelligenceNetwork:
|
||||
if key not in self._patterns:
|
||||
self._patterns[key] = ToolPattern(
|
||||
tool_signature_hash=sig_hash,
|
||||
auth_mode=key[0],
|
||||
model_family=key[1],
|
||||
tenant_key=key[0],
|
||||
auth_mode=key[1],
|
||||
model_family=key[2],
|
||||
)
|
||||
|
||||
pattern = self._patterns[key]
|
||||
@@ -875,6 +945,7 @@ class ToolIntelligenceNetwork:
|
||||
retrieved_items: list[dict[str, Any]] | None = None,
|
||||
auth_mode: str | None = None,
|
||||
model_family: str | None = None,
|
||||
tenant_key: str | None = None,
|
||||
) -> None:
|
||||
"""Record a retrieval event.
|
||||
|
||||
@@ -893,19 +964,25 @@ class ToolIntelligenceNetwork:
|
||||
retrieved_items: Optional list of retrieved items for field-level learning.
|
||||
auth_mode: Tenant auth slice. Defaults to `DEFAULT_AUTH_MODE`.
|
||||
model_family: Target model family. Defaults to `DEFAULT_MODEL_FAMILY`.
|
||||
tenant_key: PR-F3 per-tenant aggregation slice. When ``None``,
|
||||
reads from the request-scoped ``ContextVar`` populated by
|
||||
the proxy handler.
|
||||
"""
|
||||
if not self._config.enabled:
|
||||
return
|
||||
|
||||
key = _make_pattern_key(auth_mode, model_family, tool_signature_hash)
|
||||
if tenant_key is None:
|
||||
tenant_key = _get_current_tenant_key_or_default()
|
||||
key = _make_pattern_key(auth_mode, model_family, tool_signature_hash, tenant_key)
|
||||
|
||||
# LOW FIX #22: Emit retrieval metric
|
||||
self._emit_metric(
|
||||
"toin.retrieval",
|
||||
{
|
||||
"signature_hash": tool_signature_hash,
|
||||
"auth_mode": key[0],
|
||||
"model_family": key[1],
|
||||
"tenant_key": key[0],
|
||||
"auth_mode": key[1],
|
||||
"model_family": key[2],
|
||||
"retrieval_type": retrieval_type,
|
||||
"has_query": query is not None,
|
||||
"query_fields_count": len(query_fields) if query_fields else 0,
|
||||
@@ -918,8 +995,9 @@ class ToolIntelligenceNetwork:
|
||||
# First time seeing this tool via retrieval
|
||||
self._patterns[key] = ToolPattern(
|
||||
tool_signature_hash=tool_signature_hash,
|
||||
auth_mode=key[0],
|
||||
model_family=key[1],
|
||||
tenant_key=key[0],
|
||||
auth_mode=key[1],
|
||||
model_family=key[2],
|
||||
)
|
||||
|
||||
pattern = self._patterns[key]
|
||||
@@ -1243,18 +1321,23 @@ class ToolIntelligenceNetwork:
|
||||
signature_hash: str,
|
||||
auth_mode: str | None = None,
|
||||
model_family: str | None = None,
|
||||
tenant_key: str | None = None,
|
||||
) -> ToolPattern | None:
|
||||
"""Get pattern data for a specific `(auth_mode, model_family, sig_hash)` slice.
|
||||
"""Get pattern data for a specific
|
||||
`(tenant_key, auth_mode, model_family, sig_hash)` slice.
|
||||
|
||||
Defaults to `(DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, signature_hash)`
|
||||
when callers haven't supplied tenant info — preserves source-compat
|
||||
with pre-B5 callers that look up by bare hash.
|
||||
Defaults to `(DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE,
|
||||
DEFAULT_MODEL_FAMILY, signature_hash)` when callers haven't
|
||||
supplied tenant info — preserves source-compat with pre-B5
|
||||
callers that look up by bare hash.
|
||||
|
||||
HIGH FIX: Returns a deep copy to prevent external mutation of internal state.
|
||||
"""
|
||||
import copy
|
||||
|
||||
key = _make_pattern_key(auth_mode, model_family, signature_hash)
|
||||
if tenant_key is None:
|
||||
tenant_key = _get_current_tenant_key_or_default()
|
||||
key = _make_pattern_key(auth_mode, model_family, signature_hash, tenant_key)
|
||||
|
||||
with self._lock:
|
||||
pattern = self._patterns.get(key)
|
||||
@@ -1298,10 +1381,16 @@ class ToolIntelligenceNetwork:
|
||||
Used for federated learning: aggregate patterns from multiple
|
||||
Headroom instances without sharing actual data.
|
||||
|
||||
Backward-compatible with v1.0 dumps that keyed patterns by bare
|
||||
structure_hash: those are promoted to the
|
||||
`(DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)` slice via
|
||||
`_deserialize_pattern_key`.
|
||||
Backward-compatible with two pre-F3 formats:
|
||||
|
||||
- v1.0 dumps that keyed patterns by bare ``sig_hash``: promoted
|
||||
to ``(DEFAULT_TENANT_KEY, DEFAULT_AUTH_MODE,
|
||||
DEFAULT_MODEL_FAMILY, sig_hash)``.
|
||||
- v2.0 / pre-F3 dumps that used the 3-tuple
|
||||
``(auth_mode, model_family, sig_hash)`` key: promoted to the
|
||||
``DEFAULT_TENANT_KEY`` (``"global"``) slice.
|
||||
|
||||
See ``_deserialize_pattern_key`` for the parsing tiers.
|
||||
|
||||
Args:
|
||||
data: Exported pattern data.
|
||||
@@ -1317,11 +1406,13 @@ class ToolIntelligenceNetwork:
|
||||
key = _deserialize_pattern_key(serialized_key)
|
||||
imported = ToolPattern.from_dict(pattern_dict)
|
||||
self._sanitize_loaded_pattern(imported)
|
||||
# Make sure dataclass fields agree with the dict key — pre-B5
|
||||
# dumps don't carry auth_mode/model_family on the pattern;
|
||||
# promote from the (possibly default) key.
|
||||
imported.auth_mode = key[0]
|
||||
imported.model_family = key[1]
|
||||
# Make sure dataclass fields agree with the dict key — pre-F3
|
||||
# dumps don't carry tenant_key on the pattern, and pre-B5
|
||||
# dumps don't carry auth_mode/model_family either; promote
|
||||
# from the (possibly default) key.
|
||||
imported.tenant_key = key[0]
|
||||
imported.auth_mode = key[1]
|
||||
imported.model_family = key[2]
|
||||
|
||||
if key in self._patterns:
|
||||
# Merge with existing
|
||||
@@ -1669,9 +1760,9 @@ def _create_default_toin_backend() -> Any:
|
||||
# `tenant_prefix` is retained for storage-backend namespacing
|
||||
# (Redis key prefix, Postgres schema name, etc.) so multi-tenant
|
||||
# SaaS deployments can carve up shared infrastructure. PR-B5 made
|
||||
# the in-memory aggregation key per-tenant via `auth_mode` /
|
||||
# `model_family`, so `tenant_prefix` is now functionally redundant
|
||||
# for *learning* — it only matters for storage layout. Keep it.
|
||||
# the in-memory aggregation key per tenant via `tenant_key`, so
|
||||
# `tenant_prefix` is now functionally redundant for *learning* —
|
||||
# it only matters for storage layout. Keep it.
|
||||
kwargs = {
|
||||
"url": os.environ.get("HEADROOM_TOIN_URL", ""),
|
||||
"tenant_prefix": os.environ.get("HEADROOM_TOIN_TENANT_PREFIX", ""),
|
||||
|
||||
@@ -124,7 +124,7 @@ class TestTOINDoubleCountFix:
|
||||
|
||||
# Get the pattern
|
||||
with toin._lock:
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
user_count_after_101 = pattern.user_count
|
||||
|
||||
# Now call again with same instance (instance_100)
|
||||
@@ -133,7 +133,7 @@ class TestTOINDoubleCountFix:
|
||||
toin.record_compression(sig, 100, 10, 1000, 100, strategy="test_strategy")
|
||||
|
||||
with toin._lock:
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
user_count_after_102 = pattern.user_count
|
||||
|
||||
# Restore instance_id
|
||||
|
||||
+11
-11
@@ -76,7 +76,7 @@ class TestAllSeenInstancesUnboundedGrowth:
|
||||
# Simulate adding users via record_compression
|
||||
# (the cap is enforced there, not when directly adding to set)
|
||||
pattern = ToolPattern(tool_signature_hash=sig.structure_hash)
|
||||
toin._patterns[("unknown", "unknown", sig.structure_hash)] = pattern
|
||||
toin._patterns[("global", "unknown", "unknown", sig.structure_hash)] = pattern
|
||||
|
||||
# Direct manipulation should still work for testing
|
||||
for i in range(200):
|
||||
@@ -101,7 +101,7 @@ class TestAllSeenInstancesUnboundedGrowth:
|
||||
# Record compressions from 150 "users" (simulated)
|
||||
# by directly manipulating the pattern
|
||||
pattern = ToolPattern(tool_signature_hash=sig.structure_hash)
|
||||
toin._patterns[("unknown", "unknown", sig.structure_hash)] = pattern
|
||||
toin._patterns[("global", "unknown", "unknown", sig.structure_hash)] = pattern
|
||||
|
||||
# Track 150 unique users
|
||||
for i in range(150):
|
||||
@@ -183,7 +183,7 @@ class TestAllSeenInstancesSerialization:
|
||||
)
|
||||
|
||||
# Manually add more users to simulate multi-user scenario
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
for i in range(50):
|
||||
instance_hash = hashlib.sha256(f"extra_user_{i}".encode()).hexdigest()[:8]
|
||||
if instance_hash not in pattern._all_seen_instances:
|
||||
@@ -202,7 +202,7 @@ class TestAllSeenInstancesSerialization:
|
||||
toin2 = ToolIntelligenceNetwork(config)
|
||||
|
||||
# Verify user count is preserved
|
||||
pattern2 = toin2._patterns.get(("unknown", "unknown", sig.structure_hash))
|
||||
pattern2 = toin2._patterns.get(("global", "unknown", "unknown", sig.structure_hash))
|
||||
assert pattern2 is not None
|
||||
assert pattern2.user_count == original_user_count
|
||||
|
||||
@@ -252,7 +252,7 @@ class TestUserCountMergeLogic:
|
||||
imported.sample_size = 5
|
||||
|
||||
# Merge
|
||||
toin._patterns[("unknown", "unknown", "test_hash")] = existing
|
||||
toin._patterns[("global", "unknown", "unknown", "test_hash")] = existing
|
||||
toin._merge_patterns(existing, imported)
|
||||
|
||||
# After merge: 5 existing + 2 new = 7 unique users
|
||||
@@ -287,7 +287,7 @@ class TestUserCountMergeLogic:
|
||||
imported.sample_size = 20
|
||||
|
||||
# Merge
|
||||
toin._patterns[("unknown", "unknown", "test_hash")] = existing
|
||||
toin._patterns[("global", "unknown", "unknown", "test_hash")] = existing
|
||||
toin._merge_patterns(existing, imported)
|
||||
|
||||
# After merge: 120 existing + 10 new = 130 unique users
|
||||
@@ -728,7 +728,7 @@ class TestTOINHighPriorityFixes:
|
||||
query_fields=[f"unique_field_{i}"],
|
||||
)
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
assert len(pattern.field_retrieval_frequency) <= 100
|
||||
|
||||
def test_commonly_retrieved_fields_bounded(self):
|
||||
@@ -757,7 +757,7 @@ class TestTOINHighPriorityFixes:
|
||||
query_fields=[f"common_field_{i}"],
|
||||
)
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
assert len(pattern.commonly_retrieved_fields) <= 20
|
||||
|
||||
def test_strategy_success_rate_updates(self):
|
||||
@@ -778,7 +778,7 @@ class TestTOINHighPriorityFixes:
|
||||
strategy="TEST_STRATEGY",
|
||||
)
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
initial_rate = pattern.strategy_success_rates["TEST_STRATEGY"]
|
||||
assert initial_rate == 1.0 # Starts at 1.0
|
||||
|
||||
@@ -790,7 +790,7 @@ class TestTOINHighPriorityFixes:
|
||||
strategy="TEST_STRATEGY",
|
||||
)
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
after_retrieval = pattern.strategy_success_rates["TEST_STRATEGY"]
|
||||
assert after_retrieval < initial_rate # Should decrease
|
||||
|
||||
@@ -805,7 +805,7 @@ class TestTOINHighPriorityFixes:
|
||||
strategy="TEST_STRATEGY",
|
||||
)
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
after_compressions = pattern.strategy_success_rates["TEST_STRATEGY"]
|
||||
assert after_compressions > after_retrieval # Should increase
|
||||
|
||||
|
||||
@@ -178,6 +178,35 @@ def test_strip_preserves_value_semantics() -> None:
|
||||
assert out["anthropic-version"] == "2023-06-01"
|
||||
|
||||
|
||||
def test_strip_default_tenant_header_removed() -> None:
|
||||
"""Tenant header is ingress-only and must not reach providers."""
|
||||
out = _strip_internal_headers(
|
||||
{
|
||||
"Authorization": "Bearer sk-ant-...",
|
||||
"X-Headroom-Tenant-ID": "tenant_a",
|
||||
}
|
||||
)
|
||||
|
||||
assert "X-Headroom-Tenant-ID" not in out
|
||||
assert out["Authorization"] == "Bearer sk-ant-..."
|
||||
|
||||
|
||||
def test_strip_custom_tenant_header_removed(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Configured tenant header is also stripped, even outside x-headroom-*."""
|
||||
monkeypatch.setenv("HEADROOM_TENANT_KEY_HEADER", "X-Tenant-ID")
|
||||
|
||||
out = _strip_internal_headers(
|
||||
{
|
||||
"Authorization": "Bearer sk-ant-...",
|
||||
"X-Tenant-ID": "tenant_a",
|
||||
"x-request-id": "rid-1",
|
||||
}
|
||||
)
|
||||
|
||||
assert "X-Tenant-ID" not in out
|
||||
assert out["x-request-id"] == "rid-1"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# End-to-end: x-headroom-* never reaches the upstream
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -529,9 +529,10 @@ class TestEndToEndTOINIntegration:
|
||||
store = get_compression_store()
|
||||
store.process_pending_feedback()
|
||||
|
||||
# PR-B5: pattern key is now `(auth_mode, model_family, sig_hash)`.
|
||||
# Callers that don't supply auth/model land on the
|
||||
# `("unknown", "unknown", sig_hash)` slot.
|
||||
# PR-B5 + PR-F3: pattern key is now `(tenant_key, auth_mode,
|
||||
# model_family, sig_hash)`. Callers that don't supply
|
||||
# tenant/auth/model land on the
|
||||
# `("global", "unknown", "unknown", sig_hash)` slot.
|
||||
from headroom.telemetry.toin import _make_pattern_key
|
||||
|
||||
pattern = fresh_toin._patterns.get(_make_pattern_key(None, None, signature.structure_hash))
|
||||
@@ -607,9 +608,10 @@ class TestEndToEndTOINIntegration:
|
||||
store.process_pending_feedback()
|
||||
|
||||
# Step 3: Verify TOIN learned
|
||||
# PR-B5: pattern key is now `(auth_mode, model_family, sig_hash)`.
|
||||
# Callers that don't supply auth/model land on the
|
||||
# `("unknown", "unknown", sig_hash)` slot.
|
||||
# PR-B5 + PR-F3: pattern key is now `(tenant_key, auth_mode,
|
||||
# model_family, sig_hash)`. Callers that don't supply
|
||||
# tenant/auth/model land on the
|
||||
# `("global", "unknown", "unknown", sig_hash)` slot.
|
||||
from headroom.telemetry.toin import _make_pattern_key
|
||||
|
||||
pattern = fresh_toin._patterns.get(_make_pattern_key(None, None, signature.structure_hash))
|
||||
|
||||
@@ -38,6 +38,7 @@ pytest.importorskip("fastapi")
|
||||
|
||||
from headroom.proxy.helpers import COMPRESSION_TIMEOUT_SECONDS # noqa: F401
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
from headroom.proxy.tenant_key import get_current_tenant_key, set_request_tenant_key
|
||||
|
||||
|
||||
def _make_proxy(compression_max_workers: int | None = None):
|
||||
@@ -83,6 +84,23 @@ def test_compression_executor_minimum_one_worker() -> None:
|
||||
assert proxy.compression_max_workers == 1
|
||||
|
||||
|
||||
def test_compression_executor_preserves_request_contextvars() -> None:
|
||||
"""Tenant context survives the thread-pool hop used by compression."""
|
||||
proxy = _make_proxy(compression_max_workers=1)
|
||||
|
||||
async def _drive() -> str:
|
||||
set_request_tenant_key("tenant_executor")
|
||||
try:
|
||||
return await proxy._run_compression_in_executor(
|
||||
get_current_tenant_key,
|
||||
timeout=10.0,
|
||||
)
|
||||
finally:
|
||||
set_request_tenant_key(None)
|
||||
|
||||
assert asyncio.run(_drive()) == "tenant_executor"
|
||||
|
||||
|
||||
def test_in_flight_gauge_tracks_running_compressions() -> None:
|
||||
"""While a compression is running, ``_compression_in_flight`` reads ≥ 1.
|
||||
After it completes, it returns to 0. The high-water mark records the
|
||||
|
||||
@@ -0,0 +1,228 @@
|
||||
"""F3 handler-integration tests: tenant_key plumbing through the proxy.
|
||||
|
||||
Pins that the OpenAI / Anthropic handlers call ``resolve_tenant_key``
|
||||
once at request entry, populate ``request.state.tenant_key`` /
|
||||
``request.state.tenant_key_source``, AND set the request-scoped
|
||||
ContextVar so downstream TOIN ``record_compression`` calls observe the
|
||||
right tenant slice.
|
||||
|
||||
Uses the same ``_DummyOpenAIHandler`` pattern as
|
||||
``tests/test_openai_codex_routing.py`` so we don't need the Rust core
|
||||
or a live upstream HTTP server.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
from fastapi import Request
|
||||
|
||||
from headroom.proxy.handlers.openai import OpenAIHandlerMixin
|
||||
from headroom.proxy.tenant_key import (
|
||||
GLOBAL_TENANT_KEY,
|
||||
SOURCE_GLOBAL,
|
||||
SOURCE_HEADER,
|
||||
get_current_tenant_key,
|
||||
set_request_tenant_key,
|
||||
)
|
||||
|
||||
|
||||
def _jwt(payload: dict) -> str:
|
||||
"""Build a 3-segment JWT for OAuth classification."""
|
||||
header = {"alg": "none", "typ": "JWT"}
|
||||
|
||||
def encode(part: dict) -> str:
|
||||
raw = json.dumps(part, separators=(",", ":")).encode("utf-8")
|
||||
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
||||
|
||||
return f"{encode(header)}.{encode(payload)}."
|
||||
|
||||
|
||||
# ── Test fixtures (Dummy handler) ─────────────────────────────────────────
|
||||
|
||||
|
||||
class _DummyMetrics:
|
||||
async def record_request(self, **kwargs): # noqa: ANN003
|
||||
return None
|
||||
|
||||
async def record_failed(self, **kwargs): # noqa: ANN003
|
||||
return None
|
||||
|
||||
|
||||
class _DummyTokenizer:
|
||||
def count_messages(self, messages):
|
||||
return len(messages)
|
||||
|
||||
|
||||
class _ResponseStub:
|
||||
status_code = 200
|
||||
headers = {"content-type": "application/json", "content-length": "42"}
|
||||
content = b'{"id":"resp_123","output":[{"type":"message"}]}'
|
||||
|
||||
def json(self):
|
||||
return {"usage": {"input_tokens": 2, "output_tokens": 1}}
|
||||
|
||||
|
||||
class _DummyOpenAIHandler(OpenAIHandlerMixin):
|
||||
"""Minimal handler that captures the request that flows through F3."""
|
||||
|
||||
OPENAI_API_URL = "https://api.openai.com"
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.rate_limiter = None
|
||||
self.metrics = _DummyMetrics()
|
||||
self.config = SimpleNamespace(
|
||||
optimize=False,
|
||||
retry_max_attempts=3,
|
||||
retry_base_delay_ms=10,
|
||||
retry_max_delay_ms=50,
|
||||
connect_timeout_seconds=10,
|
||||
openai_extra_headers=None,
|
||||
)
|
||||
self.usage_reporter = None
|
||||
self.openai_provider = SimpleNamespace(get_context_limit=lambda model: 128_000)
|
||||
self.openai_pipeline = SimpleNamespace(apply=lambda **kwargs: None)
|
||||
self.anthropic_backend = None
|
||||
self.cost_tracker = None
|
||||
self.memory_handler = None
|
||||
self.session_tracker_store = SimpleNamespace(
|
||||
compute_session_id=lambda *a, **k: "sess-openai-1",
|
||||
)
|
||||
self.captured_request: Request | None = None
|
||||
|
||||
async def _next_request_id(self) -> str:
|
||||
return "req-1"
|
||||
|
||||
def _extract_tags(self, headers: dict[str, str]) -> dict[str, str]:
|
||||
return {}
|
||||
|
||||
async def _retry_request(self, method: str, url: str, headers: dict, body: dict):
|
||||
return _ResponseStub()
|
||||
|
||||
async def _run_compression_in_executor(self, fn, *, timeout: float):
|
||||
return fn()
|
||||
|
||||
async def _stream_response(self, **kwargs):
|
||||
return SimpleNamespace(status_code=200)
|
||||
|
||||
|
||||
def _build_request(headers: dict[str, str], body: dict) -> Request:
|
||||
payload = json.dumps(body).encode("utf-8")
|
||||
|
||||
async def receive():
|
||||
return {"type": "http.request", "body": payload, "more_body": False}
|
||||
|
||||
scope = {
|
||||
"type": "http",
|
||||
"asgi": {"version": "3.0"},
|
||||
"http_version": "1.1",
|
||||
"method": "POST",
|
||||
"scheme": "https",
|
||||
"path": "/v1/responses",
|
||||
"raw_path": b"/v1/responses",
|
||||
"query_string": b"",
|
||||
"headers": [
|
||||
(key.lower().encode("utf-8"), value.encode("utf-8")) for key, value in headers.items()
|
||||
],
|
||||
"client": ("127.0.0.1", 12345),
|
||||
"server": ("testserver", 443),
|
||||
}
|
||||
return Request(scope, receive)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_tenant_key_contextvar():
|
||||
"""Clean ContextVar after every test so they don't leak."""
|
||||
set_request_tenant_key(None)
|
||||
yield
|
||||
set_request_tenant_key(None)
|
||||
|
||||
|
||||
# ── F3 plumbing pins ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_handler_populates_request_state_tenant_key_for_header_path(monkeypatch):
|
||||
"""Header-set tenant_id ⇒ request.state.tenant_key + source="header"."""
|
||||
handler = _DummyOpenAIHandler()
|
||||
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
|
||||
|
||||
request = _build_request(
|
||||
headers={
|
||||
"Authorization": "Bearer sk-test-key",
|
||||
"X-Headroom-Tenant-ID": "cust_acme",
|
||||
},
|
||||
body={"model": "gpt-5.4", "input": "hi"},
|
||||
)
|
||||
|
||||
anyio.run(handler.handle_openai_responses, request)
|
||||
|
||||
assert getattr(request.state, "tenant_key", None) == "cust_acme"
|
||||
assert getattr(request.state, "tenant_key_source", None) == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_handler_populates_request_state_tenant_key_for_global_fallback(monkeypatch):
|
||||
"""No tenant header, even with bearer auth, ⇒ tenant_key is global."""
|
||||
handler = _DummyOpenAIHandler()
|
||||
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
|
||||
|
||||
# PAYG bearer (sk-...). F1 will classify auth_mode=PAYG, but F3
|
||||
# requires an explicit tenant header instead of deriving from secrets.
|
||||
request = _build_request(
|
||||
headers={"Authorization": "Bearer sk-ant-api03-aaaabbbbccccdddd"},
|
||||
body={"model": "gpt-5.4", "input": "hi"},
|
||||
)
|
||||
|
||||
anyio.run(handler.handle_openai_responses, request)
|
||||
|
||||
assert getattr(request.state, "tenant_key_source", None) == SOURCE_GLOBAL
|
||||
tenant_key = getattr(request.state, "tenant_key", None)
|
||||
assert tenant_key == GLOBAL_TENANT_KEY
|
||||
|
||||
|
||||
def test_handler_populates_global_when_no_signals(monkeypatch):
|
||||
"""No header, no bearer ⇒ tenant_key="global" with source="global"."""
|
||||
handler = _DummyOpenAIHandler()
|
||||
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
|
||||
|
||||
# Neither auth nor explicit tenant header.
|
||||
request = _build_request(
|
||||
headers={},
|
||||
body={"model": "gpt-5.4", "input": "hi"},
|
||||
)
|
||||
|
||||
anyio.run(handler.handle_openai_responses, request)
|
||||
|
||||
assert getattr(request.state, "tenant_key", None) == GLOBAL_TENANT_KEY
|
||||
assert getattr(request.state, "tenant_key_source", None) == SOURCE_GLOBAL
|
||||
|
||||
|
||||
def test_handler_populates_contextvar_so_toin_records_under_tenant(monkeypatch):
|
||||
"""Handler must call set_request_tenant_key — TOIN reads the ContextVar.
|
||||
|
||||
We verify by asserting the contextvar is populated AFTER the
|
||||
handler returns (the handler doesn't reset it on success — that
|
||||
would defeat the whole point, since downstream record_compression
|
||||
happens deep inside the pipeline). The contextvar is per-task; the
|
||||
next request's handler will overwrite it.
|
||||
"""
|
||||
handler = _DummyOpenAIHandler()
|
||||
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
|
||||
|
||||
request = _build_request(
|
||||
headers={"X-Headroom-Tenant-ID": "tenant_for_toin"},
|
||||
body={"model": "gpt-5.4", "input": "hi"},
|
||||
)
|
||||
|
||||
# Run inside an explicit context so the test can read back the
|
||||
# ContextVar the handler set. Without `anyio.run`'s default
|
||||
# context isolation, we'd see a stale value across tests.
|
||||
async def _drive() -> str:
|
||||
await handler.handle_openai_responses(request)
|
||||
return get_current_tenant_key()
|
||||
|
||||
seen = anyio.run(_drive)
|
||||
assert seen == "tenant_for_toin"
|
||||
@@ -0,0 +1,280 @@
|
||||
"""Unit tests for ``headroom.proxy.tenant_key.resolve_tenant_key``.
|
||||
|
||||
Phase F PR-F3 — covers the two resolution paths (header / global), the structured-log requirement, and the sanitization edge
|
||||
cases (empty, too-long, unicode, control chars, hyphens, mixed
|
||||
ASCII).
|
||||
|
||||
The handler-integration tests (``request.state`` plumbing) live with
|
||||
the proxy handler tests; this file pins the pure resolver contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.proxy.tenant_key import (
|
||||
DEFAULT_TENANT_KEY_HEADER,
|
||||
GLOBAL_TENANT_KEY,
|
||||
SOURCE_GLOBAL,
|
||||
SOURCE_HEADER,
|
||||
TENANT_KEY_HEADER_ENV_VAR,
|
||||
get_current_tenant_key,
|
||||
resolve_tenant_key,
|
||||
set_request_tenant_key,
|
||||
)
|
||||
|
||||
|
||||
def _request(
|
||||
headers: dict[str, str] | None = None, auth_mode: str | None = None
|
||||
) -> SimpleNamespace:
|
||||
"""Build a minimal Starlette-shaped Request fixture.
|
||||
|
||||
SimpleNamespace + a plain dict for `headers` is sufficient because
|
||||
``resolve_tenant_key`` only touches ``request.headers.get`` /
|
||||
``.items`` and ``request.state.auth_mode``.
|
||||
"""
|
||||
state = SimpleNamespace(auth_mode=auth_mode)
|
||||
return SimpleNamespace(headers=headers or {}, state=state)
|
||||
|
||||
|
||||
# ── Header path ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_header_path_returns_header_source() -> None:
|
||||
"""Header present + sanitized non-empty ⇒ header source."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "cust_abc123"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "cust_abc123"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_header_path_lowercase_header_lookup() -> None:
|
||||
"""Header lookup is case-insensitive (Starlette norm)."""
|
||||
req = _request(headers={"x-headroom-tenant-id": "tenant42"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "tenant42"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_header_path_custom_header_via_env(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""``HEADROOM_TENANT_KEY_HEADER`` overrides the default name."""
|
||||
monkeypatch.setenv(TENANT_KEY_HEADER_ENV_VAR, "X-Custom-Tenant")
|
||||
req = _request(headers={"X-Custom-Tenant": "company9"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "company9"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_header_path_default_name_is_x_headroom_tenant_id() -> None:
|
||||
"""Default header name is documented as ``X-Headroom-Tenant-ID``."""
|
||||
assert DEFAULT_TENANT_KEY_HEADER == "X-Headroom-Tenant-ID"
|
||||
|
||||
|
||||
def test_header_path_falls_through_when_empty_after_sanitization() -> None:
|
||||
"""Pure-non-allowed header (e.g. all unicode) ⇒ fall through, NOT header source."""
|
||||
# All-emoji header sanitizes to empty string — neither header
|
||||
# so we drop to the global fallback.
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "🦀🚀"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert source == SOURCE_GLOBAL
|
||||
assert tenant_key == GLOBAL_TENANT_KEY
|
||||
|
||||
|
||||
# ── Hash path ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_bearer_without_header_uses_global_namespace() -> None:
|
||||
req = _request(
|
||||
headers={"authorization": "Bearer sk-ant-api03-abc123def"},
|
||||
auth_mode="payg",
|
||||
)
|
||||
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
|
||||
assert tenant_key == GLOBAL_TENANT_KEY
|
||||
assert source == SOURCE_GLOBAL
|
||||
|
||||
|
||||
def test_auth_mode_without_header_does_not_partition_tenants() -> None:
|
||||
bearer = "Bearer same-tok-aaabbbccc"
|
||||
|
||||
payg = resolve_tenant_key(_request(headers={"authorization": bearer}, auth_mode="payg"))
|
||||
oauth = resolve_tenant_key(_request(headers={"authorization": bearer}, auth_mode="oauth"))
|
||||
|
||||
assert payg == (GLOBAL_TENANT_KEY, SOURCE_GLOBAL)
|
||||
assert oauth == (GLOBAL_TENANT_KEY, SOURCE_GLOBAL)
|
||||
|
||||
|
||||
def test_global_fallback_when_no_signals() -> None:
|
||||
"""No header, no auth, no key ⇒ literal ``"global"``."""
|
||||
req = _request()
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == GLOBAL_TENANT_KEY == "global"
|
||||
assert source == SOURCE_GLOBAL
|
||||
|
||||
|
||||
def _capture_tenant_resolution_logs(monkeypatch: pytest.MonkeyPatch) -> list[SimpleNamespace]:
|
||||
records: list[SimpleNamespace] = []
|
||||
|
||||
def _info(_message: str, *, extra: dict[str, object] | None = None) -> None:
|
||||
records.append(SimpleNamespace(**(extra or {})))
|
||||
|
||||
monkeypatch.setattr("headroom.proxy.tenant_key.logger.info", _info)
|
||||
return records
|
||||
|
||||
|
||||
def test_global_fallback_emits_structured_log(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The ``"global"`` source MUST log (no silent fallback)."""
|
||||
req = _request()
|
||||
matching = _capture_tenant_resolution_logs(monkeypatch)
|
||||
resolve_tenant_key(req)
|
||||
|
||||
assert len(matching) == 1
|
||||
assert matching[0].source == "global" # type: ignore[attr-defined]
|
||||
assert matching[0].tenant_key == "global" # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def test_header_source_emits_structured_log(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Header path also emits the structured log."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "tenant1"})
|
||||
matching = _capture_tenant_resolution_logs(monkeypatch)
|
||||
resolve_tenant_key(req)
|
||||
assert len(matching) == 1
|
||||
assert matching[0].source == "header" # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def test_bearer_without_header_emits_global_structured_log(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Bearer auth without a tenant header still emits the global log."""
|
||||
req = _request(
|
||||
headers={"authorization": "Bearer sk-ant-api03-abc123def"},
|
||||
auth_mode="payg",
|
||||
)
|
||||
matching = _capture_tenant_resolution_logs(monkeypatch)
|
||||
resolve_tenant_key(req)
|
||||
assert len(matching) == 1
|
||||
assert matching[0].source == "global" # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# ── Sanitization edges ────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_sanitization_allows_alphanumeric() -> None:
|
||||
"""ASCII alphanumerics survive untouched."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "Cust42TenantA"})
|
||||
tenant_key, _ = resolve_tenant_key(req)
|
||||
assert tenant_key == "Cust42TenantA"
|
||||
|
||||
|
||||
def test_sanitization_allows_hyphens_and_underscores() -> None:
|
||||
"""Hyphens and underscores survive (common in SaaS tenant IDs)."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "cust_a-b_c-9"})
|
||||
tenant_key, _ = resolve_tenant_key(req)
|
||||
assert tenant_key == "cust_a-b_c-9"
|
||||
|
||||
|
||||
def test_sanitization_drops_control_chars() -> None:
|
||||
"""Control chars (``\\x00`` / ``\\x1f``) get stripped."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "ten\x00ant\x1f1"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "tenant1"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_sanitization_drops_tab_and_newline() -> None:
|
||||
"""Whitespace (``\\t`` / ``\\n``) gets stripped."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "tenant\t\n1"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "tenant1"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_sanitization_drops_punctuation() -> None:
|
||||
"""`|`, `:`, `.`, `,`, `/` stripped (TOIN store-key separator safety)."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "ten|ant:1.0/x"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "tenant10x"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_sanitization_drops_unicode_falls_through_when_pure_unicode() -> None:
|
||||
"""Pure-unicode header (CJK / emoji) sanitizes to empty ⇒ falls through."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "测试テスト"})
|
||||
_, source = resolve_tenant_key(req)
|
||||
# No header source — sanitized to empty, dropped to global.
|
||||
assert source == SOURCE_GLOBAL
|
||||
|
||||
|
||||
def test_sanitization_truncates_at_64_chars() -> None:
|
||||
"""Headers > 64 ASCII alphanum chars are TRUNCATED, not rejected."""
|
||||
raw = "a" * 100
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": raw})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "a" * 64
|
||||
assert len(tenant_key) == 64
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
def test_sanitization_empty_header_falls_through() -> None:
|
||||
"""Empty header value ⇒ fall through (header treated as absent)."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": ""})
|
||||
_, source = resolve_tenant_key(req)
|
||||
assert source == SOURCE_GLOBAL
|
||||
|
||||
|
||||
def test_sanitization_whitespace_only_falls_through() -> None:
|
||||
"""Whitespace-only header ⇒ sanitizes to empty ⇒ falls through."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": " \t\n "})
|
||||
_, source = resolve_tenant_key(req)
|
||||
assert source == SOURCE_GLOBAL
|
||||
|
||||
|
||||
def test_sanitization_mixed_unicode_and_ascii_preserves_ascii() -> None:
|
||||
"""``cust_我_42`` sanitizes to ``cust__42`` (ASCII chars survive)."""
|
||||
req = _request(headers={"X-Headroom-Tenant-ID": "cust_我_42"})
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "cust__42"
|
||||
assert source == SOURCE_HEADER
|
||||
|
||||
|
||||
# ── ContextVar surface ────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_get_current_tenant_key_defaults_to_global() -> None:
|
||||
"""Outside a request context, ``get_current_tenant_key`` ⇒ ``"global"``."""
|
||||
set_request_tenant_key(None) # Make sure we're cleared.
|
||||
assert get_current_tenant_key() == GLOBAL_TENANT_KEY
|
||||
|
||||
|
||||
def test_set_request_tenant_key_round_trips() -> None:
|
||||
"""``set`` then ``get`` returns the value."""
|
||||
set_request_tenant_key("tenant_abc")
|
||||
try:
|
||||
assert get_current_tenant_key() == "tenant_abc"
|
||||
finally:
|
||||
set_request_tenant_key(None)
|
||||
|
||||
|
||||
def test_set_request_tenant_key_none_resets_to_global() -> None:
|
||||
"""Clearing returns to the global default."""
|
||||
set_request_tenant_key("tenant_abc")
|
||||
set_request_tenant_key(None)
|
||||
assert get_current_tenant_key() == GLOBAL_TENANT_KEY
|
||||
|
||||
|
||||
# ── Header precedence over auth headers ───────────────────────────────
|
||||
|
||||
|
||||
def test_header_takes_precedence_over_auth_header() -> None:
|
||||
"""When both a tenant header and bearer auth are present, the tenant header wins."""
|
||||
req = _request(
|
||||
headers={
|
||||
"X-Headroom-Tenant-ID": "cust_explicit",
|
||||
"authorization": "Bearer sk-ant-api03-aaaaa",
|
||||
},
|
||||
auth_mode="payg",
|
||||
)
|
||||
tenant_key, source = resolve_tenant_key(req)
|
||||
assert tenant_key == "cust_explicit"
|
||||
assert source == SOURCE_HEADER
|
||||
+2
-1
@@ -457,7 +457,8 @@ class TestTOINExportImport:
|
||||
assert len(export["patterns"]) == 1
|
||||
# PR-B5: keys are now serialized "auth|model|hash" tuples; default
|
||||
# auth/model produce the "unknown|unknown|<hash>" string.
|
||||
assert f"unknown|unknown|{sig.structure_hash}" in export["patterns"]
|
||||
# PR-F3: prepended with the tenant_key slot — default is "global".
|
||||
assert f"global|unknown|unknown|{sig.structure_hash}" in export["patterns"]
|
||||
|
||||
def test_import_patterns_new_pattern(self):
|
||||
"""import_patterns adds new patterns."""
|
||||
|
||||
@@ -106,7 +106,7 @@ class TestGetRecommendationObservations:
|
||||
toin.get_recommendation(sig)
|
||||
|
||||
# Check observations incremented
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
assert pattern.observations == 1
|
||||
|
||||
# Call again
|
||||
@@ -133,7 +133,7 @@ class TestGetRecommendationObservations:
|
||||
result = toin.get_recommendation(sig)
|
||||
assert result.source == "local" # Not enough samples
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig.structure_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig.structure_hash)]
|
||||
assert pattern.observations == 1
|
||||
|
||||
def test_no_increment_for_unknown_pattern(self):
|
||||
@@ -199,7 +199,7 @@ class TestRecordRetrievalPopulatesFields:
|
||||
query_fields=["error_message"],
|
||||
)
|
||||
|
||||
pattern = toin._patterns[("unknown", "unknown", sig_hash)]
|
||||
pattern = toin._patterns[("global", "unknown", "unknown", sig_hash)]
|
||||
assert pattern.total_retrievals == 5
|
||||
assert pattern.search_retrievals == 5
|
||||
assert len(pattern.field_retrieval_frequency) > 0
|
||||
|
||||
+18
-16
@@ -91,7 +91,7 @@ class TestStrategySuccessRates:
|
||||
)
|
||||
|
||||
# Set high success rate
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.strategy_success_rates["smart_sample"] = 0.8
|
||||
pattern.optimal_strategy = "smart_sample"
|
||||
|
||||
@@ -117,7 +117,7 @@ class TestStrategySuccessRates:
|
||||
)
|
||||
|
||||
# Set low success rate
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.strategy_success_rates["bad_strategy"] = 0.2
|
||||
pattern.optimal_strategy = "bad_strategy"
|
||||
|
||||
@@ -146,7 +146,7 @@ class TestStrategySuccessRates:
|
||||
)
|
||||
|
||||
# Set up multiple strategies with different success rates
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.strategy_success_rates = {
|
||||
"bad_strategy": 0.2,
|
||||
"good_strategy": 0.9,
|
||||
@@ -179,7 +179,7 @@ class TestPreserveFieldsMerging:
|
||||
compressed_tokens=500,
|
||||
strategy="smart_sample",
|
||||
)
|
||||
local_pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
|
||||
local_pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
|
||||
local_pattern.preserve_fields = ["field_a", "field_b"]
|
||||
|
||||
# Import pattern with different preserve_fields
|
||||
@@ -198,7 +198,7 @@ class TestPreserveFieldsMerging:
|
||||
fresh_toin.import_patterns(import_data)
|
||||
|
||||
# Verify merge
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
|
||||
assert "field_a" in pattern.preserve_fields
|
||||
assert "field_b" in pattern.preserve_fields
|
||||
assert "field_c" in pattern.preserve_fields
|
||||
@@ -219,7 +219,7 @@ class TestPreserveFieldsMerging:
|
||||
compressed_tokens=500,
|
||||
strategy="smart_sample",
|
||||
)
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
|
||||
pattern.preserve_fields = [f"field_{i}" for i in range(8)]
|
||||
|
||||
# Import with 5 more fields
|
||||
@@ -237,7 +237,7 @@ class TestPreserveFieldsMerging:
|
||||
fresh_toin.import_patterns(import_data)
|
||||
|
||||
# Should be capped at 10
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
|
||||
assert len(pattern.preserve_fields) <= 10
|
||||
|
||||
|
||||
@@ -259,7 +259,7 @@ class TestUserCountTracking:
|
||||
strategy="smart_sample",
|
||||
)
|
||||
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
assert pattern.user_count == 1
|
||||
assert len(pattern._seen_instance_hashes) == 1
|
||||
assert fresh_toin._instance_id in pattern._seen_instance_hashes
|
||||
@@ -280,7 +280,7 @@ class TestUserCountTracking:
|
||||
strategy="smart_sample",
|
||||
)
|
||||
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
assert pattern.user_count == 1 # Still 1
|
||||
|
||||
def test_instance_hashes_serialized_and_loaded(self):
|
||||
@@ -309,7 +309,9 @@ class TestUserCountTracking:
|
||||
# Load in new instance
|
||||
toin2 = ToolIntelligenceNetwork(TOINConfig(storage_path=storage_path))
|
||||
|
||||
pattern = toin2._patterns.get(("unknown", "unknown", signature.structure_hash))
|
||||
pattern = toin2._patterns.get(
|
||||
("global", "unknown", "unknown", signature.structure_hash)
|
||||
)
|
||||
assert pattern is not None
|
||||
assert pattern.user_count >= 1
|
||||
assert len(pattern._seen_instance_hashes) >= 1
|
||||
@@ -345,7 +347,7 @@ class TestUserCountTracking:
|
||||
|
||||
fresh_toin.import_patterns(import_data)
|
||||
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", sig_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", sig_hash)]
|
||||
# Should have local + 2 imported = 3
|
||||
assert pattern.user_count >= 3
|
||||
|
||||
@@ -374,7 +376,7 @@ class TestFieldRetrievalFrequencyWeighting:
|
||||
|
||||
# Record retrievals for "status" field
|
||||
status_hash = fresh_toin._hash_field_name("status")
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.field_retrieval_frequency = {
|
||||
status_hash: 50,
|
||||
fresh_toin._hash_field_name("category"): 10,
|
||||
@@ -403,7 +405,7 @@ class TestFieldRetrievalFrequencyWeighting:
|
||||
strategy="smart_sample",
|
||||
)
|
||||
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
field_a = fresh_toin._hash_field_name("field_a")
|
||||
field_b = fresh_toin._hash_field_name("field_b")
|
||||
field_c = fresh_toin._hash_field_name("field_c")
|
||||
@@ -450,7 +452,7 @@ class TestQueryContextUsage:
|
||||
)
|
||||
|
||||
# Low retrieval rate = aggressive compression
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.total_retrievals = 0
|
||||
|
||||
# Query with exhaustive keyword
|
||||
@@ -476,7 +478,7 @@ class TestQueryContextUsage:
|
||||
strategy="smart_sample",
|
||||
)
|
||||
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.total_retrievals = 0
|
||||
|
||||
hint = fresh_toin.get_recommendation(signature, "find every user")
|
||||
@@ -498,7 +500,7 @@ class TestQueryContextUsage:
|
||||
strategy="smart_sample",
|
||||
)
|
||||
|
||||
pattern = fresh_toin._patterns[("unknown", "unknown", signature.structure_hash)]
|
||||
pattern = fresh_toin._patterns[("global", "unknown", "unknown", signature.structure_hash)]
|
||||
pattern.total_retrievals = 0
|
||||
# Add a problematic query pattern
|
||||
pattern.common_query_patterns = ["status:*"]
|
||||
|
||||
@@ -4,9 +4,12 @@ Pins three guarantees:
|
||||
|
||||
1. `get_recommendation()` returns `None` and emits a `DeprecationWarning`
|
||||
exactly once per process. The request-time hint API is retired.
|
||||
2. The aggregation key is `(auth_mode, model_family, structure_hash)` —
|
||||
two patterns with the same `structure_hash` but different `auth_mode`
|
||||
or `model_family` are tracked as distinct rows in the TOIN store.
|
||||
2. The aggregation key is `(tenant_key, auth_mode, model_family,
|
||||
structure_hash)` — two patterns with the same `structure_hash` but
|
||||
different `auth_mode`, `model_family`, or `tenant_key` are tracked
|
||||
as distinct rows in the TOIN store. PR-F3 added `tenant_key` so two
|
||||
tenants on the same proxy don't cross-pollinate compression
|
||||
patterns.
|
||||
3. Recording a compression event does NOT alter the bytes SmartCrusher
|
||||
produces for an identical input. SmartCrusher is deterministic; TOIN
|
||||
only observes.
|
||||
@@ -22,6 +25,7 @@ import pytest
|
||||
from headroom.telemetry import (
|
||||
DEFAULT_AUTH_MODE,
|
||||
DEFAULT_MODEL_FAMILY,
|
||||
DEFAULT_TENANT_KEY,
|
||||
TOINConfig,
|
||||
ToolIntelligenceNetwork,
|
||||
ToolSignature,
|
||||
@@ -120,15 +124,18 @@ def test_aggregation_key_includes_auth_mode_and_model_family():
|
||||
)
|
||||
|
||||
sig_hash = sig.structure_hash
|
||||
assert ("payg", "claude-3-5", sig_hash) in toin._patterns
|
||||
assert ("oauth", "claude-3-5", sig_hash) in toin._patterns
|
||||
assert ("payg", "gpt-4o", sig_hash) in toin._patterns
|
||||
# PR-F3: keys are 4-tuples — `(tenant_key, auth_mode, model_family,
|
||||
# sig_hash)`. With no contextvar set the default is "global".
|
||||
assert (DEFAULT_TENANT_KEY, "payg", "claude-3-5", sig_hash) in toin._patterns
|
||||
assert (DEFAULT_TENANT_KEY, "oauth", "claude-3-5", sig_hash) in toin._patterns
|
||||
assert (DEFAULT_TENANT_KEY, "payg", "gpt-4o", sig_hash) in toin._patterns
|
||||
# Three distinct slices, each with sample_size=1.
|
||||
assert len(toin._patterns) == 3
|
||||
for key, pattern in toin._patterns.items():
|
||||
assert pattern.auth_mode == key[0]
|
||||
assert pattern.model_family == key[1]
|
||||
assert pattern.tool_signature_hash == key[2]
|
||||
assert pattern.tenant_key == key[0]
|
||||
assert pattern.auth_mode == key[1]
|
||||
assert pattern.model_family == key[2]
|
||||
assert pattern.tool_signature_hash == key[3]
|
||||
assert pattern.sample_size == 1
|
||||
|
||||
|
||||
@@ -146,9 +153,15 @@ def test_aggregation_key_defaults_to_unknown_when_caller_omits_tenant():
|
||||
strategy="smart_crusher",
|
||||
)
|
||||
|
||||
expected_key = (DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig.structure_hash)
|
||||
expected_key = (
|
||||
DEFAULT_TENANT_KEY,
|
||||
DEFAULT_AUTH_MODE,
|
||||
DEFAULT_MODEL_FAMILY,
|
||||
sig.structure_hash,
|
||||
)
|
||||
assert expected_key in toin._patterns
|
||||
pattern = toin._patterns[expected_key]
|
||||
assert pattern.tenant_key == DEFAULT_TENANT_KEY
|
||||
assert pattern.auth_mode == DEFAULT_AUTH_MODE
|
||||
assert pattern.model_family == DEFAULT_MODEL_FAMILY
|
||||
|
||||
@@ -168,16 +181,185 @@ def test_storage_round_trip_preserves_aggregation_key(tmp_path: Path):
|
||||
strategy="smart_crusher",
|
||||
auth_mode="oauth",
|
||||
model_family="gpt-4o",
|
||||
tenant_key="cust_xyz",
|
||||
)
|
||||
toin1.save()
|
||||
|
||||
toin2 = ToolIntelligenceNetwork(TOINConfig(storage_path=str(storage)))
|
||||
key = ("oauth", "gpt-4o", sig.structure_hash)
|
||||
key = ("cust_xyz", "oauth", "gpt-4o", sig.structure_hash)
|
||||
assert key in toin2._patterns
|
||||
assert toin2._patterns[key].tenant_key == "cust_xyz"
|
||||
assert toin2._patterns[key].auth_mode == "oauth"
|
||||
assert toin2._patterns[key].model_family == "gpt-4o"
|
||||
|
||||
|
||||
# ── PR-F3: per-tenant isolation property ───────────────────────────────────
|
||||
|
||||
|
||||
def test_tenants_do_not_cross_pollinate_patterns():
|
||||
"""Two tenants compressing identical tool outputs see two isolated pools.
|
||||
|
||||
This is the load-bearing F3 invariant: tenant A's compression
|
||||
patterns must be invisible to tenant B's `get_pattern` calls and
|
||||
vice-versa. The threat model F3 closes is: a multi-tenant proxy
|
||||
that learns a shared pattern pool would cross-pollinate
|
||||
compression decisions between distinct customers, and (worse)
|
||||
leak structural fingerprints across tenancy boundaries when TOIN
|
||||
dumps are shared.
|
||||
"""
|
||||
toin = ToolIntelligenceNetwork()
|
||||
sig = ToolSignature.from_items([{"id": "1", "status": "ok"}])
|
||||
sig_hash = sig.structure_hash
|
||||
|
||||
# Tenant A: 3 compressions, no retrievals (perfect-compression
|
||||
# signal — pattern.skip_compression_recommended will stay False
|
||||
# but strategy_success_rate will trend high).
|
||||
for _ in range(3):
|
||||
toin.record_compression(
|
||||
tool_signature=sig,
|
||||
original_count=10,
|
||||
compressed_count=2,
|
||||
original_tokens=1000,
|
||||
compressed_tokens=200,
|
||||
strategy="smart_crusher",
|
||||
tenant_key="tenant_a",
|
||||
)
|
||||
|
||||
# Tenant B: 3 compressions PLUS 3 retrievals (over-compressed
|
||||
# signal — strategy_success_rate will trend down). If isolation
|
||||
# is broken, tenant A's "smart_crusher worked great" signal would
|
||||
# contaminate tenant B's pattern.
|
||||
for _ in range(3):
|
||||
toin.record_compression(
|
||||
tool_signature=sig,
|
||||
original_count=10,
|
||||
compressed_count=2,
|
||||
original_tokens=1000,
|
||||
compressed_tokens=200,
|
||||
strategy="smart_crusher",
|
||||
tenant_key="tenant_b",
|
||||
)
|
||||
for _ in range(3):
|
||||
toin.record_retrieval(
|
||||
tool_signature_hash=sig_hash,
|
||||
retrieval_type="full",
|
||||
strategy="smart_crusher",
|
||||
tenant_key="tenant_b",
|
||||
)
|
||||
|
||||
# Two distinct keys must exist, each with its own dataclass.
|
||||
key_a = ("tenant_a", DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)
|
||||
key_b = ("tenant_b", DEFAULT_AUTH_MODE, DEFAULT_MODEL_FAMILY, sig_hash)
|
||||
assert key_a in toin._patterns
|
||||
assert key_b in toin._patterns
|
||||
|
||||
pattern_a = toin._patterns[key_a]
|
||||
pattern_b = toin._patterns[key_b]
|
||||
|
||||
# Compression counts isolated: tenant A saw 3 comps + 0 retrievals.
|
||||
assert pattern_a.total_compressions == 3
|
||||
assert pattern_a.total_retrievals == 0
|
||||
|
||||
# Tenant B saw 3 comps + 3 retrievals — divergent learning signal.
|
||||
assert pattern_b.total_compressions == 3
|
||||
assert pattern_b.total_retrievals == 3
|
||||
|
||||
# The success rate diverges: tenant A's "smart_crusher" gets
|
||||
# rewarded; tenant B's gets penalized for triggering retrievals.
|
||||
rate_a = pattern_a.strategy_success_rates.get("smart_crusher", 0.0)
|
||||
rate_b = pattern_b.strategy_success_rates.get("smart_crusher", 0.0)
|
||||
assert rate_a > rate_b, (
|
||||
f"PR-F3: tenant A and B success rates must diverge (tenant A: {rate_a}, tenant B: {rate_b})"
|
||||
)
|
||||
|
||||
# `get_pattern` honors tenant_key — tenant A's lookup returns
|
||||
# tenant A's pattern, not tenant B's.
|
||||
fetched_a = toin.get_pattern(sig_hash, tenant_key="tenant_a")
|
||||
fetched_b = toin.get_pattern(sig_hash, tenant_key="tenant_b")
|
||||
assert fetched_a is not None
|
||||
assert fetched_b is not None
|
||||
assert fetched_a.total_retrievals == 0
|
||||
assert fetched_b.total_retrievals == 3
|
||||
|
||||
|
||||
def test_tenant_key_default_is_global_when_no_caller_provides_one():
|
||||
"""Pre-F3 callers and CLI / batch jobs land in the literal "global" slice.
|
||||
|
||||
Pins the migration story: when no tenant_key is supplied (and no
|
||||
contextvar is set), patterns aggregate under the literal
|
||||
``"global"`` namespace, which is exactly where pre-F3 patterns
|
||||
live after the legacy 3-tuple → 4-tuple promotion.
|
||||
"""
|
||||
toin = ToolIntelligenceNetwork()
|
||||
sig = ToolSignature.from_items([{"id": "1"}])
|
||||
|
||||
toin.record_compression(
|
||||
tool_signature=sig,
|
||||
original_count=10,
|
||||
compressed_count=5,
|
||||
original_tokens=1000,
|
||||
compressed_tokens=500,
|
||||
strategy="smart_crusher",
|
||||
)
|
||||
|
||||
keys = list(toin._patterns.keys())
|
||||
assert len(keys) == 1
|
||||
assert keys[0][0] == DEFAULT_TENANT_KEY == "global"
|
||||
|
||||
|
||||
def test_tenant_key_reads_from_contextvar_when_unspecified():
|
||||
"""When caller doesn't pass tenant_key, TOIN reads the request ContextVar."""
|
||||
from headroom.proxy.tenant_key import set_request_tenant_key
|
||||
|
||||
toin = ToolIntelligenceNetwork()
|
||||
sig = ToolSignature.from_items([{"id": "1"}])
|
||||
|
||||
set_request_tenant_key("tenant_from_ctx")
|
||||
try:
|
||||
toin.record_compression(
|
||||
tool_signature=sig,
|
||||
original_count=10,
|
||||
compressed_count=5,
|
||||
original_tokens=1000,
|
||||
compressed_tokens=500,
|
||||
strategy="smart_crusher",
|
||||
)
|
||||
finally:
|
||||
set_request_tenant_key(None)
|
||||
|
||||
# The contextvar's value made it into the key — not the default
|
||||
# "global". This is the actual integration path for the proxy.
|
||||
keys = list(toin._patterns.keys())
|
||||
assert len(keys) == 1
|
||||
assert keys[0][0] == "tenant_from_ctx"
|
||||
|
||||
|
||||
def test_explicit_tenant_key_overrides_contextvar():
|
||||
"""An explicit tenant_key arg always wins over the ContextVar."""
|
||||
from headroom.proxy.tenant_key import set_request_tenant_key
|
||||
|
||||
toin = ToolIntelligenceNetwork()
|
||||
sig = ToolSignature.from_items([{"id": "1"}])
|
||||
|
||||
set_request_tenant_key("from_contextvar")
|
||||
try:
|
||||
toin.record_compression(
|
||||
tool_signature=sig,
|
||||
original_count=10,
|
||||
compressed_count=5,
|
||||
original_tokens=1000,
|
||||
compressed_tokens=500,
|
||||
strategy="smart_crusher",
|
||||
tenant_key="explicit_arg",
|
||||
)
|
||||
finally:
|
||||
set_request_tenant_key(None)
|
||||
|
||||
keys = list(toin._patterns.keys())
|
||||
assert len(keys) == 1
|
||||
assert keys[0][0] == "explicit_arg"
|
||||
|
||||
|
||||
def test_record_does_not_alter_compression_decision():
|
||||
"""SmartCrusher output is byte-identical regardless of TOIN observation state.
|
||||
|
||||
|
||||
@@ -97,6 +97,7 @@ def test_publish_command_writes_toml(fresh_toin: ToolIntelligenceNetwork, tmp_pa
|
||||
|
||||
row = rec_list[0]
|
||||
assert set(row.keys()) == {
|
||||
"tenant_key",
|
||||
"auth_mode",
|
||||
"model_family",
|
||||
"structure_hash",
|
||||
@@ -105,6 +106,8 @@ def test_publish_command_writes_toml(fresh_toin: ToolIntelligenceNetwork, tmp_pa
|
||||
"confidence",
|
||||
"observations",
|
||||
}
|
||||
# PR-F3: tenant_key defaults to "global" when no explicit one is set.
|
||||
assert row["tenant_key"] == "global"
|
||||
assert row["auth_mode"] == "payg"
|
||||
assert row["model_family"] == "claude-3-5"
|
||||
assert row["structure_hash"] == sig.structure_hash
|
||||
|
||||
Reference in New Issue
Block a user