mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(tool-search): make the core-tools override authoritative and the warning recurrent (#3745)
Three defects found while tracing why a customer benchmark showed **+108% cost** with Headroom in front of Claude Code. Follow-on to #3739. ## 1. The override was not authoritative on the OpenAI path `helpers.py` unioned `terminal` into the resident set *after* applying the operator's override. Setting `HEADROOM_TOOL_SEARCH_CORE_TOOLS` to a list that excluded `terminal` got it back anyway, silently — the one tool most likely to carry a large schema. `resolved_core_tools` now takes provider additions as an `extra` argument folded into the **default** only. An explicit override replaces everything, provider additions included. That is what an override means. ## 2. The warning armed once per process The "client tool search is disabled" warning was already `logger.warning` — the defect was that it armed once per process. A long-lived proxy reported the condition on the first request and went quiet forever. **Twelve benchmark runs produced a single log line**, which is why this took so long to find. It now re-arms hourly. ## 3. One knob, two spellings Core read `HEADROOM_TOOL_SEARCH_CORE_TOOLS`; the plugin read `HEADROOM_TOOL_SEARCH_CORE`. Both now resolve to the same setting — canonical wins, legacy is honoured and warns once naming the replacement. No existing configuration breaks. ## Testing 40 tests in `test_tool_search_client_deferral.py`, 1115 passing across the related suites. ruff clean. ## Note on defaults This PR does **not** change what is resident by default. Claude Code defers all its built-ins as of v2.1.69, and matching that is the obvious next step — but there are open client-side bugs against it (Grep/Glob disappearing on macOS/Linux under `ENABLE_TOOL_SEARCH=true`, strict endpoints 400ing the first request, signed-thinking-block errors on Opus 4.8). Those are upstream, not ours; they are the reason this PR makes the knob a real off-switch first. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ### Update after self-review Two live bugs, **one of them introduced by this PR**: ``` HEADROOM_TOOL_SEARCH_CORE_TOOLS="bash, read, terminal" -> resident set: [' read', ' terminal', 'bash'] -> read: DEFERRED, terminal: DEFERRED ``` The override never stripped whitespace — `part.strip()` gated only the emptiness test, while the key function lowercases and strips leading underscores but not spaces. So the natural spelling of a list defers exactly the tools you asked to pin. This was latent before: the OpenAI injector re-unioned its resident names *after* the override and masked it. **Removing that union in this PR is what made it bite.** Separately, an override could defer the client's own `ToolSearch` — Claude Code reaches tools held in a local registry through it and nothing else can, so an override that omitted it orphaned that whole category. A search-tool floor now survives every override. Also: the hourly window was stamped only when the hint **fired**, so once the operator fixed the condition the stamp stopped advancing and every later request paid the full O(tools) scan for the life of the process — a hot-path regression this PR added. It now throttles *scans*, and claiming the slot before scanning collapses the old two-call protocol. The legacy-var flag is now lock-guarded with a reset helper, the gateway reference hook honours `resolved_core_tools()` instead of the raw default (it ignored the override entirely), and the hint test no longer monkeypatches stdlib `time.monotonic` process-wide. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
a2bf5ed1ba
commit
a9757c9c7e
@@ -2379,8 +2379,8 @@ class AnthropicHandlerMixin:
|
||||
# loads them all into local context. That is a client-side decision
|
||||
# we cannot reverse from here, so emit a single actionable hint for
|
||||
# users who launch `claude` manually (the wrap path sets the env var).
|
||||
# Gate on the cheap one-time flag first so the detection scan stops
|
||||
# running once the hint has fired; never let it break a request.
|
||||
# Gate on the cheap throttle first so the detection scan runs at most
|
||||
# once per interval; never let it break a request.
|
||||
from headroom.proxy.helpers import tool_search_hint_pending
|
||||
|
||||
if tool_search_hint_pending():
|
||||
@@ -2388,16 +2388,18 @@ class AnthropicHandlerMixin:
|
||||
from headroom.proxy.helpers import (
|
||||
claude_code_tool_search_inactive,
|
||||
format_tool_search_disabled_hint,
|
||||
take_tool_search_hint_slot,
|
||||
take_tool_search_scan_slot,
|
||||
)
|
||||
|
||||
if (
|
||||
claude_code_tool_search_inactive(
|
||||
client=client,
|
||||
tools=tools,
|
||||
anthropic_beta=request.headers.get("anthropic-beta"),
|
||||
)
|
||||
and take_tool_search_hint_slot()
|
||||
# Claim the slot BEFORE scanning, so the window closes
|
||||
# whatever the scan finds. Claiming it only on a positive
|
||||
# result left the gate open forever once the operator fixed
|
||||
# the condition, re-scanning the whole tool array on every
|
||||
# request for the life of the process.
|
||||
if take_tool_search_scan_slot() and claude_code_tool_search_inactive(
|
||||
client=client,
|
||||
tools=tools,
|
||||
anthropic_beta=request.headers.get("anthropic-beta"),
|
||||
):
|
||||
logger.warning(
|
||||
"[%s] %s", request_id, format_tool_search_disabled_hint(tools)
|
||||
|
||||
+124
-26
@@ -3148,7 +3148,18 @@ _TOOL_SEARCH_TOOL_TYPE_PREFIX = "tool_search_tool_"
|
||||
_TOOL_SEARCH_BETA_MARKERS = ("advanced-tool-use", "tool-search-tool")
|
||||
|
||||
_tool_search_hint_lock = threading.Lock()
|
||||
_tool_search_hint_emitted = False
|
||||
#: Re-arm interval for the tool-search-disabled warning. It used to fire exactly
|
||||
#: once per process, which on a proxy that stays up for weeks means one line in
|
||||
#: the log at startup and silence thereafter — the condition it reports persists
|
||||
#: for the whole life of the deployment and costs tokens on every single request,
|
||||
#: so a single line is not proportionate to it. Hourly is frequent enough that an
|
||||
#: operator tailing logs will see it, rare enough that it is not noise.
|
||||
_TOOL_SEARCH_HINT_INTERVAL_S = 3600.0
|
||||
_tool_search_hint_last: float | None = None
|
||||
# Module-level indirection over the clock so tests can drive it without
|
||||
# monkeypatching stdlib ``time.monotonic``, which would freeze it for every
|
||||
# other thread in the process. Same pattern as savings_tracker.
|
||||
_monotonic = time.monotonic
|
||||
|
||||
|
||||
def claude_code_tool_search_inactive(
|
||||
@@ -3202,36 +3213,46 @@ def format_tool_search_disabled_hint(tools: list[Any]) -> str:
|
||||
|
||||
|
||||
def tool_search_hint_pending() -> bool:
|
||||
"""Cheap, lock-free check of whether the one-time hint may still fire.
|
||||
"""Cheap, lock-free check of whether the detection scan may run now.
|
||||
|
||||
Lets the request hot path skip the (O(number-of-tools)) detection scan on
|
||||
every request once the hint has already been emitted. A benign race here
|
||||
only costs one extra detection scan, never a duplicate warning — the
|
||||
actual one-shot guarantee lives in :func:`take_tool_search_hint_slot`.
|
||||
Lets the request hot path skip the O(number-of-tools) scan while the hint
|
||||
is throttled. A benign race here only costs one extra scan, never a
|
||||
duplicate warning — the rate limit itself lives in
|
||||
:func:`take_tool_search_scan_slot`.
|
||||
"""
|
||||
return not _tool_search_hint_emitted
|
||||
last = _tool_search_hint_last
|
||||
return last is None or (_monotonic() - last) >= _TOOL_SEARCH_HINT_INTERVAL_S
|
||||
|
||||
|
||||
def take_tool_search_hint_slot() -> bool:
|
||||
"""Return ``True`` exactly once per process, gating the one-time hint.
|
||||
def take_tool_search_scan_slot() -> bool:
|
||||
"""Claim the right to run the detection scan, at most once per interval.
|
||||
|
||||
Thread-safe so concurrent requests cannot each emit the warning.
|
||||
The window governs SCANS, not emissions, and that distinction is
|
||||
load-bearing. Stamping only when the hint actually fires means that once the
|
||||
operator FIXES the condition the stamp stops advancing, ``pending`` stays
|
||||
true forever, and every subsequent request pays the full scan over the
|
||||
client's tool array for the life of the process — the opposite of the gate's
|
||||
purpose, and worst on exactly the large tool surfaces this targets.
|
||||
|
||||
Claiming the slot before the scan also collapses the old two-call protocol
|
||||
into one, so there is no window in which two threads both scan and both
|
||||
emit.
|
||||
"""
|
||||
global _tool_search_hint_emitted
|
||||
if _tool_search_hint_emitted:
|
||||
global _tool_search_hint_last
|
||||
if not tool_search_hint_pending():
|
||||
return False
|
||||
with _tool_search_hint_lock:
|
||||
if _tool_search_hint_emitted:
|
||||
if not tool_search_hint_pending():
|
||||
return False
|
||||
_tool_search_hint_emitted = True
|
||||
_tool_search_hint_last = _monotonic()
|
||||
return True
|
||||
|
||||
|
||||
def reset_tool_search_hint_state() -> None:
|
||||
"""Reset the one-time hint guard. Test helper only."""
|
||||
global _tool_search_hint_emitted
|
||||
"""Reset the hint rate limit. Test helper only."""
|
||||
global _tool_search_hint_last
|
||||
with _tool_search_hint_lock:
|
||||
_tool_search_hint_emitted = False
|
||||
_tool_search_hint_last = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -3248,6 +3269,14 @@ def reset_tool_search_hint_state() -> None:
|
||||
# stays callable. Deterministic output → the tools prefix still prompt-caches.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Resident no matter what the operator's override says. The client's own
|
||||
# tool-search tool is the one thing that must never be deferred: it is what
|
||||
# loads the tools it resolves, and Claude Code uses it to reach tools held in a
|
||||
# local registry (TaskCreate, WebFetch, ...) that nothing else can reach. An
|
||||
# override says which ORDINARY tools stay inline, so letting it drop this would
|
||||
# silently orphan a whole category rather than defer it.
|
||||
_CLIENT_SIDE_SEARCH_FLOOR = frozenset({"toolsearch"})
|
||||
|
||||
# Core coding tools kept non-deferred so routine edit/read/run loops never pay a
|
||||
# search round-trip. Everything else (Slack/Linear/Sentry/Notion/Snowflake/…) is
|
||||
# deferred and loaded on demand. Anthropic recommends keeping the 3–5 (here a few
|
||||
@@ -3281,9 +3310,65 @@ _TOOL_SEARCH_CORE_TOOLS = frozenset(
|
||||
)
|
||||
|
||||
|
||||
def resolved_core_tools() -> frozenset[str]:
|
||||
#: Canonical name for the resident-tool override, and the legacy name the
|
||||
#: tool-search plugin shipped for the same idea. Two variables for one knob is a
|
||||
#: trap: an operator sets the one they know, the other path silently keeps its
|
||||
#: own list, and the two providers disagree about which tools are visible. Both
|
||||
#: are read here so either spelling works everywhere; the canonical one wins.
|
||||
CORE_TOOLS_ENV = "HEADROOM_TOOL_SEARCH_CORE_TOOLS"
|
||||
CORE_TOOLS_ENV_LEGACY = "HEADROOM_TOOL_SEARCH_CORE"
|
||||
|
||||
_core_tools_legacy_lock = threading.Lock()
|
||||
_core_tools_legacy_warned = False
|
||||
|
||||
|
||||
def reset_core_tools_legacy_warn_state() -> None:
|
||||
"""Re-arm the legacy-variable warning. Test helper only.
|
||||
|
||||
Without this the flag leaks between tests in a process, so a caplog
|
||||
assertion on this warning passes alone and fails depending on ordering.
|
||||
"""
|
||||
global _core_tools_legacy_warned
|
||||
with _core_tools_legacy_lock:
|
||||
_core_tools_legacy_warned = False
|
||||
|
||||
|
||||
def _core_tools_override() -> str | None:
|
||||
"""Return the resident-tool override from either env var, canonical first."""
|
||||
global _core_tools_legacy_warned
|
||||
raw = os.environ.get(CORE_TOOLS_ENV)
|
||||
if raw is not None:
|
||||
return raw
|
||||
legacy = os.environ.get(CORE_TOOLS_ENV_LEGACY)
|
||||
should_warn = False
|
||||
if legacy is not None:
|
||||
# Check-and-set under the lock: unguarded, two threads racing on their
|
||||
# first request could both warn.
|
||||
with _core_tools_legacy_lock:
|
||||
if not _core_tools_legacy_warned:
|
||||
_core_tools_legacy_warned = True
|
||||
should_warn = True
|
||||
if should_warn:
|
||||
logger.warning(
|
||||
"event=tool_search_core_env_legacy old=%s new=%s "
|
||||
"hint=honoring the legacy variable; rename it, both paths read the new one",
|
||||
CORE_TOOLS_ENV_LEGACY,
|
||||
CORE_TOOLS_ENV,
|
||||
)
|
||||
return legacy
|
||||
|
||||
|
||||
def resolved_core_tools(extra: frozenset[str] = frozenset()) -> frozenset[str]:
|
||||
"""Tools that stay resident, with a deployment override.
|
||||
|
||||
``extra`` carries a provider's own additions to the default resident set
|
||||
(the OpenAI Responses path keeps ``terminal`` resident alongside the shared
|
||||
coding loop). It is folded into the DEFAULT only. An explicit
|
||||
``HEADROOM_TOOL_SEARCH_CORE_TOOLS`` replaces the whole set, additions
|
||||
included -- otherwise the knob would silently fail to defer a tool the
|
||||
operator had just asked to defer, and the two provider paths would honour
|
||||
the same variable differently.
|
||||
|
||||
The default keeps the coding loop resident so routine edit/read/run never
|
||||
pays a search round-trip. That default is now out of step with Claude Code,
|
||||
which since v2.1.69 defers its built-ins too and thereby cuts built-in
|
||||
@@ -3299,10 +3384,21 @@ def resolved_core_tools() -> frozenset[str]:
|
||||
against real traffic instead of a guess shipped as a default.
|
||||
"""
|
||||
|
||||
raw = os.environ.get("HEADROOM_TOOL_SEARCH_CORE_TOOLS")
|
||||
raw = _core_tools_override()
|
||||
if raw is None:
|
||||
return _TOOL_SEARCH_CORE_TOOLS
|
||||
return frozenset(_tool_search_resident_key(part) for part in raw.split(",") if part.strip())
|
||||
if not extra:
|
||||
return _TOOL_SEARCH_CORE_TOOLS
|
||||
return _TOOL_SEARCH_CORE_TOOLS | {_tool_search_resident_key(name) for name in extra}
|
||||
# ``part.strip()`` before keying, not just for the emptiness test: the key
|
||||
# function lowercases and strips leading underscores but NOT spaces, so the
|
||||
# natural spelling "bash, read, terminal" used to resolve to {" read",
|
||||
# " terminal", "bash"} and defer the two tools the operator asked to pin.
|
||||
named = {_tool_search_resident_key(part.strip()) for part in raw.split(",") if part.strip()}
|
||||
# The client's own tool-search tool survives every override. Deferring it
|
||||
# hides the only thing that can load the tools it resolves -- including
|
||||
# tools the client keeps in a local registry, which nothing else can reach
|
||||
# -- so an override that omits it would silently orphan them.
|
||||
return frozenset(named | _CLIENT_SIDE_SEARCH_FLOOR)
|
||||
|
||||
|
||||
_TOOL_SEARCH_DEFAULT_TYPE = "tool_search_tool_regex_20251119"
|
||||
@@ -3407,7 +3503,7 @@ def _client_tool_search_names() -> frozenset[str]:
|
||||
if not extra.strip():
|
||||
return _CLIENT_TOOL_SEARCH_NAMES
|
||||
return _CLIENT_TOOL_SEARCH_NAMES | {
|
||||
_tool_search_resident_key(part) for part in extra.split(",") if part.strip()
|
||||
_tool_search_resident_key(part.strip()) for part in extra.split(",") if part.strip()
|
||||
}
|
||||
|
||||
|
||||
@@ -3933,7 +4029,7 @@ def inject_tool_search_deferral_openai(
|
||||
stays valid; the injected search tool is itself resident.
|
||||
"""
|
||||
if core_tools is None:
|
||||
core_tools = resolved_core_tools()
|
||||
core_tools = resolved_core_tools(_OPENAI_TOOL_SEARCH_RESIDENT_NAMES)
|
||||
if not openai_tool_search_client_supported(client):
|
||||
return tools
|
||||
if not _model_supports_openai_tool_search(model):
|
||||
@@ -3954,9 +4050,11 @@ def inject_tool_search_deferral_openai(
|
||||
deferred = 0
|
||||
# Normalize for the same reason as the Anthropic path above: clients may use
|
||||
# different casing or a leading namespace marker for the same resident tool.
|
||||
resident_keys = {_tool_search_resident_key(name) for name in core_tools} | {
|
||||
_tool_search_resident_key(name) for name in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
|
||||
}
|
||||
# No unconditional union with _OPENAI_TOOL_SEARCH_RESIDENT_NAMES here: it is
|
||||
# folded into the default by resolved_core_tools() above, so an explicit
|
||||
# override drops it too. Unioning it in at this point would pin ``terminal``
|
||||
# resident even when the operator set the core set to something without it.
|
||||
resident_keys = {_tool_search_resident_key(name) for name in core_tools}
|
||||
for tool in tools:
|
||||
if not isinstance(tool, dict):
|
||||
out.append(tool)
|
||||
|
||||
@@ -25,9 +25,9 @@ from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from headroom.proxy.helpers import (
|
||||
_TOOL_SEARCH_CORE_TOOLS,
|
||||
anthropic_model_is_first_party,
|
||||
inject_tool_search_deferral,
|
||||
resolved_core_tools,
|
||||
tools_are_anthropic_shaped,
|
||||
)
|
||||
from headroom.proxy.turn_hooks import register_turn_hook
|
||||
@@ -57,7 +57,12 @@ class DeferralHook:
|
||||
return
|
||||
if not tools_are_anthropic_shaped(ctx.tools):
|
||||
return
|
||||
after = inject_tool_search_deferral(ctx.tools, core_tools=_TOOL_SEARCH_CORE_TOOLS)
|
||||
# resolved_core_tools(), not the raw default: passing the constant
|
||||
# ignored HEADROOM_TOOL_SEARCH_CORE_TOOLS entirely, so an operator who
|
||||
# set it got the built-in list on this path and their own on the proxy
|
||||
# path -- the two-paths-disagree failure the one-knob change exists to
|
||||
# prevent, reintroduced by the reference hook.
|
||||
after = inject_tool_search_deferral(ctx.tools, core_tools=resolved_core_tools())
|
||||
if after is ctx.tools:
|
||||
return
|
||||
ctx.tools = after
|
||||
|
||||
@@ -22,7 +22,7 @@ from headroom.proxy.helpers import (
|
||||
claude_code_tool_search_inactive,
|
||||
format_tool_search_disabled_hint,
|
||||
reset_tool_search_hint_state,
|
||||
take_tool_search_hint_slot,
|
||||
take_tool_search_scan_slot,
|
||||
tool_search_hint_pending,
|
||||
)
|
||||
|
||||
@@ -159,11 +159,11 @@ def test_hint_slot_fires_once() -> None:
|
||||
reset_tool_search_hint_state()
|
||||
try:
|
||||
assert tool_search_hint_pending() is True
|
||||
assert take_tool_search_hint_slot() is True
|
||||
assert take_tool_search_scan_slot() is True
|
||||
# Once consumed, the cheap gate flips so the hot path stops scanning.
|
||||
assert tool_search_hint_pending() is False
|
||||
assert take_tool_search_hint_slot() is False
|
||||
assert take_tool_search_hint_slot() is False
|
||||
assert take_tool_search_scan_slot() is False
|
||||
assert take_tool_search_scan_slot() is False
|
||||
finally:
|
||||
reset_tool_search_hint_state()
|
||||
|
||||
|
||||
@@ -184,7 +184,7 @@ def test_resident_set_is_overridable(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""So deferring built-ins can be measured instead of guessed."""
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", "bash,read")
|
||||
|
||||
assert resolved_core_tools() == frozenset({"bash", "read"})
|
||||
assert resolved_core_tools() == frozenset({"bash", "read", "toolsearch"})
|
||||
|
||||
tools = [
|
||||
{"name": "bash", "input_schema": {}},
|
||||
@@ -335,3 +335,173 @@ def test_openai_toolsearch_name_alone_does_not_stand_us_down() -> None:
|
||||
by_name = {t.get("name"): t for t in out if isinstance(t, dict)}
|
||||
assert by_name["ToolSearch"].get("defer_loading") is None
|
||||
assert by_name["slack_0"].get("defer_loading") is True
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the resident override must mean the same thing on both paths
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_openai_keeps_terminal_resident_by_default() -> None:
|
||||
tools = [_fn("terminal"), _fn("bash"), *[_fn(f"slack_{i}") for i in range(14)]]
|
||||
|
||||
by_name = {
|
||||
t.get("name"): t
|
||||
for t in inject_tool_search_deferral_openai(tools, "gpt-5.5")
|
||||
if isinstance(t, dict)
|
||||
}
|
||||
|
||||
assert by_name["terminal"].get("defer_loading") is None
|
||||
assert by_name["bash"].get("defer_loading") is None
|
||||
assert by_name["slack_0"].get("defer_loading") is True
|
||||
|
||||
|
||||
def test_openai_override_can_drop_terminal(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""An explicit core set is authoritative — provider additions included.
|
||||
|
||||
``terminal`` used to be unioned in unconditionally, so setting the knob to a
|
||||
set without it still pinned it resident. The two provider paths then honoured
|
||||
the same variable differently, and an operator asking to defer a tool was
|
||||
silently refused.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", "bash,read")
|
||||
tools = [_fn("terminal"), _fn("bash"), _fn("grep"), *[_fn(f"slack_{i}") for i in range(14)]]
|
||||
|
||||
by_name = {
|
||||
t.get("name"): t
|
||||
for t in inject_tool_search_deferral_openai(tools, "gpt-5.5")
|
||||
if isinstance(t, dict)
|
||||
}
|
||||
|
||||
assert by_name["bash"].get("defer_loading") is None
|
||||
assert by_name["terminal"].get("defer_loading") is True
|
||||
assert by_name["grep"].get("defer_loading") is True
|
||||
|
||||
|
||||
def test_default_resident_set_is_unchanged_for_anthropic() -> None:
|
||||
"""The OpenAI addition must not leak into the Anthropic default."""
|
||||
assert "terminal" not in resolved_core_tools()
|
||||
assert "terminal" in resolved_core_tools(frozenset({"terminal"}))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the tool-search-disabled warning must not go quiet forever
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_hint_rearms_after_the_interval(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""One line per process is not proportionate to a permanent condition.
|
||||
|
||||
The warning reports a deployment that has its client's tool deferral turned
|
||||
off — which costs tokens on every request for the life of the deployment.
|
||||
Firing once and never again means a proxy up for weeks says it at startup
|
||||
and is silent through everything after.
|
||||
"""
|
||||
from headroom.proxy import helpers as H
|
||||
|
||||
H.reset_tool_search_hint_state()
|
||||
try:
|
||||
clock = {"t": 1000.0}
|
||||
monkeypatch.setattr(H, "_monotonic", lambda: clock["t"])
|
||||
|
||||
assert H.take_tool_search_scan_slot() is True
|
||||
assert H.take_tool_search_scan_slot() is False
|
||||
|
||||
clock["t"] += H._TOOL_SEARCH_HINT_INTERVAL_S - 1
|
||||
assert H.take_tool_search_scan_slot() is False, "must not re-arm early"
|
||||
|
||||
clock["t"] += 2
|
||||
assert H.take_tool_search_scan_slot() is True, "must re-arm after the interval"
|
||||
finally:
|
||||
H.reset_tool_search_hint_state()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# one knob, two spellings
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_legacy_env_var_is_honoured(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The plugin shipped HEADROOM_TOOL_SEARCH_CORE for the same idea.
|
||||
|
||||
Two variables for one knob means an operator sets the one they know and the
|
||||
other path silently keeps its own list, so the two providers disagree about
|
||||
which tools are visible. Either spelling now works everywhere.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE", "bash,read")
|
||||
|
||||
assert resolved_core_tools() == frozenset({"bash", "read", "toolsearch"})
|
||||
|
||||
|
||||
def test_canonical_env_var_wins_over_legacy(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE", "bash,read")
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", "grep")
|
||||
|
||||
assert resolved_core_tools() == frozenset({"grep", "toolsearch"})
|
||||
|
||||
|
||||
def test_neither_set_keeps_the_default(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.delenv("HEADROOM_TOOL_SEARCH_CORE", raising=False)
|
||||
monkeypatch.delenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", raising=False)
|
||||
|
||||
assert "bash" in resolved_core_tools()
|
||||
assert "read" in resolved_core_tools()
|
||||
|
||||
|
||||
def test_empty_override_defers_everything_non_typed(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""An explicit empty set is a real instruction, not an unset variable."""
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", "")
|
||||
|
||||
assert resolved_core_tools() == frozenset({"toolsearch"})
|
||||
|
||||
|
||||
def test_the_override_tolerates_spaces_after_commas(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The natural spelling of a list must not defer the tools it names.
|
||||
|
||||
The key function lowercases and strips leading underscores but not spaces,
|
||||
so ``"bash, read, terminal"`` resolved to ``{" read", " terminal", "bash"}``
|
||||
and deferred exactly the two tools the operator asked to keep resident.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", "bash, read , terminal")
|
||||
|
||||
assert resolved_core_tools() == frozenset({"bash", "read", "terminal", "toolsearch"})
|
||||
|
||||
|
||||
def test_an_override_cannot_defer_the_clients_search_tool(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Deferring ToolSearch hides the only thing that can load what it resolves.
|
||||
|
||||
Claude Code reaches tools held in a local registry through it, and nothing
|
||||
else can. An override names which ORDINARY tools stay inline, so honouring
|
||||
one that omits this would orphan a whole category rather than defer it.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_TOOL_SEARCH_CORE_TOOLS", "Bash,Read")
|
||||
tools = [CLAUDE_CODE_TOOL_SEARCH, {"name": "Bash", "input_schema": {}}, *_mcp(14)]
|
||||
|
||||
by_name = {t.get("name"): t for t in inject_tool_search_deferral(tools) if isinstance(t, dict)}
|
||||
|
||||
assert by_name["ToolSearch"].get("defer_loading") is None
|
||||
assert by_name["mcp__srv0__do"].get("defer_loading") is True
|
||||
|
||||
|
||||
def test_the_scan_gate_closes_even_when_the_hint_does_not_fire(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Once the operator FIXES the condition, the scan must stop running.
|
||||
|
||||
Stamping the window only on emission meant the gate stayed open forever
|
||||
after the fix, so every later request paid the full O(tools) scan for the
|
||||
life of the process -- worst on the large tool surfaces this targets.
|
||||
"""
|
||||
from headroom.proxy import helpers as H
|
||||
|
||||
H.reset_tool_search_hint_state()
|
||||
try:
|
||||
clock = {"t": 1000.0}
|
||||
monkeypatch.setattr(H, "_monotonic", lambda: clock["t"])
|
||||
|
||||
assert H.take_tool_search_scan_slot() is True # scanned; found nothing
|
||||
assert H.tool_search_hint_pending() is False # ...and the gate closed anyway
|
||||
finally:
|
||||
H.reset_tool_search_hint_state()
|
||||
|
||||
Reference in New Issue
Block a user