fix(langchain): check optional integration availability (#4042)

## Description

Closes #4039.

The optional integration uses the existing LANGCHAIN_AVAILABLE guard;
installed LangChain binding precedence remains intact. Current head:
`ac9e49af43501d2b3bbb6d0a21cb07fe94030bc9`, normally updated onto main
`f1bc`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- Install langchain-core in the existing CI lint job, with the
optional-extra version range, so the normal mypy invocation covers the
integration. Remove the pytest-launched mypy subprocess.
- Keep our nested unwrap-precedence regression (sync and async) and two
real loopback HTTP cases: bind_tools and nested caller-overrides. Remove
the vendor tool-response/usage-parsing matrix, as requested by the
maintainer.
- Replace the private LiteLLM cache invalidator with the public
register_model API; preserve the independent cost_per_token billing
oracle and fixture/catalog restoration.
- Correct the compatibility doc numbering and document the normal typing
path in CONTRIBUTING.
- Preserve the published startup SDK/provider module and parent-package
restoration fixes.

## Testing

- [ ] Unit tests pass (pytest) — no whole-project suite pass claimed.
- [ ] Linting passes (ruff check .) — changed-file Ruff passed; no new
whole-repository Ruff claim.
- [x] Type checking passes (mypy headroom)
- [x] Manual testing performed

### Test Output

Current published head, Python 3.13.14, installed langchain-core 1.6.10,
langchain-openai 1.7.1, LiteLLM 1.105.0 and a fresh native build:

`python -m pytest tests/test_integrations/langchain/test_binding_http.py
tests/test_integrations/langchain/test_langchain_1x_compat.py
tests/test_startup_log_noise.py tests/test_xai_cache_economics.py -q`

```text
28 passed, 1 skipped in 14.73s
```

The skip is optional trafilatura unavailable. A SQLite ResourceWarning
was emitted at interpreter teardown; it is not suppressed or claimed
fixed.

Normal mypy command in an isolated lint environment with mypy 1.20.2 and
installed langchain-core: success, no issues in 569 source files. This
local lint environment uses Python 3.14; the runtime consumer suite uses
Python 3.13.14. Existing untyped-body/unused-ignore notes remain.

Ordinary push gates: Rust workspace tests, fresh native extension build,
181 critical Python tests, and commitlint passed. Two prior normal
publication attempts failed seven native parity cases; neither was
bypassed or repinned. Publication passed with a separate clean Cargo
target. This establishes the successful isolated build, not a proven
root cause for the earlier build discrepancy.

## Real Behavior Proof

- Environment: installed SDKs, actual localhost HTTP server, no
provider-network or billing requests.
- Exact command / steps: an independent runtime driver invoked
HeadroomChatModel with bind_tools and nested RunnableBinding objects
against the real local ChatOpenAI transport; the retained HTTP
regressions exercised both cases again on the published head.
- Observed result: two actual requests; bind_tools sent lookup; nested
caller temperature 0.3 won, and inner top_p 0.8 survived. The sync/async
unwrap regression retains the real RunnableBinding precedence check. The
pricing oracle and startup teardown regressions passed.
- Not tested: live provider invoices, paid requests, remote production
deployment, or current-head hosted CI completion. Historical CI/approval
is not attributed to this new head.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; optional integration guard and test
isolation.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no binding precedence, runtime
defaults, authentication or serialization changes; CI lint now
explicitly installs the existing optional typing dependency.
- Kill switch / disable path: existing optional extra remains optional;
no new runtime switch.
- Unsafe override required: no.
- Qualification impact: optional-extra typing and real binding consumers
are exercised; normal local native/critical gates passed.
- Rollback path: revert the bounded guard/CI/test edits; no
persistent-state migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I did not edit CHANGELOG.md — release-please owns it

## Additional Notes

AI-assisted without subagents. No force push, skipped hook, new approval
or merge. The reduced HTTP scope follows the maintainer's explicit
request:
https://github.com/headroomlabs-ai/headroom/pull/4042#issuecomment-6157529674
.


### Maintainer current-main requalification

Exact head `ff329f3e487940f85d56ba0b35bd9994afadb506`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Requalified against upstream main
c01cf07d94. The only merge conflict
combined main's Python 3.11 minimum with the contributor's
optional-integration typing guidance; both retained. No contributor
availability/binding policy rewritten. Current author/chopratejas/Devin
discussion reviewed, including the intentional removal of vendor-owned
response-parsing coverage. Nested binding precedence, real loopback HTTP
controls, public LiteLLM register_model isolation, independent billing
oracle and parent-module restoration remain covered.

Fresh focused tests: 29 passed with installed langchain-core 1.4.2 and
langchain-openai 1.2.2. Full mypy against the Linux CI target checked
580 source files: PR reports 22 errors; an actual detached current-main
run with the same environment reports those same 22 plus a
RunnableBinding truthy-function error that this PR removes. The 22
shared errors are in unrelated unchanged prefix-tracker, optional
memory/MCP and CrewAI modules; they are not contributor blockers and
this is not a clean full-mypy claim. Five changed Python files pass Ruff
lint/format and main-relative whitespace checks. Current-main DLL
supports local startup, without a branch-specific native-build or live
model-provider claim. No new source blocker; approval awaits exact-head
hosted qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
Mansurshakh Japarov
2026-11-06 09:24:06 -06:00
committed by GitHub
co-authored by JerrettDavis
parent a8905214d9
commit cde584d7fe
7 changed files with 206 additions and 74 deletions
+2 -2
View File
@@ -119,7 +119,7 @@ jobs:
- name: Verify Ruff version alignment
id: ruff-version
run: echo "version=$(python scripts/verify-ruff-version.py --print-version)" >> "$GITHUB_OUTPUT"
- run: python -m pip install --upgrade pip "ruff==${{ steps.ruff-version.outputs.version }}" "mypy==1.20.2"
- run: python -m pip install --upgrade pip "ruff==${{ steps.ruff-version.outputs.version }}" "mypy==1.20.2" "langchain-core>=1.3.3,<4.0"
- name: ruff check
run: ruff check .
- name: ruff format --check
@@ -393,7 +393,7 @@ jobs:
python -m pip install --upgrade pip
pip install torch --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple
WHEEL="$(ls dist/*.whl)"
pip install "${WHEEL}[dev]" pytest-split
pip install "${WHEEL}[dev,langchain]" pytest-split
# cwd's ./headroom source tree shadows the installed wheel; copy the
# compiled extension in so tests import it (no second cargo build).
SITE="$(python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')"
+5
View File
@@ -130,6 +130,11 @@ Enable or disable automatic Copilot review in **Settings → Rules → Rulesets
- Type hints on public functions; Google-style docstrings.
- Cover new behavior + edge cases; aim >80% coverage on new code.
- Python 3.11+. Optional features go behind extras.
- For an optional-integration change, run its owner tests and `mypy headroom`
with the relevant extra installed. A hook environment without that dependency
does not check the imported library's types.
CI lint installs `langchain-core` for its normal mypy run; keep typechecking
there rather than launching a second mypy subprocess from pytest.
- **Headroom writes LF and normalizes on read, on every platform.** Pass
`newline="\n"` to every `write_text`/`open` that writes a context, memory,
or state file, and normalize `\r\n`/`\r` when you read one back. Without
@@ -377,7 +377,7 @@ class HeadroomChatModel(BaseChatModel):
pass them explicitly.
"""
bound_kwargs: dict[str, Any] = {}
if not RunnableBinding:
if not LANGCHAIN_AVAILABLE:
return model, bound_kwargs
# isinstance, not hasattr: a Mock answers hasattr("bound") truthfully
# and would be unwrapped into one of its own auto-created children.
@@ -0,0 +1,178 @@
"""Cover Headroom's binding-kwargs merge with real LangChain and loopback HTTP."""
import asyncio
import json
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
import pytest
MODEL = "headroom-binding-loopback"
def _tool(name):
return {
"type": "function",
"function": {
"name": name,
"description": "Look up the weather in a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}
def _choice(name):
return {"type": "function", "function": {"name": name}}
@pytest.fixture(autouse=True)
def offline_headroom(monkeypatch, tmp_path):
"""Disable automatic egress and isolate normal product state/config."""
monkeypatch.setenv("HEADROOM_OFFLINE", "1")
# Track these too: apply_offline_env() sets them lazily during product use.
monkeypatch.setenv("HF_HUB_OFFLINE", "1")
monkeypatch.setenv("TRANSFORMERS_OFFLINE", "1")
monkeypatch.setenv("DO_NOT_TRACK", "1")
monkeypatch.setenv("HEADROOM_CONFIG_DIR", str(tmp_path / "config"))
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path / "workspace"))
monkeypatch.setenv("HEADROOM_MODEL_LIMITS", json.dumps({"context_limits": {MODEL: 128000}}))
for name in ("LANGCHAIN_TRACING", "LANGCHAIN_TRACING_V2", "LANGSMITH_TRACING"):
monkeypatch.setenv(name, "false")
monkeypatch.setenv("OTEL_SDK_DISABLED", "true")
monkeypatch.setenv("OPENAI_PROXY", "")
@pytest.fixture
def openai_loopback():
"""Run only a local HTTP fixture, without hosted calls or real credentials."""
pytest.importorskip("langchain_openai")
requests = []
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
requests.append((self.path, body))
message = {"role": "assistant", "content": "Loopback reply"}
finish_reason = "stop"
response = json.dumps(
{
"id": "chatcmpl-loopback",
"object": "chat.completion",
"created": 1,
"model": MODEL,
"choices": [{"index": 0, "message": message, "finish_reason": finish_reason}],
"usage": {"prompt_tokens": 41, "completion_tokens": 9, "total_tokens": 50},
}
).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(response)))
self.end_headers()
self.wfile.write(response)
def log_message(self, *args):
pass
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
thread = Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
yield f"http://127.0.0.1:{server.server_port}/v1", requests
finally:
server.shutdown()
server.server_close()
thread.join(timeout=5)
assert not thread.is_alive(), "Loopback HTTP server did not stop"
@pytest.mark.parametrize("binding", ["bind_tools", "nested"])
def test_binding_reaches_openai_http(openai_loopback, binding):
import httpx
from langchain_core.runnables import RunnableBinding
from langchain_openai import ChatOpenAI
from langsmith import tracing_context
from headroom.integrations import HeadroomChatModel
base_url, requests = openai_loopback
with tracing_context(enabled=False), httpx.Client(trust_env=False) as http_client:
original = ChatOpenAI(
model=MODEL,
api_key="loopback-dummy-key",
base_url=base_url,
max_retries=0,
timeout=5,
use_responses_api=False,
http_client=http_client,
)
kwargs = {}
expected_name = "inner_weather"
if binding == "bind_tools":
wrapped = HeadroomChatModel(original, auto_detect_provider=False).bind_tools(
[_tool(expected_name)], tool_choice=expected_name
)
else:
inner = original.bind_tools(
[_tool("inner_weather")],
tool_choice="inner_weather",
temperature=0.1,
top_p=0.8,
)
outer = RunnableBinding(
bound=inner,
kwargs={
"tools": [_tool("outer_weather")],
"tool_choice": _choice("outer_weather"),
"temperature": 0.2,
},
)
wrapped = HeadroomChatModel(outer, auto_detect_provider=False)
expected_name = "caller_weather"
kwargs = {
"tools": [_tool(expected_name)],
"tool_choice": _choice(expected_name),
"temperature": 0.3,
}
wrapped.cache = False
wrapped.invoke("What is the weather in Paris?", **kwargs)
_, body = requests[0]
assert body["tools"] == [_tool(expected_name)]
assert body["tool_choice"] == _choice(expected_name)
if binding == "nested":
assert body["temperature"] == 0.3
assert body["top_p"] == 0.8
def test_unwrap_binding_matches_real_runnable_precedence():
pytest.importorskip("langchain_core")
from langchain_core.runnables import RunnableBinding, RunnableLambda
from langsmith import tracing_context
from headroom.integrations import HeadroomChatModel
def affine(value, scale=1, offset=0):
return value * scale + offset
target = RunnableLambda(affine)
inner = RunnableBinding(bound=target, kwargs={"scale": 2, "offset": 3})
outer = RunnableBinding(bound=inner, kwargs={"scale": 5})
unwrapped, kwargs = HeadroomChatModel._unwrap_binding(outer)
plain, plain_kwargs = HeadroomChatModel._unwrap_binding(target)
assert unwrapped is target
with tracing_context(enabled=False):
assert outer.invoke(4) == unwrapped.invoke(4, **kwargs) == 23
caller_kwargs = {**kwargs, "scale": 7}
assert outer.invoke(4, scale=7) == unwrapped.invoke(4, **caller_kwargs) == 31
assert asyncio.run(outer.ainvoke(4)) == asyncio.run(unwrapped.ainvoke(4, **kwargs)) == 23
assert asyncio.run(outer.ainvoke(4, scale=7)) == 31
assert asyncio.run(unwrapped.ainvoke(4, **caller_kwargs)) == 31
assert plain.invoke(4, **plain_kwargs) == 4
# Caller overrides must not mutate either reusable binding.
assert inner.invoke(4) == 11
assert outer.invoke(4) == 23
@@ -3,27 +3,21 @@
Each test here corresponds to a defect that shipped in 0.37.0 and was found by
running the documented examples against langchain-core 1.6:
1. ``HeadroomChatModel.bind_tools()`` returned a model that emitted no tool
calls, because ``_generate`` called the private method on the
``RunnableBinding`` and the bound kwargs were dropped.
2. ``wrap_tools_with_headroom`` produced tools that raised ``TypeError`` on
1. ``wrap_tools_with_headroom`` produced tools that raised ``TypeError`` on
invoke, because ``StructuredTool`` calls its ``func`` with unpacked kwargs
while ``BaseTool.invoke`` takes a single input.
3. The wrapped tool lost the original ``args_schema``, so a model saw a tool
2. The wrapped tool lost the original ``args_schema``, so a model saw a tool
with no parameters.
4. ``HeadroomDocumentCompressor`` silently subclassed a local stub rather than
3. ``HeadroomDocumentCompressor`` silently subclassed a local stub rather than
LangChain's ``BaseDocumentCompressor``, so retrievers rejected it.
"""
import asyncio
import json
from typing import Any
import pytest
try:
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage
from langchain_core.tools import tool
LANGCHAIN_AVAILABLE = True
@@ -61,60 +55,8 @@ def query_database(query: str) -> str:
return BIG_RESULT
class _RecordingModel(GenericFakeChatModel):
"""Fake model that records the kwargs its _generate actually received."""
last_kwargs: dict[str, Any] = {}
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
type(self).last_kwargs = dict(kwargs)
kwargs.pop("tools", None)
return super()._generate(messages, stop=stop, run_manager=run_manager, **kwargs)
def bind_tools(self, tools, **kwargs):
# Mirrors what real providers do: return a RunnableBinding carrying the
# tools in .kwargs rather than a new model instance.
return self.bind(tools=list(tools), **kwargs)
class TestBindToolsSurvivesWrapping:
"""Defect 1: bound kwargs must reach the underlying model."""
def test_bound_tools_reach_generate(self):
from headroom.integrations import HeadroomChatModel
_RecordingModel.last_kwargs = {}
inner = _RecordingModel(messages=iter([AIMessage("ok")]))
wrapped = HeadroomChatModel(inner).bind_tools([query_database])
wrapped.invoke("call the tool")
assert "tools" in _RecordingModel.last_kwargs, (
"bind_tools kwargs were dropped before reaching the wrapped model; "
"an agent built on this model would never call a tool"
)
def test_unbound_model_passes_no_tools(self):
from headroom.integrations import HeadroomChatModel
_RecordingModel.last_kwargs = {}
inner = _RecordingModel(messages=iter([AIMessage("ok")]))
HeadroomChatModel(inner).invoke("hello")
assert "tools" not in _RecordingModel.last_kwargs
def test_unwrap_binding_ignores_non_bindings(self):
from headroom.integrations import HeadroomChatModel
inner = GenericFakeChatModel(messages=iter([AIMessage("ok")]))
model, kwargs = HeadroomChatModel._unwrap_binding(inner)
assert model is inner
assert kwargs == {}
class TestWrappedToolIsUsable:
"""Defects 2 and 3: the wrapped tool must invoke, and keep its schema."""
"""Defects 1 and 2: the wrapped tool must invoke, and keep its schema."""
def test_invoke_with_keyword_arguments(self):
from headroom.integrations import wrap_tools_with_headroom
+5
View File
@@ -118,6 +118,11 @@ class TestLiteLLMLogSuppression:
"""The env flag must exist before litellm itself is imported."""
import os
import headroom.providers as providers
# Import also replaces the parent package attribute. Track it even when
# initially absent so teardown restores or removes the temporary module.
monkeypatch.setattr(providers, "litellm", None, raising=False)
monkeypatch.delenv("LITELLM_SUPPRESS_DEBUG_INFO", raising=False)
# delitem, not pop: teardown puts the real modules back. A bare pop made
# every later `import litellm` load a second copy, so a test pinning a
+11 -9
View File
@@ -72,6 +72,10 @@ def test_xai_cache_savings_and_billing_use_their_respective_price_sources(
}
if catalog_read is not None:
row["cache_read_input_token_cost"] = catalog_read
original_row = litellm.model_cost[model]
# Register through the public API; keep the isolated row free of inherited rates.
monkeypatch.setitem(litellm.model_cost, model, dict(row))
litellm.register_model({model: dict(row)})
monkeypatch.setitem(litellm.model_cost, model, row)
resolve_rates.cache_clear()
try:
@@ -83,21 +87,19 @@ def test_xai_cache_savings_and_billing_use_their_respective_price_sources(
# A provider-wide counterfactual estimate is not a billed cache rate.
# Use the real calculator for the same reported usage, as billing does.
if catalog_read is None:
canonical_input, _ = litellm.cost_per_token(
model=model,
prompt_tokens=10_000,
completion_tokens=0,
cache_read_input_tokens=10_000,
)
else:
canonical_input = 10_000 * catalog_read
canonical_input, _ = litellm.cost_per_token(
model=model,
prompt_tokens=10_000,
completion_tokens=0,
cache_read_input_tokens=10_000,
)
tracker = CostTracker()
tracker.record_tokens(model, tokens_saved=0, tokens_sent=10_000, cache_read_tokens=10_000)
assert tracker.totals()[1] == pytest.approx(canonical_input)
assert tracker.stats()["total_input_cost_usd"] == pytest.approx(canonical_input)
finally:
resolve_rates.cache_clear()
litellm.register_model({model: original_row})
@pytest.mark.parametrize("user_agent", ["grok/1.2.3", "grok-shell/0.2.112"])