mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description With `HEADROOM_THINKING_COMPACT=1`, a cold turn drops old reasoning outright and caches that prefix, but the next warm turn recomputes from the client's full history and Kompresses those same turns instead, so the forwarded prefix no longer matches what was cached and every turn after a cold one misses. Closes #4548 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `compact_reasoning_openai_chat` gains `drop_first_turns`: the first N assistant turns (excluding kept last turns) take the drop path even when warm; stats report `dropped_upto`. - `PrefixCacheTracker.reasoning_drop_upto` stores the per-session sticky floor; a cold turn raises it, never lowers it. - OpenAI chat handler passes the floor in and records it after a cold turn; sticky drop runs even without a Kompress model. - New `tests/test_thinking_compact_sticky_drop.py` (fails on `main`, passes with fix). ## Testing - [x] Unit tests added and passing ``` $ python -m pytest tests/test_thinking_compact_sticky_drop.py tests/test_thinking_compactor_version_gate.py tests/test_cold_prefix.py -q 23 passed in 3.31s $ ruff check <changed files> -> All checks passed! # without the fix: 4 failed (new test file) ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, local checkout rebased on upstream/main - Exact command / steps: ran the compactor on a 3-turn history with drop=True (cold), then on the 4-turn follow-up with drop=False and drop_first_turns from the cold stats - Observed result: the first two assistant messages in the warm output are byte-identical to the cold output; the newer turn is Kompressed, the active turn kept - Not tested: a live Kimi/GLM upstream cache-hit measurement through the proxy ## Runtime Rollout Safety - Rollout-managed feature(s): none; only affects the opt-in HEADROOM_THINKING_COMPACT path - Minimum rollout channel: n/a - Stable/default behavior changed: no (feature off by default) - Kill switch / disable path: unset HEADROOM_THINKING_COMPACT - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) ### Maintainer current-main requalification Exact head `5f7f8be6fabe7e5a0ab33b3c12995d730e071d72`, upstream main `2d3ee51846435ef5b320eea4806af14e602b0d8c`. Re-reviewed contributor headc3c3aa649aand current discussion. The new raise_reasoning_drop_floor method updates the request's resolved turn snapshot as well as the live floor; response recording persists this request's deliberate cold increase while excluding sibling mutations. The two added regressions exercise response recording and subsequent resumed warm compaction, addressing the previously reproduced missing path. No remaining source blocker found in this scope. Integrated upstream/main7592e68145cleanly and reviewed the composed tracker/handler/compactor paths. 39 sticky-drop/cold-prefix/version/memory/Gemini tests passed; a broader prefix-tracker/restart/client-rewrite/compactor-cache run passed 110 tests with one skip. Ruff check/format and whitespace pass. Existing native extension used for Python startup, not a fresh native build. No live upstream cache-hit measurement claimed. Formal approval awaits exact-head hosted qualification; no contributor logic rewritten and nothing merged. Final-main refresh: main advanced during publication to2d3ee51846when release 0.43.1 was merged externally. Publication was refused before push; integrated the release merge cleanly and re-reviewed unchanged runtime inputs plus generated metadata. 88 release/version/MCP/sticky-drop tests pass on the final composed tree; whitespace passes. Prior 149 test passes remain explicitly scoped to unchanged tracker/runtime inputs. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
184 lines
6.7 KiB
Python
184 lines
6.7 KiB
Python
"""#4548: reasoning dropped on a cold turn stays dropped on the next warm turn."""
|
|
|
|
import contextvars
|
|
from types import SimpleNamespace
|
|
|
|
from headroom.transforms.thinking_compactor import compact_reasoning_openai_chat
|
|
|
|
|
|
class _HalfKompress:
|
|
def compress(self, text, allow_download=False):
|
|
words = text.split()
|
|
return SimpleNamespace(compressed=" ".join(words[: len(words) // 2]))
|
|
|
|
|
|
def _history(n, tag):
|
|
msgs = []
|
|
for k in range(n):
|
|
msgs.append({"role": "user", "content": f"q{k}"})
|
|
reasoning = " ".join(f"{tag}{k}w{j}" for j in range(60))
|
|
msgs.append({"role": "assistant", "content": f"a{k}", "reasoning_content": reasoning})
|
|
return msgs
|
|
|
|
|
|
def test_cold_drop_reports_dropped_upto():
|
|
out, stats = compact_reasoning_openai_chat(
|
|
_history(3, "c"), kompress=None, keep_last_turns=1, drop=True
|
|
)
|
|
assert [m.get("reasoning_content") for m in out[1::2]][:2] == ["", ""]
|
|
assert stats["dropped_upto"] == 2
|
|
|
|
|
|
def test_warm_turn_after_cold_reuses_cold_prefix():
|
|
cold, stats = compact_reasoning_openai_chat(
|
|
_history(3, "w"), kompress=None, keep_last_turns=1, drop=True
|
|
)
|
|
warm, _ = compact_reasoning_openai_chat(
|
|
_history(4, "w"),
|
|
kompress=_HalfKompress(),
|
|
keep_last_turns=1,
|
|
drop=False,
|
|
drop_first_turns=stats["dropped_upto"],
|
|
)
|
|
assert warm[:4] == cold[:4] # previously dropped turns byte-identical
|
|
assert 0 < len(warm[5]["reasoning_content"].split()) < 60 # newer turn Kompressed
|
|
assert len(warm[7]["reasoning_content"].split()) == 60 # active turn kept
|
|
|
|
|
|
def test_no_sticky_floor_is_unchanged():
|
|
out, stats = compact_reasoning_openai_chat(
|
|
_history(3, "u"), kompress=_HalfKompress(), keep_last_turns=1
|
|
)
|
|
assert all(len(m["reasoning_content"].split()) == 30 for m in out[1:4:2])
|
|
assert stats["dropped_upto"] == 0
|
|
|
|
|
|
def test_inline_think_spans_stay_dropped():
|
|
think = "<think>" + " ".join(f"t{j}" for j in range(60)) + "</think>"
|
|
msgs = [
|
|
{"role": "user", "content": "q"},
|
|
{"role": "assistant", "content": think + "a0"},
|
|
{"role": "user", "content": "q"},
|
|
{"role": "assistant", "content": think + "a1"},
|
|
]
|
|
out, _ = compact_reasoning_openai_chat(
|
|
msgs, kompress=_HalfKompress(), keep_last_turns=1, drop_first_turns=1
|
|
)
|
|
assert out[1]["content"] == "a0"
|
|
assert out[3]["content"] == think + "a1"
|
|
|
|
|
|
def _tracker_with_floor(floor):
|
|
from headroom.cache.prefix_tracker import PrefixCacheTracker
|
|
|
|
tracker = PrefixCacheTracker("openai")
|
|
tracker.update_from_response(cache_read_tokens=0, cache_write_tokens=0, messages=[])
|
|
tracker.reasoning_drop_upto = floor
|
|
return tracker
|
|
|
|
|
|
def test_floor_survives_shared_round_trip_and_warm_prefix_matches():
|
|
from headroom.cache.prefix_tracker import SessionTrackerStore
|
|
|
|
cold, stats = compact_reasoning_openai_chat(
|
|
_history(3, "r"), kompress=None, keep_last_turns=1, drop=True
|
|
)
|
|
store = SessionTrackerStore()
|
|
meta, table = store.encode_shared("k", _tracker_with_floor(stats["dropped_upto"]))
|
|
restored = store.decode_shared(meta, table)
|
|
assert restored.reasoning_drop_upto == 2
|
|
warm, _ = compact_reasoning_openai_chat(
|
|
_history(4, "r"),
|
|
kompress=_HalfKompress(),
|
|
keep_last_turns=1,
|
|
drop_first_turns=restored.reasoning_drop_upto,
|
|
)
|
|
assert warm[:4] == cold[:4]
|
|
|
|
|
|
def test_old_metadata_without_floor_decodes_to_zero():
|
|
from headroom.cache.prefix_tracker import SessionTrackerStore
|
|
|
|
store = SessionTrackerStore()
|
|
meta, table = store.encode_shared("k", _tracker_with_floor(3))
|
|
meta["state"] = meta["state"][:5]
|
|
meta["turns"] = [[t, s[:5]] for t, s in meta["turns"]]
|
|
assert store.decode_shared(meta, table).reasoning_drop_upto == 0
|
|
|
|
|
|
def test_resumed_turn_restores_its_own_floor():
|
|
tracker = _tracker_with_floor(2)
|
|
tracker._turn_states[1] = tracker._turn_state()
|
|
tracker.reasoning_drop_upto = 3 # a cold fork raised it
|
|
tracker._resume_turn(1)
|
|
assert tracker.reasoning_drop_upto == 2
|
|
|
|
|
|
def test_request_floor_ignores_a_concurrent_fork_raising_the_live_floor():
|
|
from headroom.cache.prefix_tracker import _resolved_turn
|
|
|
|
tracker = _tracker_with_floor(2)
|
|
tracker._turn_states[1] = tracker._turn_state()
|
|
request_a = contextvars.copy_context()
|
|
request_a.run(_resolved_turn.set, (tracker, 1, tracker._turn_states[1]))
|
|
tracker.reasoning_drop_upto = 3 # sibling fork went cold after A resolved
|
|
|
|
floor = request_a.run(tracker.request_reasoning_drop_floor)
|
|
assert floor == 2
|
|
out, _ = compact_reasoning_openai_chat(
|
|
_history(4, "f"), kompress=_HalfKompress(), keep_last_turns=1, drop_first_turns=floor
|
|
)
|
|
assert len(out[5]["reasoning_content"].split()) == 30 # Kompressed, not dropped
|
|
|
|
|
|
def test_cold_request_records_raised_floor_through_response():
|
|
from headroom.cache.prefix_tracker import _resolved_turn
|
|
|
|
tracker = _tracker_with_floor(0)
|
|
tracker._turn_states[1] = tracker._turn_state()
|
|
request = contextvars.copy_context()
|
|
request.run(_resolved_turn.set, (tracker, 1, tracker._turn_states[1]))
|
|
|
|
cold, stats = compact_reasoning_openai_chat(
|
|
_history(3, "r"), kompress=None, keep_last_turns=1, drop=True
|
|
)
|
|
floor = max(request.run(tracker.request_reasoning_drop_floor), stats["dropped_upto"])
|
|
request.run(tracker.raise_reasoning_drop_floor, floor)
|
|
request.run(tracker.update_from_response, 0, 0, cold)
|
|
|
|
assert tracker._turn_states[1][5] == 2
|
|
tracker._resume_turn(1)
|
|
warm, _ = compact_reasoning_openai_chat(
|
|
_history(4, "r"),
|
|
kompress=_HalfKompress(),
|
|
keep_last_turns=1,
|
|
drop=False,
|
|
drop_first_turns=tracker.request_reasoning_drop_floor(),
|
|
)
|
|
assert [m.get("reasoning_content") for m in warm[1::2]][:2] == ["", ""]
|
|
assert warm[:5] == cold[:5]
|
|
|
|
|
|
def test_cold_request_floor_write_isolated_from_sibling_through_response():
|
|
from headroom.cache.prefix_tracker import _resolved_turn
|
|
|
|
tracker = _tracker_with_floor(2)
|
|
tracker._turn_states[1] = tracker._turn_state()
|
|
request_a = contextvars.copy_context()
|
|
request_a.run(_resolved_turn.set, (tracker, 1, tracker._turn_states[1]))
|
|
tracker.reasoning_drop_upto = 3 # sibling fork mutated live floor
|
|
|
|
floor = request_a.run(tracker.request_reasoning_drop_floor)
|
|
assert floor == 2
|
|
request_a.run(tracker.update_from_response, 0, 0, [])
|
|
|
|
assert tracker._turn_states[1][5] == 2
|
|
tracker._resume_turn(1)
|
|
out, _ = compact_reasoning_openai_chat(
|
|
_history(4, "s"),
|
|
kompress=_HalfKompress(),
|
|
keep_last_turns=1,
|
|
drop_first_turns=tracker.request_reasoning_drop_floor(),
|
|
)
|
|
assert len(out[5]["reasoning_content"].split()) == 30 # Kompressed, not dropped
|