Commit Graph
5 Commits
Author SHA1 Message Date
Luis ArayaandJerrettDavis 36540379e0 fix(proxy): skip memory-context search on OpenAI paths in cache mode (#4634)
## Description

Follow-up to #4606. In `--mode cache`, the proxy preserves prompt cache
stability across turns by never mutating or appending to user turns.
While #4606 addressed this for Anthropic's `/v1/messages` by skipping
`search_and_format_context`, the three OpenAI-compatible paths
(`/v1/chat/completions`, `/v1/responses`, and `/v1/responses` WebSocket)
still executed the memory search and injected memory context into the
user turn even in cache mode.

This PR aligns the OpenAI handlers with Anthropic: when in cache mode
with `inject_context` enabled, it logs `skipped_cache_mode`, stamps
`memory_skip_reason="cache_mode"` in the request/frame tags, and skips
`search_and_format_context`. Memory tool injection and user input bytes
remain unchanged.

Closes #4622

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement

## Changes Made

- `headroom/proxy/handlers/openai.py`:
- `handle_openai_chat`: skip `search_and_format_context` when
`is_cache_mode` is active, stamp `tags["memory_skip_reason"] =
"cache_mode"`, log `decision="skipped_cache_mode"`, and keep the latest
user message unmutated.
- `handle_openai_responses`: skip `search_and_format_context` when
`is_cache_mode` is active, stamp `tags["memory_skip_reason"] =
"cache_mode"`, and log `decision="skipped_cache_mode"`.
- `handle_openai_responses_ws`: skip `search_and_format_context` when
`is_cache_mode` is active, stamp `ws_tags["memory_skip_reason"] =
"cache_mode"`, log `decision="skipped_cache_mode"`, and avoid input
mutation.
- `tests/test_proxy_system_prompt_immutable.py`:
- Updated `test_cache_mode_skips_memory_context_search_openai_chat` to
verify `memory_skip_reason="cache_mode"` on recorded outcomes and
preservation of memory tools.
- Updated `test_cache_mode_skips_memory_context_search_openai_responses`
to verify `memory_skip_reason="cache_mode"` on recorded outcomes and
preservation of memory tools.
- `tests/test_codex_ws_per_frame_memory.py`:
- Expanded `test_cache_mode_skips_memory_context_search_ws` with
multiple response.create frames verifying input preservation, memory
tool preservation, and `memory_skip_reason="cache_mode"` in turn outcome
tags.

## Testing

- [x] Unit tests pass:
  - `tests/test_proxy_system_prompt_immutable.py`: 10 passed
  - `tests/test_codex_ws_per_frame_memory.py`: 12 passed
- [x] Linting passes: `ruff check` on modified files
- [x] Formatting passes: `ruff format --check` on modified files
- [x] New regression tests added for all 3 OpenAI paths in cache mode
- [x] Manual testing performed

### Test Output

```text
tests/test_proxy_system_prompt_immutable.py .......... [ 45%]
tests/test_codex_ws_per_frame_memory.py ............   [100%]
======================= 22 passed, 4 warnings in 18.06s =======================
```

## Real Behavior Proof

- Environment: Windows, Python 3.12.16, `HEADROOM_OFFLINE=1`.
- Exact command / steps:
1. Ran `pytest tests/test_proxy_system_prompt_immutable.py` exercising
`/v1/chat/completions` and `/v1/responses` in cache mode with mock
memory handler and outcome recorder.
2. Ran `pytest tests/test_codex_ws_per_frame_memory.py -k
test_cache_mode_skips_memory_context_search_ws` exercising two
`response.create` frames in cache mode with mock WebSocket and outcome
recorder.
3. Checked that `memory_handler.search_and_format_context` is never
awaited on any path.
4. Checked that `memory_skip_reason == "cache_mode"` reaches
`RequestOutcome.tags` across Chat, Responses, and WebSocket turns.
5. Checked that input bytes and client/memory tools (`client_tool`,
`mem_search`) remain untouched in the forwarded payloads.
- Observed result:
- `memory_handler.search_and_format_context.assert_not_awaited()` passed
on all routes.
- `outcome.tags.get("memory_skip_reason") == "cache_mode"` passed on
Chat, Responses, and WS frames.
- Forwarded inputs matched raw client inputs without injected memory
context.
- Forwarded tools retained `client_tool` and memory tools
(`mem_search`).
  - 22 tests passed across both test files.
- Not tested: Live cloud provider endpoints.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: yes; in cache mode, memory context is
no longer injected into OpenAI requests, ensuring next-turn prefix bytes
remain byte-identical, and request/frame tags now record
`memory_skip_reason="cache_mode"`.
- Kill switch / disable path: N/A (revert commit)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I did not edit `CHANGELOG.md`

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 14:38:43 -06:00
365f6fe523 fix(proxy): skip memory-context search in cache mode on OpenAI routes (#4632)
## Description

Follow-up to #4600 / #4606 as requested in #4622. PR #4606 implemented
this optimization for Anthropic `/v1/messages`. This PR brings the three
OpenAI routes (`handle_openai_chat`, `handle_openai_responses`, and
`handle_openai_responses_ws`) into alignment.

In `--mode cache`, memory context injection is disabled so prompt prefix
bytes remain byte-stable across turns. Previously, all three OpenAI
paths still executed the memory-context search
(`search_and_format_context`: embedding query, vector index lookups,
graph searches) and appended the result to the latest user turn in cache
mode, introducing latency (~1–37ms+ per turn) and altering the forwarded
user turn.

With this PR:
1. `handle_openai_chat`: checks `is_cache_mode(self.config.mode)` before
executing `search_and_format_context`, emits
`decision="skipped_cache_mode"`, and keeps the user turn intact.
2. `handle_openai_responses`: checks `is_cache_mode(self.config.mode)`
before searching, emits `decision="skipped_cache_mode"`, and keeps
`body["input"]` intact.
3. `handle_openai_responses_ws`: checks
`is_cache_mode(self.config.mode)` before measuring `memory_context`,
emits `decision="skipped_cache_mode"`, and leaves
`ws_response_body["input"]` untouched.
4. Memory tool injection remains completely unchanged across all routes.

Closes #4622

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement

## Changes Made

- `headroom/proxy/handlers/openai.py`: Check
`is_cache_mode(self.config.mode)` when `inject_context` is enabled
across `handle_openai_chat`, `handle_openai_responses`, and
`handle_openai_responses_ws`. Emit `decision="skipped_cache_mode"` with
`bytes_injected=0` and bypass `search_and_format_context`.
- `tests/test_proxy_system_prompt_immutable.py`: Added
`test_cache_mode_skips_memory_context_search_openai_chat` and
`test_cache_mode_skips_memory_context_search_openai_responses` to verify
that `search_and_format_context` is never awaited in cache mode and user
message / input fields remain byte-equal to client values.

## Testing

- [x] Unit tests pass (`pytest
tests/test_proxy_system_prompt_immutable.py`)
- [x] Linting passes (`ruff check headroom/proxy/handlers/openai.py
tests/test_proxy_system_prompt_immutable.py`)
- [x] Formatting passes (`ruff format --check
headroom/proxy/handlers/openai.py
tests/test_proxy_system_prompt_immutable.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_proxy_system_prompt_immutable.py -q
10 passed, 1 warning in 31.92s

$ ruff check headroom/proxy/handlers/openai.py tests/test_proxy_system_prompt_immutable.py
All checks passed!

$ ruff format --check headroom/proxy/handlers/openai.py tests/test_proxy_system_prompt_immutable.py
2 files already formatted
```

## Real Behavior Proof

- Environment: Linux / WSL2 Ubuntu, Python 3.13, local proxy test
harness (`create_app` + `TestClient`).
- Exact command / steps: Executed requests against
`/v1/chat/completions` and `/v1/responses` with proxy configured in
`--mode cache` and memory context enabled. Verified upstream payload
captures and mock call history.
- Observed result: In cache mode, the memory search is bypassed on all
OpenAI requests; search_and_format_context was never awaited (0 calls),
user message content and input fields remain byte-equal without injected
memory context, and proxy emitted decision=skipped_cache_mode with
bytes_injected=0.
- Not tested: Live WebSocket connection to OpenAI upstream with
streaming audio/realtime endpoints.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no. Cache mode requires byte
stability across turns; memory search is skipped in cache mode. Token
mode is untouched.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: git revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation: n/a
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- Aligns OpenAI Chat, Responses, and WebSocket routes with Anthropic
behavior established in #4606.
- Preserves memory tool injection without modification.

Co-authored-by: shaiknaziya20 <naziyashaik5512@gmail.com>


### Maintainer combined-head validation

Reviewed head `66a09054548423e33af47121ac7e87134d023fe4` includes a
clean main sync through `12dca26aa` plus test-only maintainer
follow-ups: the existing WebSocket dummy config now explicitly uses
token mode, and a two-frame cache-mode regression covers skipped
searches, unchanged input, retained memory tools, passthrough
cancellation frames, and skip telemetry. No contributor runtime logic
was changed.

Windows / Python 3.12.13, with the existing local Rust extension:
`.venv/Scripts/python.exe -m pytest
tests/test_proxy_system_prompt_immutable.py
tests/test_codex_ws_per_frame_memory.py
tests/test_openai_codex_ws_lifecycle.py -q --tb=short
--disable-warnings` => **89 passed**. The new WebSocket regression fails
on current main because both frames still execute memory searches. Ruff
check/format and main-relative whitespace checks pass.

Additional runtime proof used a real Uvicorn proxy and a separate local
HTTP upstream, both bound to ephemeral loopback ports, with production
HTTP forwarding intact (no `_retry_request` patch). A deterministic
recording memory backend returned `LOOPBACK_MEMORY` when searched, so
the probe can distinguish an actual skip from an unconfigured/disabled
memory path. The probe sent the same authenticated Chat and Responses
requests in token mode, then cache mode, using user text `latest user
prompt` and fixed Responses instructions.

Exact invocation from the isolated worktree:
`PYTHONPATH=G:/git/headroom-review4632 HEADROOM_OFFLINE=1
G:/git/headroom/.venv/Scripts/python.exe
G:/git/headroom/.codex/review4632-loopback.py` (environment variables
set with PowerShell `$env:` assignments). Results: **four HTTP 200
responses**; token mode performed one memory search per route and
forwarded the returned memory text; cache mode performed zero searches
per route, forwarded the original user text unchanged, and recorded
`memory_skip_reason=cache_mode`. The proxy and upstream were shut down
after the run.

Limits: the loopback provider and recording memory backend are
deterministic local fixtures; no external provider, embedding model,
production memory service, load test, or real upstream WebSocket was
exercised. WebSocket multi-frame/input/tool/telemetry behavior is
covered by the focused regression above. Updated-head hosted
qualification is inspected by maintainers; no contributor CI action is
needed. Nothing merged.

---------

Co-authored-by: shaiknaziya20 <naziyashaik5512@gmail.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 12:37:00 -06:00
Krzysztof KrawczyńskiandJD Davis 0396ea2fc2 fix(codex): resolve per-turn project context (#2636)
## Description

Codex Desktop can send turns from multiple projects through one HTTP or
WebSocket route, so a static `/p/<project>` route cannot reliably
identify each turn.

This PR resolves project context per turn from explicit overrides or
Codex thread/turn metadata. Unsafe or inconsistent resolution skips
project memory and learning while the model request continues.

Related to #2355 and overlaps #2379.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Resolve `thread_id` and `turn_id` to the matching rollout cwd.
- Use the resolver for HTTP and per-frame WebSocket traffic.
- Pin each WebSocket to its first resolved project and fail closed on
mismatch.
- Strip Codex-only metadata before forwarding upstream.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
121 passed, 1 pre-existing deprecation warning
All checks passed!
Success: no issues found in 510 source files
```

## Real Behavior Proof

- Environment: macOS; isolated Codex state and project fixtures plus the
combined local deployment.
- Exact command / steps: Exercised two same-basename projects over HTTP
and concurrent WebSockets, including resume/fork, missing, stale,
locked, malformed, and mismatched contexts.
- Observed result: Each turn resolved to the correct project; unsafe
cases skipped project memory and learning without blocking the model
request. The integrated patch also ran in the local deployment for more
than 39 hours.
- Not tested: Linux, Windows, VMs, or GitHub CI.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A; this change uses stable/default
behavior.
- Stable/default behavior changed: Codex HTTP and WebSocket turns
resolve project context per turn; unsafe resolution skips project memory
and learning.
- Kill switch / disable path: Run without `--memory`; use `--no-learn`
to disable traffic learning.
- Unsafe override required: No.
- Qualification impact: Codex project-context isolation for HTTP and
WebSocket traffic.
- Rollback path: Revert this PR. No schema, data, or configuration
migration is required.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] Documentation changes are not required for this internal routing
fix
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

Waiting follow-up: [project-scoped
TrafficLearner](https://github.com/krzkraw/headroom/tree/krz/project-scoped-traffic-learner).
It depends on the per-turn project API introduced here.

No database or configuration migration is required.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2636"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 15:35:51 -05:00
AxelRayandJD Davis 46755b5f72 fix(memory): avoid injecting tools into tool-free requests (#3677)
## Description

Memory tool injection should be suppressed only for a client that is
explicitly known not to support tools. An omitted or empty `tools` field
is not enough to make that determination because Codex and other clients
may advertise tool capability out of band.

Closes #3676

## Changes Made

- Carry the client capability decision through the Anthropic and OpenAI
Responses paths.
- Preserve existing and sticky memory tools for Codex and unknown
clients.
- Suppress fresh and sticky memory-tool injection only for an explicitly
tool-incapable client.
- Add endpoint-level and helper regressions for that distinction.

## Testing

- Targeted memory and routing tests passed locally.
- `ruff check .` passed.
- `ruff format --check .` passed.
- `mypy` passed for the changed modules.
- GitHub CI is running on the current head.

## Runtime Rollout Safety

- No new feature flag, schema, or migration is introduced.
- Existing memory injection configuration remains the available disable
path.
- Rollback is a normal revert of this change.

## Review Readiness

- Self-review completed.
- The PR is ready for human review.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-09-29 15:31:35 -05:00
Rod BoevandTejas Chopra 38479fcda1 fix(codex): rerun memory lookup on every response.create WS frame (#2113)
## Description

Long-lived Codex `/v1/responses` WebSocket sessions only ran memory
decision, query construction, and context injection on the first
`response.create` frame because `handle_openai_responses_ws` kept that
logic outside the relay loop. This change extracts that path into a
local helper reused before compression for every eligible
`response.create`, while keeping sticky memory tools deduplicated by the
existing session helper.

Closes #2059

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Extracted the first-frame memory preparation path into a local async
helper inside `handle_openai_responses_ws`.
- Reused that helper for the initial frame and every later eligible
`response.create` before compression and shaping.
- Added focused two-turn WebSocket regressions for per-frame lookup,
bypass and disabled-memory handling, list-shaped later inputs,
memory-handler fail-open recovery, and sticky-tool replay.
- Raised the locked production floors for `click` and `pillow` to clear
the current `pip-audit` findings that now fail external PR merge
snapshots.

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_codex_ws_per_frame_memory.py
tests/test_openai_codex_ws_lifecycle.py -q`)
- [x] Linting passes (`uv run ruff check
headroom/proxy/handlers/openai.py
tests/test_codex_ws_per_frame_memory.py
tests/test_openai_codex_ws_lifecycle.py`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run pytest tests/test_codex_ws_per_frame_memory.py tests/test_openai_codex_ws_lifecycle.py -q

============================= test session starts =============================
platform win32 -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0
rootdir: D:\Repos\headroom-pr-2059-codex-ws-memory-lookup
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.9.3, asyncio-1.3.0, cov-7.0.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 35 items

tests\test_codex_ws_per_frame_memory.py .........                        [ 25%]
tests\test_openai_codex_ws_lifecycle.py ..........................       [100%]

============================= 35 passed in 1.89s ==============================

uv run ruff check headroom/proxy/handlers/openai.py tests/test_codex_ws_per_frame_memory.py tests/test_openai_codex_ws_lifecycle.py
All checks passed!

uv run ruff format headroom/proxy/handlers/openai.py tests/test_codex_ws_per_frame_memory.py tests/test_openai_codex_ws_lifecycle.py --check
3 files already formatted
```

## Real Behavior Proof

- Environment: Windows, Python 3.12.13, in-process WebSocket harness
with a fake memory handler
- Exact command / steps: `uv run pytest
tests/test_codex_ws_per_frame_memory.py
tests/test_openai_codex_ws_lifecycle.py -q`, which opens one connection,
sends distinct `response.create` frames, and records each memory lookup
plus the forwarded tool set
- Observed result: the handler performs one lookup per eligible frame,
later-frame compression sees current-turn memory-prepared input, and
both forwarded turns carry the same deduplicated `memory_search` and
`memory_save` definitions
- Not tested: live Codex subscription WebSocket

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Security CI currently flags `click 8.3.1` and `pillow 12.2.0`, so this
PR carries the narrow floor bump to `click>=8.3.3` and `pillow>=12.3.0`
as a supply-chain unblock for the same final merge snapshot.
- `CHANGELOG.md` remains untouched because Headroom generates release
notes from conventional commits.
- This PR is scoped to the OpenAI Responses WebSocket relay lifecycle;
it does not change the HTTP `/v1/responses` path or the Anthropic
handler.

---------

Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-13 14:42:27 -04:00