fix(wrap): stop blocking the launch on the Serena pre-index (#3557)

## Description

`headroom wrap` ran `serena project index` synchronously, so the agent
did not start until the index finished or hit the stall budget (300s by
default, configurable since #3183). As reported in #3436, that is a wait
the user always pays and often gets nothing for: in a directory holding
several repositories the index cannot finish inside any sensible budget,
so every launch waited the whole budget out and then threw the partial
work away. Even a small, already-warm project paid about 15s per launch
for a cache Serena fills lazily anyway.

This moves the index off the launch path. It is now spawned and left to
run alongside the agent, so the launch is immediate and the cache still
warms. Serena's MCP server answers symbol queries from whatever is
already cached and indexes the rest on demand, so nothing is lost by not
waiting.

Closes #3436

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `_index_serena_project` now spawns `serena project index` and returns
immediately instead of calling `communicate(timeout=...)`. It prints one
line saying the index is warming in the background.
- The child's `stdout`/`stderr` are now `DEVNULL` rather than pipes.
This is load-bearing, not tidying: nothing drains those pipes once the
wrap stops waiting, and the indexer writes a progress line per file, so
a full 64 KiB pipe buffer would block the child permanently. Serena
already records its own failures in `.serena/logs/indexing.txt`.
- New `_stop_background_serena_index`, registered with `atexit`, stops
the child when the wrap exits so a long index does not outlive the
session that started it. It reuses the existing
`_kill_serena_index_tree`, which already handles the `uvx` → `serena`
grandchild on both platforms (#2938).
- `HEADROOM_SERENA_INDEX_TIMEOUT` keeps its name and its meaning — how
long the launch may wait — with a new default of 0. A positive integer
opts back into the previous blocking behaviour with that budget; an
unusable value warns and falls back to the background, as it did before.
`_resolve_serena_index_timeout_seconds` is renamed
`_resolve_serena_index_wait_seconds` and the `_SERENA_INDEX_TIMEOUT =
300` constant is gone, since nothing defaults to it any more.
- Tests in `tests/test_cli/test_wrap_serena_boost.py` are updated for
the new default and extended: the launch never waits, output is
redirected rather than piped, the exit hook kills a running index and
leaves a finished one alone, and the blocking path still works when the
env var asks for it.

### Two things a reviewer may want to check

**Killing a half-finished index is cheap.** Upstream
`ProjectCommands._index_project` calls `ls_mgr.save_all_caches()` every
30 seconds during the loop, and `request_document_symbols` skips files
whose content hash it has already cached. So a killed index loses at
most the last 30 seconds of work, and the next wrap resumes roughly
where this one stopped rather than starting over. A hard kill of the
wrap process itself (SIGKILL, an ungraceful terminal close) skips
`atexit` and does leave the indexer running — that is the one gap.

**The indexer and Serena's MCP server now write `.serena/cache/**`
concurrently**, where previously the pre-index always finished first.
Upstream `solidlsp.util.cache.save_cache` writes the pickle in place
with no atomic rename, so a torn write is possible. Serena tolerates
this by design: `_load_document_symbols_cache` wraps the load in
`try/except` with the comment *"cache can become corrupt, so just skip
loading it"*, and `_save_document_symbols_cache` catches and logs rather
than raising. The worst case is a cache rebuild, not a failure.

Issue #3436 offered a second option — skip the pre-index whenever a
`.serena/cache` directory already exists. I did not take it: it leaves
the first and most expensive launch blocking, and a cache that exists
but is stale or partial would then never be refreshed. Backgrounding
removes the wait in both cases.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_cli/test_wrap_serena_boost.py -q -p no:randomly
======================== 49 passed, 1 skipped in 0.55s ========================

$ python -m pytest tests/test_cli/ -q -p no:randomly --ignore=tests/test_cli/test_mcp.py
============ 5 failed, 723 passed, 2 skipped, 2 warnings in 39.52s ============
# The 5 failures and the test_mcp.py collection error reproduce identically on
# an unmodified upstream/main checkout on this machine: Windows has no
# signal.SIGHUP, the symlink tests need privileges, and the local pydantic-core
# is newer than pydantic expects. None involve the changed code.

$ python -m ruff check . && python -m ruff format --check .
All checks passed!
1577 files already formatted

$ python -m mypy headroom --ignore-missing-imports --python-version 3.13
Found 12 errors in 3 files (checked 532 source files)
# All 12 pre-exist in headroom/ccr/mcp_server.py, headroom/memory/mcp_server.py
# and headroom/release_version.py (local MCP SDK version drift). Zero in
# headroom/cli/wrap.py. Local mypy here is 2.3.1, not CI's 1.20.2.
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, uvx 0.11.28, serena-agent
from PyPI, real `serena project index` child processes (no mocks). A
scratch project holding 533 real `.py` files with a
`.serena/project.yml` and a cold cache.
- Exact command / steps: imported the patched `headroom.cli.wrap` from
this branch, `chdir` into the scratch project, then (1) called
`_index_serena_project(verbose=True)` and timed the return, checked the
child with `tasklist`, slept, and inspected `.serena/cache`; (2)
repeated it and called `_stop_background_serena_index()` while the child
was still running; (3) repeated with `HEADROOM_SERENA_INDEX_TIMEOUT=5`
and with `HEADROOM_SERENA_INDEX_TIMEOUT=60s`.
- Observed result: (1) `_index_serena_project() returned in 0.01s`,
printing `Serena: indexing project in the background (symbol tools work
meanwhile)`; the child kept running afterwards and produced
`document_symbols.pkl` (31,492,659 bytes) and `raw_document_symbols.pkl`
(6,048,168 bytes) on its own. (2) `2s later: child pid 29672 alive=True
(poll=None)` then `after the exit hook: child pid 29672 alive=False
(poll=1)`, handle cleared; the cache was empty at that point, which
matches Serena's 30-second flush cadence — a kill inside the first
window loses that window. (3) with `=5` the call blocked and printed
`Serena: pre-indexing project (waiting up to 5s)…` then `Serena: project
pre-indexed (symbol cache warmed)`, returning in 2.46s with nothing left
for the exit hook; with `=60s` it printed `Serena: ignoring
HEADROOM_SERENA_INDEX_TIMEOUT='60s' (want a positive integer number of
seconds) — indexing in the background instead` and returned in 0.01s.
- Not tested: a full `headroom wrap claude` end-to-end launch (the proof
exercises the changed function directly against real Serena rather than
starting an agent); POSIX behaviour — `start_new_session` and the
`os.killpg` branch of `_kill_serena_index_tree` are covered only by unit
tests on this Windows machine; the reporter's own multi-repository Go
tree; and the concurrent cache-write case is reasoned from upstream
Serena source, not reproduced.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — this is not behind a rollout
channel. It is the Serena pre-index step of `headroom wrap`, which runs
whenever `--code-memory serena` (the default) is active and the project
already has `.serena/project.yml`.
- Minimum rollout channel: n/a (stable; ships with the next release).
- Stable/default behavior changed: yes. The default pre-index goes from
blocking the launch for up to 300s to running in the background. No
config or flag changes name or meaning; `HEADROOM_SERENA_INDEX_TIMEOUT`
changes only its default.
- Kill switch / disable path: set `HEADROOM_SERENA_INDEX_TIMEOUT` to a
positive integer to restore the previous blocking pre-index with that
budget (`HEADROOM_SERENA_INDEX_TIMEOUT=300` reproduces today's default
exactly). `--code-memory none` still disables Serena entirely.
- Unsafe override required: no.
- Qualification impact: none beyond the updated unit tests. The
pre-index has always been best-effort and non-fatal, and it still is;
nothing downstream reads its result.
- Rollback path: revert this commit. It touches one function, one new
helper and one env-var resolver in `headroom/cli/wrap.py`, with no
persisted state, no config migration and no on-disk format change — a
revert restores the old behaviour immediately.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

"I have made corresponding changes to the documentation" means the
in-file documentation: `HEADROOM_SERENA_INDEX_TIMEOUT` is documented
only in `headroom/cli/wrap.py` and its test module, both updated here,
so there is no user-facing doc page to change. No screenshots — this is
CLI timing behaviour.

@wallrj offered to test a PR in #3436; this is ready for that if you
would like to try it against the cert-manager tree.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Parideboy
2026-09-24 11:32:55 -05:00
committed by GitHub
co-authored by Claude Opus 5 JD Davis
parent 5baa439ebf
commit df8ebdf97d
2 changed files with 387 additions and 196 deletions
+105 -47
View File
@@ -19,6 +19,7 @@ Usage:
from __future__ import annotations
import atexit
import errno
import importlib.util
import io
@@ -2142,43 +2143,49 @@ def _serena_project_skip_reason(root: Path) -> str | None:
return None
#: Upper bound on the synchronous pre-index. The agent does not launch until
#: this call returns, so the number is a stall budget, not just a safety net.
_SERENA_INDEX_TIMEOUT = 300
#: Env knob for the launch-blocking pre-index. Unset (the default) means the
#: wrap does not wait at all: ``serena project index`` runs in the background
#: while the agent starts (#3436). A positive integer restores the pre-#3436
#: behaviour of blocking the launch for at most that many seconds.
_SERENA_INDEX_TIMEOUT_ENV = "HEADROOM_SERENA_INDEX_TIMEOUT"
def _resolve_serena_index_timeout_seconds() -> int:
"""Resolve the Serena pre-index stall budget from env, else the default.
def _resolve_serena_index_wait_seconds() -> int:
"""Seconds the launch may block on the pre-index (0 = do not wait).
A wrap launched from a directory Serena has already claimed re-indexes the
whole tree on every run, and 300s of that is time the agent is not running
(#3093). The budget is therefore tunable per environment, which also keeps
it reachable from ``wrap ... -- agents`` sessions that take no flags.
The pre-index used to be synchronous, so this knob sized a stall budget
(#3093). On a directory holding several repositories no budget is big
enough — the index cannot finish, so every launch waited the whole budget
out and then threw the partial work away (#3436). The default is therefore
now 0: the index runs in the background and the agent starts immediately.
Unlike :func:`_resolve_wrap_proxy_timeout_seconds`, a bad value is not
fatal here: the pre-index is best-effort, so an unusable setting falls back
to the default rather than aborting a launch that would otherwise succeed.
It is reported unconditionally, because a knob that looks applied but is
not is the failure this issue is about.
Setting the variable to a positive integer opts back into a blocking
pre-index with that budget, which is the escape hatch for anyone who wants
a warm cache before the first prompt (and the way to get the old behaviour
back).
A bad value is not fatal: the pre-index is best-effort, so an unusable
setting falls back to the background default rather than aborting a launch
that would otherwise succeed. It is reported unconditionally, because a
knob that looks applied but is not is the failure #3093 was about.
"""
raw = os.environ.get(_SERENA_INDEX_TIMEOUT_ENV, "").strip()
if not raw:
return _SERENA_INDEX_TIMEOUT
return 0
timeout_seconds: int | None
wait_seconds: int | None
try:
timeout_seconds = int(raw)
wait_seconds = int(raw)
except ValueError:
timeout_seconds = None
if timeout_seconds is None or timeout_seconds <= 0:
wait_seconds = None
if wait_seconds is None or wait_seconds <= 0:
click.echo(
f" Serena: ignoring {_SERENA_INDEX_TIMEOUT_ENV}={raw!r} "
f"(want a positive integer number of seconds) "
f"— using {_SERENA_INDEX_TIMEOUT}s"
"— indexing in the background instead"
)
return _SERENA_INDEX_TIMEOUT
return timeout_seconds
return 0
return wait_seconds
def _kill_serena_index_tree(proc: subprocess.Popen) -> None:
@@ -2229,6 +2236,38 @@ def _kill_serena_index_tree(proc: subprocess.Popen) -> None:
pass
#: Handle on the background ``serena project index`` child, so the wrap can
#: stop it when the session ends instead of leaving it to grind on unowned.
_SERENA_INDEX_PROC: subprocess.Popen | None = None
def _stop_background_serena_index() -> None:
"""Stop the background pre-index when the wrap exits (best-effort).
Registered with :mod:`atexit` rather than plumbed into ``_launch_tool``'s
cleanup: the wrap stays alive as the agent's parent for the whole session,
Ctrl-C is swallowed by ``_ignore_child_sigint``, and SIGTERM/SIGHUP raise
``SystemExit`` through ``_exit_on_signal`` (#3205), so every ordinary exit
path unwinds normally and runs this.
Killing a half-finished index is cheap: ``serena project index`` flushes
its caches every 30s while it runs, and ``request_document_symbols`` skips
files whose content hash it has already cached, so the next wrap resumes
roughly where this one stopped instead of starting over.
"""
global _SERENA_INDEX_PROC
proc, _SERENA_INDEX_PROC = _SERENA_INDEX_PROC, None
if proc is None:
return
try:
if proc.poll() is not None: # already finished — nothing to kill
return
except Exception:
return
_kill_serena_index_tree(proc)
def _index_serena_project(*, verbose: bool = False) -> None:
"""Warm Serena's symbol cache for the current project (non-fatal).
@@ -2237,36 +2276,45 @@ def _index_serena_project(*, verbose: bool = False) -> None:
query is not paying for a cold index. Serena also indexes lazily on demand,
so any failure here is survivable.
This runs on the launch path, synchronously: the agent starts only once it
returns, so the timeout below is time the user spends staring at nothing —
``HEADROOM_SERENA_INDEX_TIMEOUT`` resizes that budget (#3093). Two guards
keep it bounded (#2938):
This does **not** block the launch (#3436). The index used to run
synchronously, which meant a directory holding several repositories waited
out the whole stall budget on every single launch and then threw the
partial work away — no budget is large enough there, because the per-file
cost is a language server rebuilding its workspace view for a root it
cannot resolve. The child is started and left to run alongside the agent;
Serena's MCP server serves symbol queries from whatever is cached, and
indexes the rest on demand, while it warms.
``HEADROOM_SERENA_INDEX_TIMEOUT`` opts back into a bounded blocking wait.
Three guards keep the child harmless:
* ``stdin`` is ``DEVNULL``. Serena prompts when it has to auto-create
``project.yml``, and because stdout is captured the question never
reaches the terminal — an inherited stdin turned that into a silent,
full-timeout hang. EOF makes it fail in about a second instead.
``project.yml``, and because its output is redirected the question never
reaches the terminal — an inherited stdin turned that into a silent hang
(#2938). EOF makes it fail in about a second instead.
``_serena_project_skip_reason`` already keeps us out of that state; this
is the belt-and-braces half, and it covers any future Serena prompt too.
* The child gets its own process group so ``_kill_serena_index_tree`` can
take out the ``uvx`` grandchild on timeout rather than orphaning it.
* ``stdout``/``stderr`` are ``DEVNULL`` rather than pipes. Nothing drains
them once the wrap stops waiting, and the indexer writes a progress line
per file, so a pipe would fill its buffer and wedge the child for good.
Serena records its own failures in ``.serena/logs/indexing.txt``.
* The child gets its own process group, both so a terminal Ctrl-C aimed at
the agent does not reach it and so ``_kill_serena_index_tree`` can take
out the ``uvx`` grandchild rather than orphaning it (#2938).
"""
global _SERENA_INDEX_PROC
if shutil.which("uvx") is None:
if verbose:
click.echo(" Serena: uvx not found — skipping pre-index")
return
timeout_seconds = _resolve_serena_index_timeout_seconds()
wait_seconds = _resolve_serena_index_wait_seconds()
popen_kwargs: dict[str, Any] = {
"stdout": subprocess.PIPE,
"stderr": subprocess.PIPE,
"stdout": subprocess.DEVNULL,
"stderr": subprocess.DEVNULL,
"stdin": subprocess.DEVNULL,
"text": True,
# ``subprocess.Popen`` directly, so the encoding defaults that
# ``headroom._subprocess.run`` applies have to be repeated here.
"encoding": "utf-8",
"errors": "replace",
"cwd": str(Path.cwd()),
}
if sys.platform == "win32":
@@ -2293,11 +2341,18 @@ def _index_serena_project(*, verbose: bool = False) -> None:
click.echo(f" Serena: pre-index skipped ({e})")
return
# Announce the wait. Indexing a large repo legitimately takes minutes and
# the output is captured, so without this line the wrap looks hung.
click.echo(" Serena: pre-indexing project (first run can take a while)…")
if wait_seconds <= 0:
_SERENA_INDEX_PROC = proc
atexit.register(_stop_background_serena_index)
click.echo(" Serena: indexing project in the background (symbol tools work meanwhile)")
return
# Opted back into a blocking pre-index. Announce the wait: indexing a large
# repo legitimately takes minutes and the output is redirected, so without
# this line the wrap looks hung.
click.echo(f" Serena: pre-indexing project (waiting up to {wait_seconds}s)…")
try:
_stdout, stderr = proc.communicate(timeout=timeout_seconds)
proc.wait(timeout=wait_seconds)
except subprocess.TimeoutExpired:
_kill_serena_index_tree(proc)
click.echo(" Serena: pre-index timed out (will index on demand)")
@@ -2311,7 +2366,10 @@ def _index_serena_project(*, verbose: bool = False) -> None:
if proc.returncode == 0:
click.echo(" Serena: project pre-indexed (symbol cache warmed)")
elif verbose:
click.echo(f" Serena: pre-index failed ({(stderr or '')[:100]})")
click.echo(
f" Serena: pre-index failed (exit {proc.returncode}; "
f"see .serena/logs/ for Serena's own log)"
)
def _setup_serena_mcp(
@@ -2385,9 +2443,9 @@ def _setup_serena_mcp(
# Serena is the active engine here (we passed the detect/uvx guards): steer
# the agent toward symbol-level tools, then warm the symbol cache. Both are
# best-effort and non-fatal, but the pre-index is *synchronous* — the agent
# does not launch until it returns or hits ``_SERENA_INDEX_TIMEOUT``. See
# ``_index_serena_project`` for how that wait is kept bounded and visible.
# best-effort and non-fatal, and neither blocks the launch: the pre-index
# runs alongside the agent and is stopped when the wrap exits (#3436). See
# ``_index_serena_project`` for the guards on that child process.
#
# Headroom no longer writes ``.serena/project.yml`` language scoping. Serena
# determines the project's languages itself during
+282 -149
View File
@@ -1,5 +1,6 @@
"""Serena "boost" wrap-time helpers: prefer-Serena instruction injection,
repo-language scoping of ``.serena/project.yml``, and symbol-cache pre-indexing.
repo-language scoping of ``.serena/project.yml``, and background symbol-cache
pre-indexing.
All Serena subprocess calls are mocked — these tests never invoke real ``uvx``.
"""
@@ -7,6 +8,7 @@ All Serena subprocess calls are mocked — these tests never invoke real ``uvx``
from __future__ import annotations
import subprocess
from collections.abc import Iterator
from pathlib import Path
from unittest.mock import Mock
@@ -95,7 +97,7 @@ def test_instruction_file_target_per_agent(tmp_path: Path, monkeypatch: pytest.M
# ---------------------------------------------------------------------------
# _index_serena_project — best-effort, timeout-guarded pre-index
# _index_serena_project — spawned in the background, never on the launch path
# ---------------------------------------------------------------------------
@@ -114,31 +116,33 @@ class _FakeProc:
self,
*,
returncode: int = 0,
stderr: str = "",
communicate_error: BaseException | None = None,
poll_result: int | None = None,
wait_error: BaseException | None = None,
) -> None:
self.pid = 4242
self.returncode = returncode
self.stdin = None
self.stdout = None
self.stderr = None
self._stderr = stderr
self._communicate_error = communicate_error
self._poll_result = poll_result
self._wait_error = wait_error
self.waits: list[float | None] = []
self.killed = False
self.waited = False
def communicate(self, timeout: float | None = None) -> tuple[str, str]:
if self._communicate_error is not None:
raise self._communicate_error
return "", self._stderr
def poll(self) -> int | None:
return self._poll_result
def wait(self, timeout: float | None = None) -> int:
self.waits.append(timeout)
self.waited = True
if self._wait_error is not None:
raise self._wait_error
return self.returncode
def kill(self) -> None:
self.killed = True
def wait(self, timeout: float | None = None) -> int:
self.waited = True
return self.returncode
def _stub_popen(monkeypatch: pytest.MonkeyPatch, proc: _FakeProc) -> Mock:
mock_popen = Mock(return_value=proc)
@@ -146,9 +150,29 @@ def _stub_popen(monkeypatch: pytest.MonkeyPatch, proc: _FakeProc) -> Mock:
return mock_popen
@pytest.fixture(autouse=True)
def _no_leaked_index_child() -> Iterator[None]:
"""Never let a fake child stay parked in the module global.
``_index_serena_project`` stashes the background child so the atexit hook
can stop it. Leaving a mock there would make an unrelated test's hook act
on it — and would fire at interpreter shutdown.
"""
yield
wrap_cli._SERENA_INDEX_PROC = None
def _stub_atexit(monkeypatch: pytest.MonkeyPatch) -> list[object]:
"""Collect atexit registrations instead of really queueing them."""
registered: list[object] = []
monkeypatch.setattr(wrap_cli.atexit, "register", registered.append)
return registered
def test_preindex_runs_serena_in_cwd(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
mock_popen = _stub_popen(monkeypatch, _FakeProc())
wrap_cli._index_serena_project()
@@ -164,25 +188,47 @@ def test_preindex_runs_serena_in_cwd(tmp_path: Path, monkeypatch: pytest.MonkeyP
assert kwargs["cwd"] == str(tmp_path) # invoked in the project cwd
def test_preindex_is_timeout_guarded(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
def test_preindex_does_not_block_the_launch(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""The whole point of #3436: spawn, say so, and let the agent start."""
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
registered = _stub_atexit(monkeypatch)
proc = _FakeProc()
_stub_popen(monkeypatch, proc)
seen: list[float | None] = []
monkeypatch.setattr(
proc, "communicate", lambda timeout=None: (seen.append(timeout), ("", ""))[1]
)
wrap_cli._index_serena_project()
assert seen == [wrap_cli._SERENA_INDEX_TIMEOUT]
assert proc.waits == [] # never waited on
assert wrap_cli._SERENA_INDEX_PROC is proc # kept, so exit can stop it
assert registered == [wrap_cli._stop_background_serena_index]
assert "background" in capsys.readouterr().out
def test_preindex_discards_child_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Pipes would wedge the child: nothing drains them once we stop waiting.
``serena project index`` writes a progress line per file, so a full pipe
buffer would block it forever instead of letting the cache warm.
"""
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
mock_popen = _stub_popen(monkeypatch, _FakeProc())
wrap_cli._index_serena_project()
kwargs = mock_popen.call_args.kwargs
assert kwargs["stdout"] == subprocess.DEVNULL
assert kwargs["stderr"] == subprocess.DEVNULL
def test_preindex_never_inherits_stdin(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Serena prompts ``[y/N]`` behind a captured stdout; stdin must be EOF (#2938)."""
"""Serena prompts ``[y/N]`` behind redirected output; stdin must be EOF (#2938)."""
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
mock_popen = _stub_popen(monkeypatch, _FakeProc())
wrap_cli._index_serena_project()
@@ -193,9 +239,11 @@ def test_preindex_never_inherits_stdin(tmp_path: Path, monkeypatch: pytest.Monke
def test_preindex_child_gets_its_own_process_group(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Without this the ``uvx`` grandchild survives the timeout kill (#2938)."""
"""Keeps Ctrl-C off the indexer, and lets the exit hook take out the ``uvx``
grandchild rather than orphaning it (#2938)."""
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
mock_popen = _stub_popen(monkeypatch, _FakeProc())
wrap_cli._index_serena_project()
@@ -217,12 +265,215 @@ def test_preindex_skips_without_uvx(monkeypatch: pytest.MonkeyPatch) -> None:
mock_popen.assert_not_called()
def test_preindex_timeout_kills_the_tree_and_is_non_fatal(
def test_preindex_spawn_failure_is_non_fatal(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
proc = _FakeProc(communicate_error=subprocess.TimeoutExpired(cmd="serena", timeout=1))
registered = _stub_atexit(monkeypatch)
monkeypatch.setattr(wrap_cli.subprocess, "Popen", Mock(side_effect=OSError("no exec")))
wrap_cli._index_serena_project(verbose=True) # must not propagate
assert registered == [] # nothing to stop later
assert wrap_cli._SERENA_INDEX_PROC is None
# ---------------------------------------------------------------------------
# _stop_background_serena_index — the indexer does not outlive the session
# ---------------------------------------------------------------------------
def test_exit_stops_a_still_running_index(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Otherwise a 13-minute index grinds on with nobody left to want it."""
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
proc = _FakeProc(poll_result=None) # still running
_stub_popen(monkeypatch, proc)
killed: list[object] = []
monkeypatch.setattr(wrap_cli, "_kill_serena_index_tree", killed.append)
wrap_cli._index_serena_project()
wrap_cli._stop_background_serena_index()
assert killed == [proc]
def test_exit_leaves_a_finished_index_alone(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
_stub_popen(monkeypatch, _FakeProc(poll_result=0)) # already exited
killed: list[object] = []
monkeypatch.setattr(wrap_cli, "_kill_serena_index_tree", killed.append)
wrap_cli._index_serena_project()
wrap_cli._stop_background_serena_index()
assert killed == []
def test_exit_hook_kills_once_and_is_safe_to_repeat(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""``atexit`` can hold several registrations from one process."""
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
proc = _FakeProc(poll_result=None)
_stub_popen(monkeypatch, proc)
killed: list[object] = []
monkeypatch.setattr(wrap_cli, "_kill_serena_index_tree", killed.append)
wrap_cli._index_serena_project()
wrap_cli._stop_background_serena_index()
wrap_cli._stop_background_serena_index()
assert killed == [proc]
def test_exit_hook_without_a_child_is_a_noop(monkeypatch: pytest.MonkeyPatch) -> None:
"""The hook also runs for wraps that never reached the pre-index."""
monkeypatch.setattr(
wrap_cli,
"_kill_serena_index_tree",
Mock(side_effect=AssertionError("nothing to kill")),
)
wrap_cli._stop_background_serena_index() # no exception
def test_exit_hook_survives_an_unpollable_child(monkeypatch: pytest.MonkeyPatch) -> None:
"""Shutdown is no place to raise: a broken handle just means no kill."""
proc = Mock()
proc.poll.side_effect = OSError("handle gone")
wrap_cli._SERENA_INDEX_PROC = proc
monkeypatch.setattr(
wrap_cli,
"_kill_serena_index_tree",
Mock(side_effect=AssertionError("must not kill an unpollable child")),
)
wrap_cli._stop_background_serena_index() # no exception
# ---------------------------------------------------------------------------
# HEADROOM_SERENA_INDEX_TIMEOUT — opting back into a blocking pre-index (#3436)
# ---------------------------------------------------------------------------
def _clear_index_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
"""Resolve from a known-empty environment, not the developer's shell."""
monkeypatch.delenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raising=False)
def test_index_wait_defaults_to_not_waiting(monkeypatch: pytest.MonkeyPatch) -> None:
"""Unset means background, which is what #3436 asked for."""
_clear_index_timeout(monkeypatch)
assert wrap_cli._resolve_serena_index_wait_seconds() == 0
def test_index_wait_reads_the_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "45")
assert wrap_cli._resolve_serena_index_wait_seconds() == 45
@pytest.mark.parametrize("raw", [" 30 ", "\t30\n"])
def test_index_wait_tolerates_surrounding_whitespace(
raw: str, monkeypatch: pytest.MonkeyPatch
) -> None:
"""An env var exported from a shell heredoc keeps its padding."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raw)
assert wrap_cli._resolve_serena_index_wait_seconds() == 30
@pytest.mark.parametrize("raw", ["", " "])
def test_index_wait_treats_a_blank_value_as_unset(
raw: str, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""``export HEADROOM_SERENA_INDEX_TIMEOUT=`` is not a misconfiguration."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raw)
assert wrap_cli._resolve_serena_index_wait_seconds() == 0
assert capsys.readouterr().out == "" # no warning noise on the default path
def test_index_wait_accepts_the_smallest_useful_budget(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "1")
assert wrap_cli._resolve_serena_index_wait_seconds() == 1
def test_index_wait_accepts_a_long_budget(monkeypatch: pytest.MonkeyPatch) -> None:
"""A deliberately huge monorepo may want to block for a long time."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "86400")
assert wrap_cli._resolve_serena_index_wait_seconds() == 86400
@pytest.mark.parametrize("raw", ["0", "-1", "abc", "30s", "1.5", "1e3", "0x10", "None"])
def test_index_wait_falls_back_to_the_background_on_an_unusable_value(
raw: str, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Best-effort means degrade to the default, never abort the launch."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raw)
assert wrap_cli._resolve_serena_index_wait_seconds() == 0
# A silently-ignored knob is the bug #3093 was about, so say so — and quote
# the value back, since the usual cause is a unit suffix the parser rejects.
out = capsys.readouterr().out
assert wrap_cli._SERENA_INDEX_TIMEOUT_ENV in out
assert repr(raw) in out
def test_index_wait_survives_a_float_unrepresentable_budget(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""``int()`` accepts integers ``float()`` cannot hold; resolving must not raise.
``wait`` would go on to raise ``OverflowError`` adding that to a monotonic
clock, which the caller's generic handler already absorbs as a non-fatal
skip — so the budget is honoured as given rather than clamped.
"""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "1" + "0" * 400)
assert wrap_cli._resolve_serena_index_wait_seconds() == 10**400
def test_preindex_blocks_for_the_env_budget(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Setting the knob restores the pre-#3436 blocking wait."""
monkeypatch.chdir(tmp_path)
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "5")
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
proc = _FakeProc()
_stub_popen(monkeypatch, proc)
wrap_cli._index_serena_project()
assert proc.waits == [5]
assert wrap_cli._SERENA_INDEX_PROC is None # waited for, nothing to stop later
def test_preindex_timeout_kills_the_tree_and_is_non_fatal(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "5")
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
proc = _FakeProc(wait_error=subprocess.TimeoutExpired(cmd="serena", timeout=5))
_stub_popen(monkeypatch, proc)
killed: list[object] = []
monkeypatch.setattr(wrap_cli, "_kill_serena_index_tree", killed.append)
@@ -236,8 +487,10 @@ def test_preindex_generic_error_kills_the_tree_and_is_non_fatal(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "5")
_stub_uvx(monkeypatch)
proc = _FakeProc(communicate_error=RuntimeError("boom"))
_stub_atexit(monkeypatch)
proc = _FakeProc(wait_error=RuntimeError("boom"))
_stub_popen(monkeypatch, proc)
killed: list[object] = []
monkeypatch.setattr(wrap_cli, "_kill_serena_index_tree", killed.append)
@@ -247,141 +500,21 @@ def test_preindex_generic_error_kills_the_tree_and_is_non_fatal(
assert killed == [proc]
def test_preindex_spawn_failure_is_non_fatal(
def test_preindex_backgrounds_on_an_unusable_budget(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.chdir(tmp_path)
_stub_uvx(monkeypatch)
monkeypatch.setattr(wrap_cli.subprocess, "Popen", Mock(side_effect=OSError("no exec")))
wrap_cli._index_serena_project(verbose=True) # must not propagate
# ---------------------------------------------------------------------------
# HEADROOM_SERENA_INDEX_TIMEOUT — the stall budget is tunable (#3093)
# ---------------------------------------------------------------------------
def _clear_index_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
"""Resolve from a known-empty environment, not the developer's shell."""
monkeypatch.delenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raising=False)
def test_index_timeout_defaults_when_env_is_unset(monkeypatch: pytest.MonkeyPatch) -> None:
_clear_index_timeout(monkeypatch)
assert wrap_cli._resolve_serena_index_timeout_seconds() == wrap_cli._SERENA_INDEX_TIMEOUT
def test_index_timeout_reads_the_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "45")
assert wrap_cli._resolve_serena_index_timeout_seconds() == 45
@pytest.mark.parametrize("raw", [" 30 ", "\t30\n"])
def test_index_timeout_tolerates_surrounding_whitespace(
raw: str, monkeypatch: pytest.MonkeyPatch
) -> None:
"""An env var exported from a shell heredoc keeps its padding."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raw)
assert wrap_cli._resolve_serena_index_timeout_seconds() == 30
@pytest.mark.parametrize("raw", ["", " "])
def test_index_timeout_treats_a_blank_value_as_unset(
raw: str, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""``export HEADROOM_SERENA_INDEX_TIMEOUT=`` is not a misconfiguration."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raw)
assert wrap_cli._resolve_serena_index_timeout_seconds() == wrap_cli._SERENA_INDEX_TIMEOUT
assert capsys.readouterr().out == "" # no warning noise on the default path
def test_index_timeout_accepts_the_smallest_useful_budget(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "1")
assert wrap_cli._resolve_serena_index_timeout_seconds() == 1
def test_index_timeout_accepts_a_budget_longer_than_the_default(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A deliberately huge monorepo may want *more* than 300s, not less."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "86400")
assert wrap_cli._resolve_serena_index_timeout_seconds() == 86400
@pytest.mark.parametrize("raw", ["0", "-1", "abc", "30s", "1.5", "1e3", "0x10", "None"])
def test_index_timeout_falls_back_on_an_unusable_value(
raw: str, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Best-effort means degrade to the default, never abort the launch."""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, raw)
assert wrap_cli._resolve_serena_index_timeout_seconds() == wrap_cli._SERENA_INDEX_TIMEOUT
# A silently-ignored knob is the bug being fixed, so say so — and quote the
# value back, since the usual cause is a unit suffix the parser rejects.
out = capsys.readouterr().out
assert wrap_cli._SERENA_INDEX_TIMEOUT_ENV in out
assert repr(raw) in out
def test_index_timeout_survives_a_float_unrepresentable_budget(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""``int()`` accepts integers ``float()`` cannot hold; resolving must not raise.
``communicate`` would go on to raise ``OverflowError`` adding that to a
monotonic clock, which the caller's generic handler already absorbs as a
non-fatal skip — so the budget is honoured as given rather than clamped.
"""
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "1" + "0" * 400)
assert wrap_cli._resolve_serena_index_timeout_seconds() == 10**400
def test_preindex_uses_the_env_budget(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""The resolved budget reaches ``communicate`` — the point of #3093."""
monkeypatch.chdir(tmp_path)
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "5")
_stub_uvx(monkeypatch)
proc = _FakeProc()
_stub_popen(monkeypatch, proc)
seen: list[float | None] = []
monkeypatch.setattr(
proc, "communicate", lambda timeout=None: (seen.append(timeout), ("", ""))[1]
)
wrap_cli._index_serena_project()
assert seen == [5]
def test_preindex_still_runs_on_an_unusable_budget(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""A bad value must not skip the pre-index or propagate out of it."""
"""A bad value must not skip the pre-index or start blocking on it."""
monkeypatch.chdir(tmp_path)
monkeypatch.setenv(wrap_cli._SERENA_INDEX_TIMEOUT_ENV, "not-a-number")
_stub_uvx(monkeypatch)
_stub_atexit(monkeypatch)
proc = _FakeProc()
mock_popen = _stub_popen(monkeypatch, proc)
seen: list[float | None] = []
monkeypatch.setattr(
proc, "communicate", lambda timeout=None: (seen.append(timeout), ("", ""))[1]
)
wrap_cli._index_serena_project() # must not propagate
mock_popen.assert_called_once()
assert seen == [wrap_cli._SERENA_INDEX_TIMEOUT]
assert proc.waits == []
# ---------------------------------------------------------------------------