mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat(learn): add agy (Antigravity CLI) as an analysis backend (#3939)
## Description
Adds the Antigravity CLI (`agy`) as a keyless LLM backend for `headroom
learn`, next to `claude`, `gemini` and `codex`. Antigravity subscription
users with no raw API key can now run the analysis through `agy`.
`agy` cannot use the existing stdin path as is. `agy -p` takes the
prompt as a flag argument and never reads stdin, so a large digest would
hit OS argument-length limits. The only stdin path is `--input-format
stream-json`, which requires `--output-format stream-json`. The backend
therefore sends the prompt as a single NDJSON user event on stdin and
reads the answer from `result.response` in the final `result` event.
Everything else (wall-clock timeout, Windows shim fallback, failure
detail, fenced-JSON parsing) reuses the existing `run()` path that
`gemini` and `codex` use.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/analyzer.py`: add an `agy` / `agy-cli` entry to
`_CLI_BACKENDS` (`agy --input-format stream-json --output-format
stream-json`). `agy` is never auto-detected, and it only runs when
`HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` is set: `agy` has no zero-tool mode,
so transcript text could drive its tools. Without the opt-in,
`HEADROOM_LEARN_CLI=agy` and `--model agy-cli` fail before any
subprocess starts; with it, a warning is logged on every run.
- `headroom/learn/analyzer.py`: in `_call_cli_llm`, wrap the prompt as
`{"event": "user", "message": {"content": ...}}` for `agy-cli`, and
extract `result.response` from the stream with a new
`_agy_result_text()` helper before the existing `_strip_fenced_json()`
parsing. A missing `result` event surfaces as the existing "unparseable
output" error, with both ends of stdout.
- `tests/test_learn/test_analyzer.py`: tests that auto-detection never
selects `agy` (even when it is the only CLI on PATH and the opt-in is
set), that `HEADROOM_LEARN_CLI=agy` and `_call_cli_llm(..., "agy-cli")`
fail without the opt-in (no subprocess call), `HEADROOM_LEARN_CLI=agy`
with the opt-in, existing CLIs still taking precedence, the exact
command and stdin payload, result-event parsing (including a fenced
response and a literal U+2028), and a missing `result` event.
- `headroom/cli/learn.py`: check the agy opt-in during model resolution,
so `HEADROOM_LEARN_CLI=agy` and `--model agy-cli` without it exit 1 with
an `Error:` line before any sessions are scanned.
- `tests/test_cli_learn.py`: a CLI test for both paths without the
opt-in.
- `wiki/learn.md`: list `agy` among valid `HEADROOM_LEARN_CLI` values
and document the `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` opt-in and its
risk.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_learn/ tests/test_cli_learn.py -q
5 failed, 345 passed, 4 skipped in 10.78s
# The same 5 fail on unmodified upstream/main on this Windows machine (5 failed, 340 passed),
# all path/git related and unrelated to this change:
# test_analyzer.py::TestDigestBuilder::test_includes_project_info
# test_scanner.py::TestDecodeProjectPath::test_home_dir_username_stays_single_component
# test_writer.py::TestClaudeLocalMdStaysOutOfGit::{test_apply_adds_exclude_entry,
# test_second_run_does_not_duplicate_the_entry, test_tracked_file_warns_instead_of_excluding}
$ uv run --frozen --extra dev pytest tests/test_learn/test_analyzer.py -q -k agy # analyzer.py reverted
FAILED TestDetectDefaultModel::test_env_var_selects_agy_with_unsafe_opt_in
FAILED TestDetectDefaultModel::test_env_var_agy_without_unsafe_opt_in_raises
FAILED TestCallCliLlm::test_agy_cli_sends_stream_json_and_parses_result_event
FAILED TestCallCliLlm::test_agy_cli_result_with_unicode_line_separator
FAILED TestCallCliLlm::test_agy_cli_without_unsafe_opt_in_does_not_run
FAILED TestCallCliLlm::test_agy_cli_missing_result_event_raises
6 failed, 2 passed, 97 deselected
# All 8 pass with the change.
$ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/learn/analyzer.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, `agy` 1.2.14 signed in via subscription, no
API keys set.
- Exact command / steps: `HEADROOM_LEARN_CLI=agy
HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1 ANTHROPIC_API_KEY= OPENAI_API_KEY=
GEMINI_API_KEY= uv run --frozen --extra dev headroom learn` in this
repository (dry run).
- Observed result: `Analyzing with agy-cli...`, then `Sessions: 3 |
Calls: 109 | Failures: 19 (17.4%)`, `Recommendations: 8`, with `[WOULD
WRITE]` blocks for `CLAUDE.local.md` and `MEMORY.md`; exit 0. Before
wiring the backend I confirmed the protocol by hand: `agy -p -` answers
the literal prompt `-` and ignores stdin, while the stream-json
invocation returns the answer in `{"event": "result", "result":
{"status": "SUCCESS", "response": "..."}}`.
- Not tested: macOS/Linux `agy` installs; `agy` under a permission mode
that prompts for tool approval (my config is `always-proceed`), where a
tool request in print mode would be bounded by the existing
`HEADROOM_LEARN_CLI_TIMEOUT_SECS` wall-clock cap.
## Runtime Rollout Safety
- Rollout-managed feature(s): None (offline `headroom learn` CLI)
- Minimum rollout channel: dev
- Stable/default behavior changed: No. `agy` is never auto-detected; it
is only used when selected with `HEADROOM_LEARN_CLI=agy` / `--model
agy-cli` together with `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1`.
- Kill switch / disable path: Set an API key, `HEADROOM_LEARN_CLI`, or
`--model` to pick another backend.
- Unsafe override required: None
- Qualification impact: None
- Rollback path: Revert this PR.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A (CLI change).
## Additional Notes
- No tracking issue exists for an `agy` analysis backend, so `Closes #`
is left empty. Related Antigravity work (#3428 `headroom init -g agy`,
#3610 Antigravity transcripts in `headroom learn`) is independent of
this change.
- Unlike `claude-cli` (#3892), `agy` has no flag to start a session with
zero tools. I deliberately did not add `--dangerously-skip-permissions`:
the digest contains untrusted transcript content, and the backend should
run under the user's own permission settings.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3939"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -161,7 +161,7 @@ def learn(
|
||||
"""
|
||||
import os
|
||||
|
||||
from ..learn.analyzer import SessionAnalyzer, _detect_default_model
|
||||
from ..learn.analyzer import SessionAnalyzer, _detect_default_model, _require_agy_opt_in
|
||||
from ..learn.registry import auto_detect_plugins, get_plugin
|
||||
|
||||
# Flag-combination validation — reject contradictory/no-op combinations up
|
||||
@@ -202,7 +202,9 @@ def learn(
|
||||
# Resolve model early to fail fast with a clear message
|
||||
try:
|
||||
resolved_model = model or _detect_default_model()
|
||||
except RuntimeError as e:
|
||||
if resolved_model == "agy-cli":
|
||||
_require_agy_opt_in()
|
||||
except (RuntimeError, ValueError) as e:
|
||||
click.echo(f"Error: {e}")
|
||||
raise SystemExit(1) from None
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ structured recommendations for CLAUDE.md / MEMORY.md.
|
||||
|
||||
Supports any LLM provider via LiteLLM: Anthropic, OpenAI, Google, Bedrock,
|
||||
Ollama, and 100+ others. Auto-detects the best available model from env vars.
|
||||
Also supports CLI-based backends (claude, gemini, codex) for subscription
|
||||
Also supports CLI-based backends (claude, gemini, codex, agy) for subscription
|
||||
users without raw API keys.
|
||||
"""
|
||||
|
||||
@@ -92,8 +92,17 @@ _CLI_BACKENDS: list[tuple[str, str, list[str]]] = [
|
||||
),
|
||||
("gemini", "gemini-cli", ["gemini", "-p"]),
|
||||
("codex", "codex-cli", ["codex", "exec", "--skip-git-repo-check"]),
|
||||
# agy's -p takes the prompt as an argument, so stdin is only read as a
|
||||
# stream-json user event; the answer arrives in the final `result` event.
|
||||
("agy", "agy-cli", ["agy", "--input-format", "stream-json", "--output-format", "stream-json"]),
|
||||
]
|
||||
|
||||
# agy has no flag that starts a zero-tool session (--sandbox still writes
|
||||
# files, --mode plan still reads them), so transcript text could drive its
|
||||
# tools under the user's permissions. It is never auto-detected and only runs
|
||||
# when the user sets this env var to "1".
|
||||
_AGY_UNSAFE_OPT_IN_ENV = "HEADROOM_LEARN_ALLOW_UNSAFE_AGY"
|
||||
|
||||
# Set of valid CLI model identifiers, derived from _CLI_BACKENDS.
|
||||
_CLI_MODEL_IDS: set[str] = {model for _, model, _ in _CLI_BACKENDS}
|
||||
|
||||
@@ -152,13 +161,23 @@ def _resolve_timeout_secs(env_var: str, default: int) -> int:
|
||||
return value
|
||||
|
||||
|
||||
def _require_agy_opt_in() -> None:
|
||||
"""Raise unless the user explicitly accepted agy's unsandboxed tool access."""
|
||||
if os.environ.get(_AGY_UNSAFE_OPT_IN_ENV) != "1":
|
||||
raise ValueError(
|
||||
"The agy backend cannot run without tool access, so session text could "
|
||||
"make it write files or run shell commands under your permissions. "
|
||||
f"Set {_AGY_UNSAFE_OPT_IN_ENV}=1 to accept this risk and use it anyway."
|
||||
)
|
||||
|
||||
|
||||
def _detect_default_model() -> str:
|
||||
"""Pick the best available model based on API keys, env config, or CLI tools.
|
||||
|
||||
Priority order:
|
||||
1. API key present → use corresponding LiteLLM model
|
||||
2. HEADROOM_LEARN_CLI env var → use specified CLI backend
|
||||
3. Auto-detect installed CLI tools (claude > gemini > codex)
|
||||
3. Auto-detect installed CLI tools (claude > gemini > codex; agy is opt-in only)
|
||||
4. Raise RuntimeError with setup instructions
|
||||
"""
|
||||
# 1. API key detection (existing behavior)
|
||||
@@ -171,6 +190,8 @@ def _detect_default_model() -> str:
|
||||
if cli_override:
|
||||
for cli_name, model, _cmd in _CLI_BACKENDS:
|
||||
if cli_name == cli_override:
|
||||
if cli_name == "agy":
|
||||
_require_agy_opt_in()
|
||||
logger.info("HEADROOM_LEARN_CLI=%s — using %s CLI backend", cli_override, cli_name)
|
||||
return model
|
||||
valid = ", ".join(name for name, _, _ in _CLI_BACKENDS)
|
||||
@@ -180,7 +201,7 @@ def _detect_default_model() -> str:
|
||||
|
||||
# 3. Auto-detect installed CLI tools
|
||||
for cli_name, model, _cmd in _CLI_BACKENDS:
|
||||
if shutil.which(cli_name):
|
||||
if cli_name != "agy" and shutil.which(cli_name):
|
||||
logger.info("No API key found — auto-detected %s CLI as LLM backend", cli_name)
|
||||
return model
|
||||
|
||||
@@ -189,8 +210,9 @@ def _detect_default_model() -> str:
|
||||
" export ANTHROPIC_API_KEY=sk-ant-... → uses claude-sonnet-4-6\n"
|
||||
" export OPENAI_API_KEY=sk-... → uses gpt-4o\n"
|
||||
" export GEMINI_API_KEY=... → uses gemini-flash-latest\n"
|
||||
"Or set HEADROOM_LEARN_CLI to a coding agent CLI (claude, gemini, codex).\n"
|
||||
"Or install one of those CLIs for auto-detection.\n"
|
||||
"Or set HEADROOM_LEARN_CLI to a coding agent CLI (claude, gemini, codex, agy).\n"
|
||||
" agy is never auto-detected and also needs HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1.\n"
|
||||
"Or install claude, gemini, or codex for auto-detection.\n"
|
||||
"Or specify a model directly: headroom learn --model <litellm-model-name>"
|
||||
)
|
||||
|
||||
@@ -715,6 +737,8 @@ def _call_cli_llm(
|
||||
--include-partial-messages (idle-timeout)
|
||||
gemini-cli → gemini -p (wall-clock timeout)
|
||||
codex-cli → codex exec (wall-clock timeout)
|
||||
agy-cli → agy --input-format stream-json --output-format stream-json
|
||||
(wall-clock timeout)
|
||||
|
||||
The claude-cli path streams JSON events, letting the analyzer kill genuine
|
||||
hangs while letting long-but-active analyses run to completion.
|
||||
@@ -729,7 +753,8 @@ def _call_cli_llm(
|
||||
Parsed JSON recommendations from the CLI tool.
|
||||
|
||||
Raises:
|
||||
ValueError: If *model* is not a known CLI backend.
|
||||
ValueError: If *model* is not a known CLI backend, or is ``agy-cli``
|
||||
without the unsafe opt-in.
|
||||
RuntimeError: If the CLI is not installed, exits non-zero, or times out.
|
||||
"""
|
||||
cmd: list[str] | None = None
|
||||
@@ -740,6 +765,13 @@ def _call_cli_llm(
|
||||
if cmd is None:
|
||||
raise ValueError(f"Unknown CLI model: {model}")
|
||||
|
||||
if model == "agy-cli":
|
||||
_require_agy_opt_in()
|
||||
logger.warning(
|
||||
"agy-cli runs with tool access under your permissions; session text "
|
||||
"could trigger file writes or shell commands during analysis"
|
||||
)
|
||||
|
||||
prompt = _SYSTEM_PROMPT + "\n\n" + _USER_PROMPT_PREFIX + digest
|
||||
hard_cap = _resolve_timeout_secs("HEADROOM_LEARN_CLI_TIMEOUT_SECS", _CLI_TIMEOUT)
|
||||
|
||||
@@ -749,10 +781,14 @@ def _call_cli_llm(
|
||||
cmd, prompt, hard_cap=hard_cap, idle_cap=idle_cap, on_progress=on_progress
|
||||
)
|
||||
|
||||
stdin = prompt
|
||||
if model == "agy-cli":
|
||||
stdin = json.dumps({"event": "user", "message": {"content": prompt}}) + "\n"
|
||||
|
||||
try:
|
||||
result = run(
|
||||
cmd,
|
||||
input=prompt,
|
||||
input=stdin,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=hard_cap,
|
||||
@@ -766,7 +802,7 @@ def _call_cli_llm(
|
||||
) from None
|
||||
cmd = shim_cmd
|
||||
try:
|
||||
result = run(cmd, input=prompt, capture_output=True, text=True, timeout=hard_cap)
|
||||
result = run(cmd, input=stdin, capture_output=True, text=True, timeout=hard_cap)
|
||||
except FileNotFoundError:
|
||||
raise RuntimeError(
|
||||
f"`{cmd[0]}` not found in PATH. Install it or use a different backend "
|
||||
@@ -787,8 +823,12 @@ def _call_cli_llm(
|
||||
if result.stderr and result.stderr.strip():
|
||||
logger.debug("CLI stderr (exit 0): %s", result.stderr[:_MAX_SNIPPET_LEN])
|
||||
|
||||
output = result.stdout
|
||||
if model == "agy-cli":
|
||||
output = _agy_result_text(output)
|
||||
|
||||
try:
|
||||
return _strip_fenced_json(result.stdout)
|
||||
return _strip_fenced_json(output)
|
||||
except json.JSONDecodeError as exc:
|
||||
stdout_snippet = _output_snippet(result.stdout or "")
|
||||
raise RuntimeError(
|
||||
@@ -975,6 +1015,19 @@ def _call_claude_cli_streaming(
|
||||
) from exc
|
||||
|
||||
|
||||
def _agy_result_text(stdout: str) -> str:
|
||||
"""Return ``result.response`` from agy stream-json output, or "" if absent."""
|
||||
# Split on "\n" only: splitlines() also breaks on U+2028/U+2029, which JSON
|
||||
# allows unescaped inside strings, and would cut the result event apart.
|
||||
for line in reversed(stdout.split("\n")):
|
||||
event = _parse_stream_event(line)
|
||||
if event is not None and event.get("event") == "result":
|
||||
result = event.get("result")
|
||||
response = result.get("response") if isinstance(result, dict) else None
|
||||
return response if isinstance(response, str) else ""
|
||||
return ""
|
||||
|
||||
|
||||
def _parse_stream_event(line: str) -> dict | None:
|
||||
"""Parse one line of claude-cli stream-json output, returning None on junk."""
|
||||
line = line.strip()
|
||||
|
||||
@@ -112,6 +112,38 @@ def test_learn_exits_cleanly_when_model_detection_fails(
|
||||
assert "Error: no model" in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("args", "env"),
|
||||
[
|
||||
(["learn"], {"HEADROOM_LEARN_CLI": "agy"}),
|
||||
(["learn", "--model", "agy-cli"], {}),
|
||||
],
|
||||
)
|
||||
def test_learn_agy_without_unsafe_opt_in_exits_cleanly(
|
||||
monkeypatch: pytest.MonkeyPatch, runner: CliRunner, args: list[str], env: dict[str, str]
|
||||
) -> None:
|
||||
for var in (
|
||||
"ANTHROPIC_API_KEY",
|
||||
"OPENAI_API_KEY",
|
||||
"GEMINI_API_KEY",
|
||||
"HEADROOM_LEARN_CLI",
|
||||
"HEADROOM_LEARN_ALLOW_UNSAFE_AGY",
|
||||
):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
for var, value in env.items():
|
||||
monkeypatch.setenv(var, value)
|
||||
monkeypatch.setattr(
|
||||
"headroom.learn.registry.auto_detect_plugins",
|
||||
lambda: pytest.fail("sessions must not be scanned without the agy opt-in"),
|
||||
)
|
||||
|
||||
result = runner.invoke(main, args, catch_exceptions=False)
|
||||
|
||||
assert result.exit_code == 1
|
||||
assert "Error:" in result.output
|
||||
assert "HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1" in result.output
|
||||
|
||||
|
||||
def test_learn_auto_agent_reports_no_detected_plugins(
|
||||
monkeypatch: pytest.MonkeyPatch, runner: CliRunner
|
||||
) -> None:
|
||||
|
||||
@@ -626,6 +626,29 @@ class TestDetectDefaultModel:
|
||||
)
|
||||
assert _detect_default_model() == "codex-cli"
|
||||
|
||||
def test_cli_fallback_never_auto_detects_agy(self, monkeypatch):
|
||||
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
|
||||
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("HEADROOM_LEARN_CLI", raising=False)
|
||||
monkeypatch.setenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", "1")
|
||||
monkeypatch.setattr(
|
||||
"headroom.learn.analyzer.shutil.which",
|
||||
lambda name: f"/usr/bin/{name}" if name == "agy" else None,
|
||||
)
|
||||
with pytest.raises(RuntimeError, match="No LLM API key found"):
|
||||
_detect_default_model()
|
||||
|
||||
def test_cli_fallback_prefers_existing_clis_over_agy(self, monkeypatch):
|
||||
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
|
||||
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
|
||||
monkeypatch.setattr(
|
||||
"headroom.learn.analyzer.shutil.which",
|
||||
lambda name: f"/usr/bin/{name}" if name in ("codex", "agy") else None,
|
||||
)
|
||||
assert _detect_default_model() == "codex-cli"
|
||||
|
||||
def test_api_key_preferred_over_cli(self, monkeypatch):
|
||||
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-ant-test")
|
||||
monkeypatch.setattr(
|
||||
@@ -648,6 +671,23 @@ class TestDetectDefaultModel:
|
||||
monkeypatch.setenv("HEADROOM_LEARN_CLI", "codex")
|
||||
assert _detect_default_model() == "codex-cli"
|
||||
|
||||
def test_env_var_selects_agy_with_unsafe_opt_in(self, monkeypatch):
|
||||
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
|
||||
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
|
||||
monkeypatch.setenv("HEADROOM_LEARN_CLI", "agy")
|
||||
monkeypatch.setenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", "1")
|
||||
assert _detect_default_model() == "agy-cli"
|
||||
|
||||
def test_env_var_agy_without_unsafe_opt_in_raises(self, monkeypatch):
|
||||
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
|
||||
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", raising=False)
|
||||
monkeypatch.setenv("HEADROOM_LEARN_CLI", "agy")
|
||||
with pytest.raises(ValueError, match="HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1"):
|
||||
_detect_default_model()
|
||||
|
||||
def test_env_var_invalid_raises(self, monkeypatch):
|
||||
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
|
||||
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||
@@ -1031,6 +1071,64 @@ class TestCallCliLlm:
|
||||
cmd = mock_run.call_args[0][0]
|
||||
assert cmd == ["gemini", "-p"]
|
||||
|
||||
@patch("headroom.learn.analyzer.subprocess.run")
|
||||
def test_agy_cli_sends_stream_json_and_parses_result_event(
|
||||
self, mock_run: MagicMock, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", "1")
|
||||
response = '```json\n{"context_file_rules": [], "memory_file_rules": []}\n```\n'
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="\n".join(
|
||||
[
|
||||
'{"event": "init", "init": {"permission_mode": "default"}}',
|
||||
'{"event": "step_update", "step_update": {"text_delta": "{"}}',
|
||||
json.dumps({"event": "result", "result": {"response": response}}),
|
||||
]
|
||||
),
|
||||
stderr="",
|
||||
)
|
||||
result = _call_cli_llm("test digest", "agy-cli")
|
||||
assert result == {"context_file_rules": [], "memory_file_rules": []}
|
||||
cmd = mock_run.call_args[0][0]
|
||||
assert cmd == ["agy", "--input-format", "stream-json", "--output-format", "stream-json"]
|
||||
sent = json.loads(mock_run.call_args.kwargs["input"])
|
||||
assert sent["event"] == "user"
|
||||
assert sent["message"]["content"].endswith("test digest")
|
||||
|
||||
@patch("headroom.learn.analyzer.subprocess.run")
|
||||
def test_agy_cli_result_with_unicode_line_separator(self, mock_run: MagicMock, monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", "1")
|
||||
response = '{"context_file_rules": [], "memory_file_rules": [], "note": "a
b"}'
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout=json.dumps(
|
||||
{"event": "result", "result": {"response": response}}, ensure_ascii=False
|
||||
)
|
||||
+ "\n",
|
||||
stderr="",
|
||||
)
|
||||
result = _call_cli_llm("test digest", "agy-cli")
|
||||
assert result["note"] == "a
b"
|
||||
|
||||
@patch("headroom.learn.analyzer.subprocess.run")
|
||||
def test_agy_cli_without_unsafe_opt_in_does_not_run(self, mock_run: MagicMock, monkeypatch):
|
||||
monkeypatch.delenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", raising=False)
|
||||
with pytest.raises(ValueError, match="HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1"):
|
||||
_call_cli_llm("test digest", "agy-cli")
|
||||
mock_run.assert_not_called()
|
||||
|
||||
@patch("headroom.learn.analyzer.subprocess.run")
|
||||
def test_agy_cli_missing_result_event_raises(self, mock_run: MagicMock, monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_LEARN_ALLOW_UNSAFE_AGY", "1")
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout='{"event": "init", "init": {}}\n',
|
||||
stderr="",
|
||||
)
|
||||
with pytest.raises(RuntimeError, match="unparseable output"):
|
||||
_call_cli_llm("test digest", "agy-cli")
|
||||
|
||||
@patch("headroom.learn.analyzer.subprocess.run")
|
||||
def test_codex_nonzero_exit_raises(self, mock_run: MagicMock):
|
||||
mock_run.return_value = MagicMock(
|
||||
|
||||
+11
-1
@@ -218,7 +218,17 @@ export HEADROOM_LEARN_CLI=codex
|
||||
headroom learn
|
||||
```
|
||||
|
||||
Valid values for `HEADROOM_LEARN_CLI`: `claude`, `gemini`, `codex`.
|
||||
Valid values for `HEADROOM_LEARN_CLI`: `claude`, `gemini`, `codex`, `agy`.
|
||||
|
||||
`agy` (Antigravity CLI) is never auto-detected. It has no mode that starts a session without tools, so instructions embedded in analyzed session text could make it write files or run shell commands with your permissions. To use it anyway, opt in explicitly:
|
||||
|
||||
```bash
|
||||
export HEADROOM_LEARN_CLI=agy
|
||||
export HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1
|
||||
headroom learn
|
||||
```
|
||||
|
||||
Without `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1`, selecting `agy` (via `HEADROOM_LEARN_CLI` or `--model agy-cli`) fails before any analysis runs.
|
||||
|
||||
## Real-World Results
|
||||
|
||||
|
||||
Reference in New Issue
Block a user