mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description
VS Code can run a Copilot chat on its "Copilot harness" (Agent Host, the
bundled Copilot SDK runtime) instead of the Local harness. That runtime
ignores `github.copilot.advanced.debug.overrideCapiUrl` /
`overrideProxyUrl`, so the chat goes straight to GitHub. Copilot's small
background calls (title, intent) still reach the proxy, so the proxy
looks connected while the main conversation never arrives. The harness
default (`chat.defaultToCopilotHarness`,
`chat.editor.preferCopilotHarness`) is tagged `experiment: startup` in
VS Code, so it can be flipped per user by rollout.
`headroom wrap vscode` now pins new chats to the Local harness.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/providers/copilot/vscode.py`: the managed block also writes
`"chat.defaultToCopilotHarness": false` and
`"chat.editor.preferCopilotHarness": false`. Because they sit inside the
marker block, `headroom unwrap vscode` removes them. A harness setting
the user already set outside the block is left alone (no duplicate key).
New helper `copilot_harness_settings_enabled(path)` reports a user value
of `true`.
- `headroom/cli/wrap.py`: one line saying new chats stay on the Local
harness and existing chats keep their engine; a warning instead when the
user's own setting is `true`. `--no-configure` prints the two extra
settings.
- `docs/content/docs/vscode-copilot.mdx`: documents the harness settings
and the troubleshooting case.
- Blocks written by older versions (including ones with
`overrideAuthType`) are replaced by marker as before.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
tests/test_provider_copilot_vscode_config.py + tests/test_cli/test_wrap_vscode.py: 32 passed.
Against origin/main source: 7 of those fail (block contents, old-block upgrade, user-set key [True]/[False], wrap output, --no-configure output).
ruff 0.17.4 check + format --check on the 5 changed Python files: clean.
mypy on the changed source files: 8 errors, all in headroom/cache/prefix_tracker.py and headroom/ccr/mcp_server.py, identical on origin/main; none in changed files.
```
## Real Behavior Proof
- Environment: macOS 27.2.1, Python 3.12.13, VS Code 1.145.2 with GitHub
Copilot signed in, core run from this branch, real GitHub Copilot
upstream. Real user `settings.json` (backed up and restored afterwards).
- Exact command / steps:
1. `headroom wrap vscode -p 8795` from this branch.
2. `code chat -n -m agent "Reply with the single word OK and nothing
else."` (opens a new VS Code window and sends one Agent-mode chat).
3. Read the proxy's `/stats`, VS Code's `agenthost.log`, then `headroom
unwrap vscode`.
- Observed result:
```text
VS Code Copilot proxy settings updated: ~/Library/Application
Support/Code/User/settings.json
Keep using Copilot's normal model picker; the selected model is
preserved.
New chats stay on VS Code's Local harness; a chat keeps the engine it
started with, so start a new chat.
settings.json after wrap:
{
// --- Headroom Copilot proxy ---
"github.copilot.advanced.debug.overrideProxyUrl":
"http://127.0.0.1:8795/p/win-local",
"github.copilot.advanced.debug.overrideCapiUrl":
"http://127.0.0.1:8795/p/win-local",
"chat.defaultToCopilotHarness": false,
"chat.editor.preferCopilotHarness": false
// --- end Headroom Copilot proxy ---
}
proxy requests before chat: 0 {}
proxy requests after one Agent-mode chat in a new VS Code window: 4
{'gpt-4o-mini-2024-07-18': 3, 'claude-opus-5.5': 1}
agent host turns during the chat: 0
Removed Headroom Copilot proxy settings from ~/Library/Application
Support/Code/User/settings.json
settings.json after unwrap:
{
}
```
Separately, on the same machine with the settings edited by hand (same
VS Code, same proxy, new window each time):
`"chat.defaultToCopilotHarness": true` -> the Agent-mode chat ran in the
Agent Host (provider `copilotcli`) and 0 requests reached the proxy;
`"chat.defaultToCopilotHarness": false` +
`"chat.editor.preferCopilotHarness": false` -> Local harness, 4 requests
reached the proxy including the main `/v1/messages` call.
`"chat.temporaryDefaultToLocalHarness": true` as a user setting had no
effect, so it is not written.
- Not tested: a VS Code where Microsoft's rollout has flipped the
default (the explicit user value is expected to win, but that could not
be reproduced); an organization that enforces managed sandbox settings,
where VS Code can require the Copilot harness regardless; VS Code
profiles, workspace-level settings, Insiders/VSCodium; Windows and
Linux; the persistent `headroom start --persistent --vscode` path in the
Enterprise client.
## Runtime Rollout Safety
- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: `headroom wrap vscode` writes two
more VS Code user settings inside its managed block. No proxy behaviour
changes.
- Kill switch / disable path: `headroom unwrap vscode` removes them; a
user who sets either key themselves keeps their value.
- Unsafe override required: no.
- Qualification impact: none expected.
- Rollback path: revert this commit; `unwrap` removes blocks of either
format.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
- This keeps users off the Copilot harness; it does not route that
harness through Headroom. Supporting it is separate work.
- Only the configured settings file is checked for a user value. A
`true` in workspace settings, another profile, or policy is not
detected.
- After `unwrap` the keys return to VS Code's default, which a rollout
may move to the Copilot harness.
- Existing chats keep the harness they started with; the wrap output
says to start a new chat.
- `e2e/copilot_live.py` and `e2e/wrap/run.py` were not run.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4899"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
181 lines
7.2 KiB
Python
181 lines
7.2 KiB
Python
"""CLI coverage for transparent VS Code Copilot setup and undo."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from pathlib import Path
|
|
from unittest.mock import patch
|
|
|
|
import pytest
|
|
from click.testing import CliRunner
|
|
|
|
from headroom.cli.main import main
|
|
from headroom.copilot_auth import CopilotSubscriptionTokenResolution
|
|
|
|
|
|
def _resolution() -> CopilotSubscriptionTokenResolution:
|
|
return CopilotSubscriptionTokenResolution(
|
|
token="copilot-token",
|
|
source="test",
|
|
confidence="test",
|
|
api_url="https://api.githubcopilot.com",
|
|
token_fingerprint="sha256:test",
|
|
)
|
|
|
|
|
|
def test_wrap_vscode_configures_actual_port_and_seeds_subscription(tmp_path: Path) -> None:
|
|
path = tmp_path / "settings.json"
|
|
captured = {}
|
|
|
|
def fake_watcher(**kwargs): # noqa: ANN003, ANN202
|
|
captured.update(kwargs)
|
|
kwargs["print_setup_lines"](9999)
|
|
|
|
with (
|
|
patch(
|
|
"headroom.cli.wrap._require_copilot_subscription_resolution", return_value=_resolution()
|
|
),
|
|
patch("headroom.cli.wrap._run_proxy_only_watcher", side_effect=fake_watcher),
|
|
):
|
|
result = CliRunner().invoke(main, ["wrap", "vscode", "--settings-file", str(path)])
|
|
|
|
assert result.exit_code == 0, result.output
|
|
settings = path.read_text(encoding="utf-8")
|
|
assert "http://127.0.0.1:9999/" in settings
|
|
assert "model" not in settings.lower()
|
|
assert "normal model picker" in result.output
|
|
# Chats on the Copilot harness never reach the proxy, and a chat keeps the
|
|
# engine it started with, so the user has to be told to start a new one.
|
|
assert "Local harness" in result.output
|
|
assert "start a new chat" in result.output
|
|
assert "Warning" not in result.output
|
|
assert captured["openai_api_url"] == "https://api.githubcopilot.com"
|
|
assert captured["copilot_api_token"] == "copilot-token"
|
|
|
|
|
|
def _run_wrap_vscode(*args: str) -> str:
|
|
def fake_watcher(**kwargs): # noqa: ANN003, ANN202
|
|
kwargs["print_setup_lines"](8787)
|
|
|
|
with (
|
|
patch(
|
|
"headroom.cli.wrap._require_copilot_subscription_resolution", return_value=_resolution()
|
|
),
|
|
patch("headroom.cli.wrap._run_proxy_only_watcher", side_effect=fake_watcher),
|
|
):
|
|
result = CliRunner().invoke(main, ["wrap", "vscode", *args])
|
|
assert result.exit_code == 0, result.output
|
|
return result.output
|
|
|
|
|
|
def _write_profiles(user_dir: Path, *profiles: dict[str, object]) -> None:
|
|
storage = user_dir / "globalStorage" / "storage.json"
|
|
storage.parent.mkdir(parents=True)
|
|
storage.write_text(json.dumps({"userDataProfiles": list(profiles)}), encoding="utf-8")
|
|
|
|
|
|
def _default_user_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
|
for var in ("APPDATA", "HOME", "USERPROFILE", "XDG_CONFIG_HOME"):
|
|
monkeypatch.setenv(var, str(tmp_path))
|
|
from headroom.providers.copilot.vscode import vscode_user_dir
|
|
|
|
return vscode_user_dir()
|
|
|
|
|
|
def test_wrap_vscode_names_profiles_that_ignore_default_settings(
|
|
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
user_dir = _default_user_dir(tmp_path, monkeypatch)
|
|
_write_profiles(
|
|
user_dir,
|
|
{"location": "6c87cdb4", "name": "Work"},
|
|
{"location": "1a2b", "name": "Shared", "useDefaultFlags": {"settings": True}},
|
|
)
|
|
work_settings = user_dir / "profiles" / "6c87cdb4" / "settings.json"
|
|
|
|
implicit = _run_wrap_vscode()
|
|
assert "overrideCapiUrl" in (user_dir / "settings.json").read_text(encoding="utf-8")
|
|
# A window in the "Work" profile reads only its own settings.json, so the
|
|
# block above never reaches it; say so instead of reporting plain success.
|
|
assert "VS Code profile 'Work'" in implicit
|
|
assert str(work_settings) in implicit
|
|
# A profile that shares the Default profile's settings already sees the block.
|
|
assert "Shared" not in implicit
|
|
# Naming the Default file explicitly configures the same file, so it warns too.
|
|
explicit = _run_wrap_vscode("--settings-file", str(user_dir / "settings.json"))
|
|
assert "VS Code profile 'Work'" in explicit
|
|
# Following the advice routes the profile, and nothing is left to warn about.
|
|
assert "Warning" not in _run_wrap_vscode("--settings-file", str(work_settings))
|
|
assert "Warning" not in _run_wrap_vscode()
|
|
|
|
|
|
def test_wrap_vscode_checks_the_profiles_of_the_user_dir_it_configures(
|
|
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
_write_profiles(
|
|
_default_user_dir(tmp_path, monkeypatch), {"location": "6c87cdb4", "name": "Work"}
|
|
)
|
|
# e.g. VS Code Insiders, VSCodium, or a portable install with its own profiles.
|
|
other_user_dir = tmp_path / "Code - Insiders" / "User"
|
|
_write_profiles(other_user_dir, {"location": "-2b3c4d", "name": "Beta"})
|
|
|
|
output = _run_wrap_vscode("--settings-file", str(other_user_dir / "settings.json"))
|
|
|
|
assert "VS Code profile 'Beta'" in output
|
|
assert "Work" not in output
|
|
|
|
|
|
def test_wrap_vscode_warns_when_the_user_opted_into_the_copilot_harness(tmp_path: Path) -> None:
|
|
path = tmp_path / "settings.json"
|
|
path.write_text('{\n "chat.editor.preferCopilotHarness": true\n}\n', encoding="utf-8")
|
|
|
|
output = _run_wrap_vscode("--settings-file", str(path))
|
|
|
|
# The wrap still succeeds and keeps the user's choice, but says what it costs.
|
|
assert '"chat.editor.preferCopilotHarness": true' in output
|
|
assert "bypass Headroom" in output
|
|
assert "chat.defaultToCopilotHarness" not in output
|
|
assert "stay on VS Code's Local harness" not in output
|
|
assert path.read_text(encoding="utf-8").count("preferCopilotHarness") == 1
|
|
|
|
|
|
def test_wrap_vscode_no_configure_prints_transparent_settings(tmp_path: Path) -> None:
|
|
path = tmp_path / "settings.json"
|
|
|
|
def fake_watcher(**kwargs): # noqa: ANN003, ANN202
|
|
kwargs["print_setup_lines"](8787)
|
|
|
|
with (
|
|
patch(
|
|
"headroom.cli.wrap._require_copilot_subscription_resolution", return_value=_resolution()
|
|
),
|
|
patch("headroom.cli.wrap._run_proxy_only_watcher", side_effect=fake_watcher),
|
|
):
|
|
result = CliRunner().invoke(
|
|
main,
|
|
["wrap", "vscode", "--no-configure", "--settings-file", str(path)],
|
|
)
|
|
|
|
assert result.exit_code == 0, result.output
|
|
assert not path.exists()
|
|
assert "overrideProxyUrl" in result.output
|
|
assert "overrideCapiUrl" in result.output
|
|
assert '"chat.defaultToCopilotHarness": false' in result.output
|
|
assert '"chat.editor.preferCopilotHarness": false' in result.output
|
|
# No `overrideAuthType`: the setting does not exist in the modern Copilot
|
|
# Chat extension, so printing it told users to add a key VS Code flags as
|
|
# unknown and which does nothing (#3076).
|
|
assert "overrideAuthType" not in result.output
|
|
|
|
|
|
def test_unwrap_vscode_removes_only_managed_settings(tmp_path: Path) -> None:
|
|
path = tmp_path / "settings.json"
|
|
original = '{\n "editor.fontSize": 14\n}\n'
|
|
path.write_text(original, encoding="utf-8")
|
|
from headroom.providers.copilot.vscode import configure_vscode_proxy_settings
|
|
|
|
configure_vscode_proxy_settings(path, "http://127.0.0.1:8787")
|
|
result = CliRunner().invoke(main, ["unwrap", "vscode", "--settings-file", str(path)])
|
|
assert result.exit_code == 0, result.output
|
|
assert path.read_text(encoding="utf-8") == original
|