fix(copilot): warn when a VS Code profile cannot see the proxy settings (#3919)

## Description

`headroom wrap vscode` writes Copilot's two endpoint overrides into the
Default profile's `settings.json` and reports success. A VS Code window
that uses a profile with its own settings reads only
`User/profiles/<id>/settings.json`, and Copilot takes these overrides
from user settings alone. In that window Copilot keeps talking to GitHub
directly: Copilot works, Headroom records nothing, and setup said it
succeeded. That is exactly what #3716 describes. The reporter hasn't
confirmed yet that they use a profile (I asked on the issue), so this
says "Refs" rather than "Closes".

After writing a VS Code user directory's `settings.json`, `wrap vscode`
now reads the profiles VS Code records in that directory's
`globalStorage/storage.json`. That file is either the Default one or one
named with `--settings-file`. For each profile that keeps its own
settings and doesn't define both overrides as live settings, it prints a
warning with the exact `--settings-file` command that routes it.

Nothing is written to profile files automatically. The docs already
scope profile-specific settings to `--settings-file`, and the block
persists and fails closed after the proxy stops. Routing every profile
silently would therefore change Copilot in profiles the user never meant
to proxy.

Refs #3716

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/providers/copilot/vscode.py`:
`unrouted_vscode_profiles(user_dir)` returns `(name, settings.json)` for
each stored profile that has its own settings
(`useDefaultFlags.settings` unset) and doesn't define both endpoint
overrides as live settings.
- It parses the profile's JSONC with `_parse_settings`, split out of
`_validate_settings` so both share one parser. As a result, a marker or
keys that exist only in comments don't count.
- It accepts only a local folder name as a location (VS Code uses
`hash(uuid).toString(16)`, which can be negative). That skips remote
URIs and rules out path traversal.
  - Missing or unexpected VS Code state produces no warning.
- `headroom/cli/wrap.py`: after writing the settings file, `wrap vscode`
prints one `Warning:` line per such profile of the user directory that
holds that file.
- A profile's own `settings.json` has no profile state beside it, so
running the printed remedy produces no warning.
- A `--settings-file` in another user directory (Insiders, VSCodium,
portable) gets that directory's profiles checked.
  - There's no warning with `--no-configure`.
- `headroom/providers/copilot/__init__.py`: exports the helper alongside
the other VS Code helpers.
- `docs/content/docs/vscode-copilot.mdx`: adds a troubleshooting entry
for "no requests while Copilot keeps working".
- `tests/test_cli/test_wrap_vscode.py`:
`test_wrap_vscode_names_profiles_that_ignore_default_settings` walks the
whole flow. It warns by default and when the Default file is named
explicitly, stays silent for a shared-settings profile, and goes silent
once the printed remedy has run.
`test_wrap_vscode_checks_the_profiles_of_the_user_dir_it_configures`
covers another user directory.
- `tests/test_provider_copilot_vscode_config.py` covers:
  - own versus shared settings
  - a negative-hash folder
  - traversal and remote locations
- profiles that override both endpoints, whether managed or hand-written
- a marker-only file, commented-out keys, a single key, and unparseable
settings
  - a missing or malformed `storage.json`

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# The new tests against the code before each commit.
# main (c46e74d0), before the first commit:
E   assert "VS Code profile 'Work'" in "  VS Code Copilot proxy settings added: ...\Code\User\settings.json\n  Keep using Copilot's normal model picker; ..."
# c887cbab, before the review follow-up. --settings-file naming the Default file printed no warning:
E   assert "VS Code profile 'Work'" in "  VS Code Copilot proxy settings updated: ...\Code\User\settings.json\n  Keep using Copilot's normal model picker; ..."
# ...and a profile whose settings only mention the marker in a comment counted as routed:
old code, marker-only profile -> []  (expected Work to be listed)
old code, commented-out keys profile -> []  (expected Work to be listed)

# With both commits:
$ pytest tests/test_cli/test_wrap_vscode.py tests/test_provider_copilot_vscode_config.py
27 passed in 1.19s

# Each mutation of the fix fails at least one test:
M1 ignore useDefaultFlags.settings: KILLED (2 failed, 25 passed)
M2 marker text counts as configured: KILLED (7 failed, 20 passed)
M3 one live key is enough: KILLED (1 failed, 26 passed)
M4 reject leading '-' in location: KILLED (2 failed, 25 passed)
M5 accept any location string: KILLED (1 failed, 26 passed)
M6 drop the CLI warning: KILLED (2 failed, 25 passed)
M7 scan only without --settings-file: KILLED (2 failed, 25 passed)
M8 always scan the Default user dir: KILLED (1 failed, 26 passed)

$ ruff check <changed files> && ruff format --check <changed files>   # ruff 0.16.8
All checks passed!
5 files left unchanged
$ mypy headroom/providers/copilot/vscode.py headroom/providers/copilot/__init__.py headroom/cli/wrap.py --ignore-missing-imports --follow-imports=silent   # mypy 1.19.1
Success: no issues found in 3 source files
# On the first commit, `mypy headroom --ignore-missing-imports` reported only
# headroom\fileperms.py:71 (Windows-only os.fchmod), a file this PR does not touch.

# Every test in the subsystems this touches, this branch vs its base (d5318ac2):
$ pytest tests/test_cli tests/test_copilot*.py tests/test_provider*.py -n 8
branch: 7 failed, 1621 passed, 8 skipped (14 more tests than base: the new ones)
base:  25 failed, 1589 passed, 8 skipped
# All 7 branch failures also fail on base: symlink, signal, service-supervisor and
# opencode-env tests that fail on this Windows machine. The other 18 base failures
# are test_wrap_persistent.py tests that vary from one xdist run to the next; that
# file passes serially (62 passed on both the branch and main).
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.11.9, VS Code 1.140.0 (official zip,
isolated `--user-data-dir`), this branch
- Exact command / steps: wrote Headroom's block into an isolated VS Code
user dir. Then opened VS Code with a small probe extension that records
`getConfiguration("github.copilot").inspect(key).globalValue` for both
keys, once in the Default profile and once with `--profile=Repro` (VS
Code created the profile and recorded it in `storage.json`). Next I ran
the `wrap vscode` command against that user dir (`APPDATA` pointed at
it). Only the Copilot subscription lookup and the proxy watcher were
stubbed, because there's no Copilot account on this machine. I ran the
command the warning printed, re-probed the profile window, and ran plain
`wrap vscode` again. For the review follow-up I repeated the CLI runs
against the same VS Code-written state with the cases from the two
findings.
- Observed result: the profile window doesn't see Headroom's settings
until the warning's command is run.
- Default profile window:
`"overrideCapiUrl":{"globalValue":"http://127.0.0.1:8788/p/a11y"}`, and
the same for `overrideProxyUrl`
- `--profile=Repro` window: `{"globalValue":null}` for both, so Copilot
falls back to the endpoints from its token, i.e. GitHub directly
- `wrap vscode` printed: `Warning: VS Code profile 'Repro' keeps its own
settings, so Copilot there still bypasses Headroom. Route it with:
headroom wrap vscode --settings-file
"V:\ud1\Code\User\profiles\6c87cdb4\settings.json"`
- after running that command, the `Repro` window sees both overrides
(`"globalValue":"http://127.0.0.1:8788/p/headroom"`), and plain `wrap
vscode` no longer warns
- review follow-up: with `Repro`'s settings holding only `// ---
Headroom Copilot proxy ---`, plain `wrap vscode` warns about `Repro`.
`--settings-file` naming the Default file warns too.
- review follow-up: the printed remedy refuses that half-written file
with the existing `Error: Incomplete Headroom marker block in
...\profiles\6c87cdb4\settings.json; refusing to edit.` Once the stray
marker is removed, the remedy succeeds and plain `wrap vscode` prints no
warning.
- Not tested: real Copilot traffic, macOS and Linux, the reporter's own
setup, and a complete local run of the full suite.
- real Copilot traffic, because there's no Copilot subscription on this
machine. I separately checked that the bundled Copilot Chat 0.66.0,
0.67.0 and 0.68.0 all read both keys as user-scope-only settings, and
that a proxy built from `main` logs both Copilot paths at INFO.
- macOS and Linux. VS Code uses the same `User/profiles` layout there,
and the unit tests cover the logic.
  - the reporter's own setup.
- the full test suite. A local run stalled at 97% and was killed after
40 minutes on this Windows machine, so I compared the touched subsystems
against the base instead; CI runs the full suite.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: `wrap vscode` prints one extra
warning line for each VS Code profile that keeps its own settings and
doesn't define both overrides. No file it writes changes.
- Kill switch / disable path: none needed. The warning is informational,
and pointing `--settings-file` at a file outside a VS Code user
directory skips the scan.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR; no persisted state or format changes

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- The second commit addresses both Devin Review findings: a commented
marker hid an unrouted profile, and an explicit Default path skipped the
scan.
- For the second finding I scan the user directory of the file being
configured rather than comparing against `vscode_settings_path()`. That
covers the explicit Default path, and it also checks profiles for
Insiders, VSCodium or portable user directories. It also avoids
resolving `APPDATA` when `--settings-file` is given.
- Question for maintainers: would you rather `wrap vscode` write the
block into every profile with its own settings, and `unwrap vscode`
remove it from all of them? It would be a small change on top of this
helper. I left it out because it changes which windows fail closed while
the proxy is down.
- `unwrap vscode` still cleans only the file it's given. A profile
configured with `--settings-file` needs the same flag to undo it, as the
docs already say.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Ayush Kumar Jha
2026-10-05 10:03:08 -05:00
committed by GitHub
co-authored by JD Davis
parent 613ae92150
commit 7b68fc263a
6 changed files with 233 additions and 3 deletions
+3
View File
@@ -159,6 +159,9 @@ both `copilot-auth login` and `wrap vscode`.
does not add or rename models.
- If connection is refused, keep the wrapper running and check loopback/remote
port reachability.
- If Headroom records no requests while Copilot keeps working, the window is
probably using a VS Code profile with its own settings. `wrap vscode` warns
about each such profile and prints the `--settings-file` path to configure it.
- Use `--port 8788` when the default port is occupied; settings update safely.
- If Headroom refuses settings, repair the reported JSONC/marker conflict or use
`--no-configure` and apply the printed settings manually.
+7
View File
@@ -116,6 +116,7 @@ from headroom.providers.copilot import (
from headroom.providers.copilot import (
configure_vscode_proxy_settings,
remove_vscode_proxy_settings,
unrouted_vscode_profiles,
vscode_proxy_url,
vscode_settings_path,
)
@@ -6688,6 +6689,12 @@ def vscode_copilot(
vscode_proxy_url(actual_port, _project_name_from_cwd()),
)
click.echo(f" VS Code Copilot proxy settings {action}: {target_settings}")
for name, profile_settings in unrouted_vscode_profiles(target_settings.parent):
click.echo(
f" Warning: VS Code profile '{name}' keeps its own settings, so Copilot "
"there still bypasses Headroom. Route it with: headroom wrap vscode "
f'--settings-file "{profile_settings}"'
)
click.echo(
" Keep using Copilot's normal model picker; the selected model is preserved."
)
+2
View File
@@ -3,6 +3,7 @@
from .vscode import (
configure_vscode_proxy_settings,
remove_vscode_proxy_settings,
unrouted_vscode_profiles,
vscode_proxy_url,
vscode_settings_path,
vscode_user_dir,
@@ -37,6 +38,7 @@ __all__ = [
"validate_configuration",
"configure_vscode_proxy_settings",
"remove_vscode_proxy_settings",
"unrouted_vscode_profiles",
"vscode_proxy_url",
"vscode_settings_path",
"vscode_user_dir",
+46 -3
View File
@@ -108,13 +108,16 @@ def _strip_jsonc_comments(value: str) -> str:
return "".join(result)
def _validate_settings(raw: str, path: Path) -> None:
def _parse_settings(raw: str) -> object:
candidate = _strip_jsonc_comments(raw)
if candidate.startswith("\ufeff"):
candidate = candidate[1:]
candidate = re.sub(r",\s*([}\]])", r"\1", candidate)
return json.loads(re.sub(r",\s*([}\]])", r"\1", candidate))
def _validate_settings(raw: str, path: Path) -> None:
try:
parsed = json.loads(candidate)
parsed = _parse_settings(raw)
except json.JSONDecodeError as exc:
raise click.ClickException(
f"Could not safely parse {path}: {exc}. Headroom did not overwrite it."
@@ -198,3 +201,43 @@ def configure_vscode_proxy_settings(path: Path, proxy_url: str) -> str:
_validate_settings(updated, path)
fsutil.write_text(path, updated)
return "updated" if had_managed_block else "added"
def unrouted_vscode_profiles(user_dir: Path) -> list[tuple[str, Path]]:
"""Return ``(name, settings.json)`` for profiles that ignore ``user_dir/settings.json``.
A window in a profile with its own settings reads only that profile's
settings.json, and Copilot takes its endpoint overrides from user settings
alone, so overrides in the user directory's settings.json never reach it.
A directory that is not a VS Code user directory has no profiles.
"""
try:
state = json.loads(
(user_dir / "globalStorage" / "storage.json").read_text(encoding="utf-8")
)
except (OSError, ValueError):
return []
profiles = state.get("userDataProfiles") if isinstance(state, dict) else None
unrouted: list[tuple[str, Path]] = []
for profile in profiles if isinstance(profiles, list) else []:
if not isinstance(profile, dict):
continue
flags = profile.get("useDefaultFlags")
location = profile.get("location")
# VS Code names local profile folders hash(uuid).toString(16); any other
# location (a remote URI) is not a folder under profiles/.
if (isinstance(flags, dict) and flags.get("settings")) or not (
isinstance(location, str) and re.fullmatch(r"[\w-]+", location)
):
continue
path = user_dir / "profiles" / location / "settings.json"
try:
settings = _parse_settings(_read_settings(path))
except (OSError, ValueError, click.ClickException):
settings = None
# Only live settings count: the managed-block marker is itself a comment.
if isinstance(settings, dict) and _PROXY_KEY in settings and _CAPI_KEY in settings:
continue
name = profile.get("name")
unrouted.append((name if isinstance(name, str) else location, path))
return unrouted
+74
View File
@@ -2,9 +2,11 @@
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from headroom.cli.main import main
@@ -46,6 +48,78 @@ def test_wrap_vscode_configures_actual_port_and_seeds_subscription(tmp_path: Pat
assert captured["copilot_api_token"] == "copilot-token"
def _run_wrap_vscode(*args: str) -> str:
def fake_watcher(**kwargs): # noqa: ANN003, ANN202
kwargs["print_setup_lines"](8787)
with (
patch(
"headroom.cli.wrap._require_copilot_subscription_resolution", return_value=_resolution()
),
patch("headroom.cli.wrap._run_proxy_only_watcher", side_effect=fake_watcher),
):
result = CliRunner().invoke(main, ["wrap", "vscode", *args])
assert result.exit_code == 0, result.output
return result.output
def _write_profiles(user_dir: Path, *profiles: dict[str, object]) -> None:
storage = user_dir / "globalStorage" / "storage.json"
storage.parent.mkdir(parents=True)
storage.write_text(json.dumps({"userDataProfiles": list(profiles)}), encoding="utf-8")
def _default_user_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
for var in ("APPDATA", "HOME", "USERPROFILE", "XDG_CONFIG_HOME"):
monkeypatch.setenv(var, str(tmp_path))
from headroom.providers.copilot.vscode import vscode_user_dir
return vscode_user_dir()
def test_wrap_vscode_names_profiles_that_ignore_default_settings(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
user_dir = _default_user_dir(tmp_path, monkeypatch)
_write_profiles(
user_dir,
{"location": "6c87cdb4", "name": "Work"},
{"location": "1a2b", "name": "Shared", "useDefaultFlags": {"settings": True}},
)
work_settings = user_dir / "profiles" / "6c87cdb4" / "settings.json"
implicit = _run_wrap_vscode()
assert "overrideCapiUrl" in (user_dir / "settings.json").read_text(encoding="utf-8")
# A window in the "Work" profile reads only its own settings.json, so the
# block above never reaches it; say so instead of reporting plain success.
assert "VS Code profile 'Work'" in implicit
assert str(work_settings) in implicit
# A profile that shares the Default profile's settings already sees the block.
assert "Shared" not in implicit
# Naming the Default file explicitly configures the same file, so it warns too.
explicit = _run_wrap_vscode("--settings-file", str(user_dir / "settings.json"))
assert "VS Code profile 'Work'" in explicit
# Following the advice routes the profile, and nothing is left to warn about.
assert "Warning" not in _run_wrap_vscode("--settings-file", str(work_settings))
assert "Warning" not in _run_wrap_vscode()
def test_wrap_vscode_checks_the_profiles_of_the_user_dir_it_configures(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_write_profiles(
_default_user_dir(tmp_path, monkeypatch), {"location": "6c87cdb4", "name": "Work"}
)
# e.g. VS Code Insiders, VSCodium, or a portable install with its own profiles.
other_user_dir = tmp_path / "Code - Insiders" / "User"
_write_profiles(other_user_dir, {"location": "-2b3c4d", "name": "Beta"})
output = _run_wrap_vscode("--settings-file", str(other_user_dir / "settings.json"))
assert "VS Code profile 'Beta'" in output
assert "Work" not in output
def test_wrap_vscode_no_configure_prints_transparent_settings(tmp_path: Path) -> None:
path = tmp_path / "settings.json"
@@ -1,5 +1,6 @@
from __future__ import annotations
import json
from pathlib import Path
import click
@@ -8,6 +9,7 @@ import pytest
from headroom.providers.copilot.vscode import (
configure_vscode_proxy_settings,
remove_vscode_proxy_settings,
unrouted_vscode_profiles,
vscode_proxy_url,
vscode_settings_path,
)
@@ -107,3 +109,102 @@ def test_configure_refuses_duplicate_managed_markers(tmp_path: Path) -> None:
with pytest.raises(click.ClickException, match="marker block"):
configure_vscode_proxy_settings(path, "http://127.0.0.1:8787")
assert path.read_text(encoding="utf-8") == original
def _write_profiles(tmp_path: Path, state: object) -> Path:
user_dir = tmp_path / "User"
storage = user_dir / "globalStorage" / "storage.json"
storage.parent.mkdir(parents=True)
storage.write_text(state if isinstance(state, str) else json.dumps(state), encoding="utf-8")
return user_dir
def test_unrouted_profiles_lists_only_profiles_with_their_own_settings(tmp_path: Path) -> None:
user_dir = _write_profiles(
tmp_path,
{
"userDataProfiles": [
{"location": "6c87cdb4", "name": "Work"},
{"location": "-1f2e3d", "name": "Negative hash"},
{"location": "aa11", "name": "Shared", "useDefaultFlags": {"settings": True}},
{"location": "../escape", "name": "Traversal"},
{"location": {"scheme": "vscode-remote", "path": "/p"}, "name": "Remote"},
"not-a-profile",
]
},
)
assert unrouted_vscode_profiles(user_dir) == [
("Work", user_dir / "profiles" / "6c87cdb4" / "settings.json"),
("Negative hash", user_dir / "profiles" / "-1f2e3d" / "settings.json"),
]
@pytest.mark.parametrize("managed", [True, False])
def test_unrouted_profiles_skips_a_profile_that_overrides_both_endpoints(
tmp_path: Path, managed: bool
) -> None:
user_dir = _write_profiles(
tmp_path, {"userDataProfiles": [{"location": "6c87cdb4", "name": "Work"}]}
)
profile_settings = user_dir / "profiles" / "6c87cdb4" / "settings.json"
profile_settings.parent.mkdir(parents=True)
if managed:
configure_vscode_proxy_settings(profile_settings, "http://127.0.0.1:8787")
else:
profile_settings.write_text(
"""{
"github.copilot.advanced.debug.overrideProxyUrl": "http://127.0.0.1:8787",
"github.copilot.advanced.debug.overrideCapiUrl": "http://127.0.0.1:8787",
}
""",
encoding="utf-8",
)
assert unrouted_vscode_profiles(user_dir) == []
@pytest.mark.parametrize(
"settings",
[
# The marker is itself a comment, so on its own it proves nothing.
"""{
// --- Headroom Copilot proxy ---
}
""",
"""{
// --- Headroom Copilot proxy ---
// "github.copilot.advanced.debug.overrideProxyUrl": "http://127.0.0.1:8787",
// "github.copilot.advanced.debug.overrideCapiUrl": "http://127.0.0.1:8787"
// --- end Headroom Copilot proxy ---
}
""",
"""{
"note": "// --- Headroom Copilot proxy ---",
"github.copilot.advanced.debug.overrideCapiUrl": "http://127.0.0.1:8787"
}
""",
'{ "github.copilot.advanced.debug.overrideCapiUrl": ',
],
ids=["marker-only", "commented-out-keys", "one-key", "unparseable"],
)
def test_unrouted_profiles_counts_only_live_override_settings(
tmp_path: Path, settings: str
) -> None:
user_dir = _write_profiles(
tmp_path, {"userDataProfiles": [{"location": "6c87cdb4", "name": "Work"}]}
)
profile_settings = user_dir / "profiles" / "6c87cdb4" / "settings.json"
profile_settings.parent.mkdir(parents=True)
profile_settings.write_text(settings, encoding="utf-8")
assert unrouted_vscode_profiles(user_dir) == [("Work", profile_settings)]
@pytest.mark.parametrize("state", ["{not json", [], {"userDataProfiles": {"a": 1}}, {}])
def test_unrouted_profiles_tolerates_unexpected_vscode_state(tmp_path: Path, state: object) -> None:
assert unrouted_vscode_profiles(_write_profiles(tmp_path, state)) == []
def test_unrouted_profiles_without_vscode_state(tmp_path: Path) -> None:
assert unrouted_vscode_profiles(tmp_path) == []