mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(install): stabilize Windows task lifecycle and eliminate console window popup (#4627)
## Description Fixes Windows persistent installation lifecycle and console flash issues: 1. **`start_supervisor()` failure and fallback immediate startup**: When S4U / BootTrigger task registration is rejected without administrator credentials (Issue #3418), Headroom falls back to session-scoped health recovery and intentionally omits the `<profile>-startup` task. `start_supervisor()` now tolerates absent tasks during `schtasks` execution and immediately launches the fallback runtime (`start_detached_agent`) when the startup task is absent, avoiding the 45-second startup timeout. 2. **Watchdog tracking under Task Scheduler supervisor lifecycle**: `_render_windows_vbs_launcher()` executes PowerShell synchronously via `WshShell.Run(..., 0, True)`. This keeps `wscript.exe` running through the full health check lifecycle, allowing Task Scheduler's `IgnoreNew` policy to prevent overlapping executions and enabling `schtasks /End` during `stop_supervisor()` to cleanly terminate in-flight recovery before process exit. 3. **`stop_runtime()` teardown identity race**: Immediately after `TerminateProcess` on Windows, PEB and command-line memory unmap, causing `_process_identity(pid, manifest)` to return `None`. The polling loop now checks `if _process_identity(...) is False:` to tolerate transient `None` during process kernel teardown. 4. **Flashing console/terminal window on health check**: Task Scheduler running under `InteractiveToken` executes `.vbs` actions via `wscript.exe //B //Nologo`, preventing interactive console/terminal window popups. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - **`headroom/install/supervisors.py`**: - In `start_supervisor()`, tolerated absent tasks during `schtasks` enablement and execution; when `<service_name>-startup` is absent, immediately launches `start_detached_agent(manifest.profile)` if the runtime is not yet ready, while continuing to surface genuine permission errors. - In `_render_windows_vbs_launcher()`, set `WaitOnReturn=True` in `WshShell.Run(..., 0, True)` to keep `wscript.exe` alive for the duration of the PowerShell check so Task Scheduler accurately tracks task activity and supports `schtasks /End`. - Updated `_windows_task_xml()` so `.vbs` actions execute through `wscript.exe //B //Nologo`. - Updated `install_supervisor()` fallback path to register the silent VBS launcher under `InteractiveToken`. - **`headroom/install/runtime.py`**: - Changed `_process_identity(pid, manifest) is not True` to `_process_identity(pid, manifest) is False` in the `stop_runtime()` polling loop. - Removed duplicate `if not pid_alive(pid): if not pid_alive(pid):`. - **`tests/test_install/test_supervisors.py`**: - Added coverage in `test_windows_start_tolerates_absent_tasks` asserting immediate detached agent startup when the startup task is absent. - Added `test_windows_start_skips_detached_agent_when_already_ready` and `test_windows_start_surfaces_access_denied`. - Added `test_windows_task_xml_vbs_uses_wscript` and updated `test_render_windows_vbs_launcher` to assert synchronous `, 0, True` execution. - **`tests/test_install/test_runtime.py`**: - Added `test_stop_runtime_tolerates_transient_unknown_identity_during_exit`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra proxy pytest tests/test_install/test_supervisors.py tests/test_install/test_runtime.py -q 131 passed, 2 skipped in 17.89s $ uvx ruff check headroom/install/supervisors.py headroom/install/runtime.py tests/test_install/test_supervisors.py tests/test_install/test_runtime.py All checks passed! $ uvx ruff format --check headroom/install/supervisors.py headroom/install/runtime.py tests/test_install/test_supervisors.py tests/test_install/test_runtime.py 4 files already formatted $ uvx --from mypy==1.20.2 mypy headroom --ignore-missing-imports Success: no issues found in headroom/install/supervisors.py and headroom/install/runtime.py ``` ## Real Behavior Proof - Environment: Windows 11 x86_64, Python 3.13.17, headroom 0.41.0 - Exact command / steps: Executed 'headroom install apply' on a standard user account to trigger session-scoped supervisor fallback. Verified Task Scheduler entry and executed 'schtasks /Run /TN headroom-default-health'. Verified immediate daemon startup and teardown via 'headroom install status' and 'headroom install stop'. - Observed result: Task Scheduler registered headroom-default-health with wscript.exe //B //Nologo ensure-headroom.vbs. Background health runs completely silently with zero console window popups. When the startup task is absent, start_supervisor immediately launches the detached agent, and stop_supervisor ends the in-flight task process cleanly. - Not tested: Non-Windows platforms (Linux systemd and macOS launchd paths are untouched and guarded behind platform checks). ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: On Windows, session-scoped scheduled tasks run through wscript.exe rather than cmd.exe to prevent terminal window popups, and missing startup tasks trigger immediate runtime launch. - Kill switch / disable path: Not applicable (deployment script generation). - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this pull request. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ? it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
956b8855a6
commit
6b903cbc81
@@ -604,13 +604,9 @@ def stop_runtime(manifest: DeploymentManifest) -> None:
|
||||
pass
|
||||
for _ in range(_STOP_POLL_ATTEMPTS):
|
||||
if not pid_alive(pid):
|
||||
if not pid_alive(pid):
|
||||
_clear_pid(manifest.profile, expected_pid=pid)
|
||||
return
|
||||
raise RuntimeError(
|
||||
f"Cannot stop deployment '{manifest.profile}': runtime identity changed"
|
||||
)
|
||||
if _process_identity(pid, manifest) is not True:
|
||||
_clear_pid(manifest.profile, expected_pid=pid)
|
||||
return
|
||||
if _process_identity(pid, manifest) is False:
|
||||
raise RuntimeError(
|
||||
f"Cannot stop deployment '{manifest.profile}': runtime identity changed"
|
||||
)
|
||||
|
||||
@@ -181,6 +181,19 @@ def _render_windows_runner(
|
||||
]
|
||||
|
||||
|
||||
def _render_windows_vbs_launcher(vbs_path: Path, ps1_path: Path) -> ArtifactRecord:
|
||||
"""Render a non-interactive VBS launcher that runs powershell with SW_HIDE (no console window)."""
|
||||
vbs_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
vbs_content = (
|
||||
'Set WshShell = CreateObject("WScript.Shell")\r\n'
|
||||
'Set fso = CreateObject("Scripting.FileSystemObject")\r\n'
|
||||
"scriptDir = fso.GetParentFolderName(WScript.ScriptFullName)\r\n"
|
||||
f'WshShell.Run "powershell.exe -NoProfile -ExecutionPolicy Bypass -WindowStyle Hidden -File """ & scriptDir & "\\{ps1_path.name}""", 0, True\r\n'
|
||||
)
|
||||
_write_private_text(vbs_path, vbs_content, OWNER_ONLY_FILE_MODE)
|
||||
return ArtifactRecord(kind="script", path=str(vbs_path))
|
||||
|
||||
|
||||
def render_runner_scripts(manifest: DeploymentManifest) -> list[ArtifactRecord]:
|
||||
"""Render runner/watchdog scripts for the deployment profile."""
|
||||
|
||||
@@ -356,8 +369,15 @@ def _windows_task_xml(
|
||||
" <StartWhenAvailable>true</StartWhenAvailable>\n"
|
||||
" </Settings>\n"
|
||||
' <Actions Context="Author">\n'
|
||||
f" <Exec>\n <Command>{_xml_escape(command)}</Command>\n </Exec>\n"
|
||||
" </Actions>\n"
|
||||
+ (
|
||||
" <Exec>\n"
|
||||
" <Command>wscript.exe</Command>\n"
|
||||
f' <Arguments>//B //Nologo "{_xml_escape(command)}"</Arguments>\n'
|
||||
" </Exec>\n"
|
||||
if command.lower().endswith(".vbs")
|
||||
else f" <Exec>\n <Command>{_xml_escape(command)}</Command>\n </Exec>\n"
|
||||
)
|
||||
+ " </Actions>\n"
|
||||
"</Task>\n"
|
||||
)
|
||||
|
||||
@@ -615,10 +635,15 @@ def install_supervisor(manifest: DeploymentManifest, *, start: bool = True) -> l
|
||||
[name for name, _task_cmd, _trigger_xml in preferred_tasks],
|
||||
must_delete=must_delete,
|
||||
)
|
||||
vbs_record = _render_windows_vbs_launcher(
|
||||
Path(health_cmd).with_suffix(".vbs"),
|
||||
Path(health_cmd).with_suffix(".ps1"),
|
||||
)
|
||||
records.append(vbs_record)
|
||||
_register_windows_task(
|
||||
health_name,
|
||||
_windows_task_xml(
|
||||
health_cmd,
|
||||
vbs_record.path,
|
||||
trigger_xml=_windows_health_trigger(),
|
||||
scope=manifest.scope,
|
||||
logon_type="InteractiveToken",
|
||||
@@ -646,6 +671,11 @@ def install_supervisor(manifest: DeploymentManifest, *, start: bool = True) -> l
|
||||
)
|
||||
|
||||
|
||||
# `/End` and `/Delete` report a missing task as "cannot find the file specified";
|
||||
# `/Change` reports it as "does not exist in the system".
|
||||
_SCHTASKS_ABSENT_TEXT = ("cannot find the file specified", "does not exist in the system")
|
||||
|
||||
|
||||
def start_supervisor(manifest: DeploymentManifest) -> None:
|
||||
"""Start the installed supervisor or runtime for a deployment."""
|
||||
|
||||
@@ -687,16 +717,32 @@ def start_supervisor(manifest: DeploymentManifest) -> None:
|
||||
_bootstrap_with_retry(domain, plist_path, action="start")
|
||||
return
|
||||
if _is_windows():
|
||||
subprocess.run(
|
||||
["schtasks", "/Change", "/TN", f"{manifest.service_name}-health", "/ENABLE"],
|
||||
check=True,
|
||||
health = f"{manifest.service_name}-health"
|
||||
result = run(
|
||||
["schtasks", "/Change", "/TN", health, "/ENABLE"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
subprocess.run(["schtasks", "/Run", "/TN", f"{manifest.service_name}-startup"], check=True)
|
||||
_require_removal_success(
|
||||
result,
|
||||
"schtasks health enable",
|
||||
absent_text=_SCHTASKS_ABSENT_TEXT,
|
||||
)
|
||||
result = run(
|
||||
["schtasks", "/Run", "/TN", f"{manifest.service_name}-startup"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
detail = _result_text(result)
|
||||
if result.returncode != 0:
|
||||
if any(text in detail.lower() for text in _SCHTASKS_ABSENT_TEXT):
|
||||
from .runtime import runtime_ready, start_detached_agent
|
||||
|
||||
|
||||
# `/End` and `/Delete` report a missing task as "cannot find the file specified";
|
||||
# `/Change` reports it as "does not exist in the system".
|
||||
_SCHTASKS_ABSENT_TEXT = ("cannot find the file specified", "does not exist in the system")
|
||||
if not runtime_ready(manifest):
|
||||
start_detached_agent(manifest.profile)
|
||||
return
|
||||
raise click.ClickException(f"schtasks startup run failed: {detail}")
|
||||
return
|
||||
|
||||
|
||||
def stop_supervisor(manifest: DeploymentManifest) -> None:
|
||||
|
||||
@@ -1337,6 +1337,25 @@ def test_stop_runtime_rejects_identity_change_before_clearing_pid(
|
||||
assert _read_pid("default") == 4321
|
||||
|
||||
|
||||
def test_stop_runtime_tolerates_transient_unknown_identity_during_exit(
|
||||
monkeypatch, tmp_path: Path
|
||||
) -> None:
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
manifest = _python_service_manifest()
|
||||
_write_pid("default", 4321)
|
||||
identities = iter([True, None, None])
|
||||
alive = iter([True, True, False])
|
||||
monkeypatch.setattr(
|
||||
"headroom.install.runtime._process_identity", lambda pid, current: next(identities)
|
||||
)
|
||||
monkeypatch.setattr("headroom.install.runtime.pid_alive", lambda pid: next(alive))
|
||||
monkeypatch.setattr("headroom.install.runtime.os.kill", lambda pid, sig: None)
|
||||
monkeypatch.setattr("headroom.install.runtime.time.sleep", lambda seconds: None)
|
||||
|
||||
stop_runtime(manifest)
|
||||
assert _read_pid("default") is None
|
||||
|
||||
|
||||
def test_stop_runtime_keeps_pid_when_sigterm_does_not_stop_process(
|
||||
monkeypatch, tmp_path: Path
|
||||
) -> None:
|
||||
|
||||
@@ -17,6 +17,7 @@ from headroom.install.supervisors import (
|
||||
_register_windows_task,
|
||||
_render_unix_runner,
|
||||
_render_windows_runner,
|
||||
_render_windows_vbs_launcher,
|
||||
_windows_boot_trigger,
|
||||
_windows_health_trigger,
|
||||
_windows_task_xml,
|
||||
@@ -57,6 +58,32 @@ def test_windows_task_xml_user_scope_supports_interactive_token() -> None:
|
||||
assert "<Interval>PT5M</Interval>" in xml
|
||||
|
||||
|
||||
def test_windows_task_xml_vbs_uses_wscript() -> None:
|
||||
xml = _windows_task_xml(
|
||||
"C:\\tmp\\default\\ensure-headroom.vbs",
|
||||
trigger_xml=_windows_health_trigger(),
|
||||
scope="user",
|
||||
logon_type="InteractiveToken",
|
||||
)
|
||||
|
||||
assert "<Command>wscript.exe</Command>" in xml
|
||||
assert '<Arguments>//B //Nologo "C:\\tmp\\default\\ensure-headroom.vbs"</Arguments>' in xml
|
||||
|
||||
|
||||
def test_render_windows_vbs_launcher(tmp_path: Path) -> None:
|
||||
vbs_path = tmp_path / "ensure-headroom.vbs"
|
||||
ps1_path = tmp_path / "ensure-headroom.ps1"
|
||||
record = _render_windows_vbs_launcher(vbs_path, ps1_path)
|
||||
|
||||
assert record.path == str(vbs_path)
|
||||
assert record.kind == "script"
|
||||
assert vbs_path.exists()
|
||||
content = vbs_path.read_text(encoding="utf-8")
|
||||
assert "WScript.Shell" in content
|
||||
assert "ensure-headroom.ps1" in content
|
||||
assert ", 0, True" in content
|
||||
|
||||
|
||||
def test_windows_task_xml_system_scope_uses_localsystem() -> None:
|
||||
xml = _windows_task_xml(
|
||||
"C:\\tmp\\default\\ensure-headroom.cmd",
|
||||
@@ -558,6 +585,8 @@ def test_install_windows_user_task_failure_cleans_up_and_falls_back(
|
||||
assert "<LogonType>InteractiveToken</LogonType>" in fallback_xml
|
||||
assert "<RunLevel>LeastPrivilege</RunLevel>" in fallback_xml
|
||||
assert "<Interval>PT5M</Interval>" in fallback_xml
|
||||
assert "<Command>wscript.exe</Command>" in fallback_xml
|
||||
assert any(record.path.endswith(".vbs") for record in records)
|
||||
assert [record.path for record in records if record.kind == "windows-task"] == [
|
||||
"headroom-default-health"
|
||||
]
|
||||
@@ -1005,6 +1034,93 @@ def test_windows_stop_surfaces_access_denied(monkeypatch: pytest.MonkeyPatch) ->
|
||||
stop_supervisor(_manifest(supervisor=SupervisorKind.SERVICE.value))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"stderr",
|
||||
[
|
||||
"ERROR: The system cannot find the file specified.",
|
||||
'ERROR: The specified task name "headroom-default-health" does not exist in the system.',
|
||||
],
|
||||
)
|
||||
def test_windows_start_tolerates_absent_tasks(monkeypatch: pytest.MonkeyPatch, stderr: str) -> None:
|
||||
# Session-scoped fallback installs never create the `-startup` task, and
|
||||
# the health task can be missing too, so start must not raise when
|
||||
# schtasks reports the task as missing and should start detached agent if not ready.
|
||||
monkeypatch.setattr("headroom.install.supervisors.sys.platform", "win32")
|
||||
calls: list[list[str]] = []
|
||||
started: list[str] = []
|
||||
|
||||
def fake_run(command: list[str], **kwargs: object) -> object:
|
||||
calls.append(command)
|
||||
return type(
|
||||
"Result",
|
||||
(),
|
||||
{
|
||||
"returncode": 1,
|
||||
"stdout": "",
|
||||
"stderr": stderr,
|
||||
},
|
||||
)()
|
||||
|
||||
monkeypatch.setattr("headroom.install.supervisors.run", fake_run)
|
||||
monkeypatch.setattr("headroom.install.runtime.runtime_ready", lambda manifest: False)
|
||||
monkeypatch.setattr(
|
||||
"headroom.install.runtime.start_detached_agent",
|
||||
lambda profile: started.append(profile),
|
||||
)
|
||||
start_supervisor(_manifest(supervisor=SupervisorKind.SERVICE.value))
|
||||
assert calls == [
|
||||
["schtasks", "/Change", "/TN", "headroom-default-health", "/ENABLE"],
|
||||
["schtasks", "/Run", "/TN", "headroom-default-startup"],
|
||||
]
|
||||
assert started == ["default"]
|
||||
|
||||
|
||||
def test_windows_start_skips_detached_agent_when_already_ready(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setattr("headroom.install.supervisors.sys.platform", "win32")
|
||||
started: list[str] = []
|
||||
|
||||
def fake_run(command: list[str], **kwargs: object) -> object:
|
||||
return type(
|
||||
"Result",
|
||||
(),
|
||||
{
|
||||
"returncode": 1,
|
||||
"stdout": "",
|
||||
"stderr": "ERROR: The system cannot find the file specified.",
|
||||
},
|
||||
)()
|
||||
|
||||
monkeypatch.setattr("headroom.install.supervisors.run", fake_run)
|
||||
monkeypatch.setattr("headroom.install.runtime.runtime_ready", lambda manifest: True)
|
||||
monkeypatch.setattr(
|
||||
"headroom.install.runtime.start_detached_agent",
|
||||
lambda profile: started.append(profile),
|
||||
)
|
||||
start_supervisor(_manifest(supervisor=SupervisorKind.SERVICE.value))
|
||||
assert started == []
|
||||
|
||||
|
||||
def test_windows_start_surfaces_access_denied(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr("headroom.install.supervisors.sys.platform", "win32")
|
||||
|
||||
def fake_run(command: list[str], **kwargs: object) -> object:
|
||||
return type(
|
||||
"Result",
|
||||
(),
|
||||
{
|
||||
"returncode": 1,
|
||||
"stdout": "",
|
||||
"stderr": "ERROR: Access is denied.",
|
||||
},
|
||||
)()
|
||||
|
||||
monkeypatch.setattr("headroom.install.supervisors.run", fake_run)
|
||||
with pytest.raises(click.ClickException, match="Access is denied"):
|
||||
start_supervisor(_manifest(supervisor=SupervisorKind.SERVICE.value))
|
||||
|
||||
|
||||
def test_macos_start_bootstraps_when_job_not_registered(monkeypatch, tmp_path: Path) -> None:
|
||||
# Post-`stop`/`restart` state: the job was booted out, so `kickstart` fails
|
||||
# (launchctl 113) and start must bootstrap the plist instead.
|
||||
|
||||
Reference in New Issue
Block a user