mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
<!-- maintainer-current-main -->
## Maintainer latest-main checkpoint
Published `c106d908edc30eb41dea5dbbb275d0edd3a19d0a` contains current
upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 82
supervisor/Gemini tests passed, 2 supervisor skips. Earlier live
behavior proofs below retain their commit/platform scope; contributor
runtime logic is unchanged.
The common shard-2 failure was independently confirmed on main's hosted
job 127069149772 and locally (four failures, six passes): the Gemini
memory fixture mocked an unused body reader and provided no request
stream. The maintainer test-only follow-up now constructs a real
FastAPI/ASGI Request carrying JSON bytes and exercises the actual body
reader. All six test-function ASTs/assertions are unchanged, and all 10
parameterized Gemini cases pass. Touched-file Ruff/format and whitespace
checks pass. This fixes the inherited fixture instead of asking
contributors to run CI or weakening the memory/content assertions. Fresh
exact-head hosted qualification is requested. Nothing merged.
<!-- /maintainer-current-main -->
## Description
`schtasks /Query /TN <name> /XML` output can declare `encoding="UTF-16"`
while returning bytes that are actually UTF-8/ANSI without NUL bytes, or
UTF-8 with a BOM. Because `_parse_windows_task_xml` relied on
`ElementTree.fromstring(raw)` first before falling back to decode on
`ParseError`, `ElementTree` raised `ParseError: encoding specified in
XML declaration is incorrect` when the byte stream did not match the
declared encoding, causing Windows deployment to fail and roll back.
The parser first preserves direct byte-level XML parsing for valid
declared encodings. When parsing reports an encoding or codec failure,
it decodes the declared supported codec (or UTF-16/NUL and UTF-8-BOM
fallback), normalizes the declaration, and parses the resulting text.
Registration read-back wraps parse/codec failures in its existing
task-created error contract so fallback cleanup remains available.
Closes #3969
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/install/supervisors.py`: in `_parse_windows_task_xml`,
decode bytes based on NUL byte presence (`utf-16` vs `utf-8-sig`), and
strip the XML declaration before parsing.
- `tests/test_install/test_supervisors.py`: added tests verifying
`_parse_windows_task_xml` accepts UTF-16-encoded bytes and
UTF-8-BOM-encoded bytes with mismatched or matching XML declarations.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest -q tests/test_install/test_supervisors.py
57 passed, 1 warning in 2.13s
$ ruff check headroom/install/supervisors.py tests/test_install/test_supervisors.py
All checks passed!
$ ruff format --check headroom/install/supervisors.py tests/test_install/test_supervisors.py
2 files already formatted
$ mypy headroom/install/supervisors.py --follow-imports=silent --ignore-missing-imports
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.14.6.
- Exact command / steps: Tested `_parse_windows_task_xml` with byte
payloads having `encoding="UTF-16"` declared in the XML prolog, encoded
both with UTF-8-sig and UTF-16.
- Observed result: Both formats parse cleanly into `ElementTree.Element`
without raising `ParseError`.
- Not tested: Live `schtasks /Create` on Russian/non-Western Windows
codepage systems.
## Runtime Rollout Safety
- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Only Windows Task Scheduler XML
parsing becomes robust against encoding mismatches.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4064"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
## Maintainer current-main qualification
Head `fe7fdd10c98fffdb5664c7d064d7d4bda062d970` incorporates current
upstream main `31dcaf41` without conflicts or contributor logic changes.
Windows/Python 3.12.13: `python -m pytest
tests/test_install/test_supervisors.py -q` reports **66 passed, 2
skipped**. Touched-file Ruff, formatting and whitespace checks pass. The
prior UTF-16BE, Windows-1252, Shift-JIS multibyte text, ANSI and
registration read-back error findings are covered; the Japanese-name
regression asserts its actual decoded content.
Actual OS read-only check: capture `subprocess.run(["schtasks",
"/Query", "/TN", r"\Microsoft\Windows\Defrag\ScheduledDefrag", "/XML"],
capture_output=True)`, then pass its raw stdout to this checkout's
`_parse_windows_task_xml`. Exit 0; 1516 actual XML bytes parsed to the
Task Scheduler Task element. This validates real query-output parsing
without creating or changing any scheduled task. Synthetic
byte/registration-failure tests separately cover mismatched and
non-Western encodings. No live task creation/rollback, non-Western OS
installation, or new native build is claimed. Hosted qualification must
finish on the published head.
### Maintainer current-main requalification
Exact head `0d8e0efc00002a4ae4c9a1791641e1955b4b8021`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.
Requalified current-main integration against upstream
c01cf07d94. The sole conflict was the
inherited Gemini request fixture; adopted main's streamed-body helper
after proving all six test-function ASTs unchanged. No contributor
XML-decoding or task-lifecycle logic rewritten. The byte-first parser,
declared-codec fallbacks and task_created registration-error cleanup
contract remain alongside main's hidden/VBS launcher behavior. Current
author/Devin discussion considered, including actual Japanese text in
the Shift-JIS regression.
Fresh supervisor/Gemini verification: 85 passed, two skips. Ruff
lint/format and main-relative whitespace pass. A real read-only
single-task schtasks query returned 1,826 XML bytes successfully parsed
by this branch. The initial all-task query returned concatenated
documents, outside this single-task parser's contract; no task was
created or modified. No live registration/rollback, non-Western OS or
branch-native build claim. Current-main DLL supports Python test
startup. No source blocker found; formal approval awaits exact-head
hosted checks. Nothing merged.
Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.
---------
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -405,12 +405,39 @@ def _parse_windows_task_xml(raw: bytes | str) -> ElementTree.Element:
|
||||
if isinstance(raw, bytes):
|
||||
try:
|
||||
return ElementTree.fromstring(raw)
|
||||
except ElementTree.ParseError:
|
||||
raw = raw.decode(errors="replace")
|
||||
except (ElementTree.ParseError, ValueError, LookupError):
|
||||
pass
|
||||
|
||||
raw_bytes = raw
|
||||
raw_str = None
|
||||
if b"\x00" in raw_bytes:
|
||||
try:
|
||||
raw_str = raw_bytes.decode("utf-16")
|
||||
except UnicodeDecodeError:
|
||||
raw_str = raw_bytes.decode(errors="replace")
|
||||
else:
|
||||
declared_enc = None
|
||||
match = re.search(rb'<\?xml\s+[^>]*encoding=["\']([^"\']+)["\']', raw_bytes)
|
||||
if match:
|
||||
enc_name = match.group(1).decode("ascii", errors="ignore").lower()
|
||||
if "utf-16" not in enc_name and "utf-32" not in enc_name:
|
||||
declared_enc = enc_name
|
||||
if declared_enc:
|
||||
try:
|
||||
raw_str = raw_bytes.decode(declared_enc)
|
||||
except (LookupError, UnicodeDecodeError):
|
||||
pass
|
||||
if raw_str is None:
|
||||
try:
|
||||
raw_str = raw_bytes.decode("utf-8-sig")
|
||||
except UnicodeDecodeError:
|
||||
raw_str = raw_bytes.decode(errors="replace")
|
||||
else:
|
||||
raw_str = raw
|
||||
normalized = re.sub(
|
||||
r'<\?xml\s+version="1\.0"\s+encoding="[^"]+"\?>',
|
||||
'<?xml version="1.0"?>',
|
||||
raw,
|
||||
r'<\?xml\s+version="1\.0"\s+encoding="[^"]+"\?>\s*',
|
||||
"",
|
||||
raw_str,
|
||||
count=1,
|
||||
)
|
||||
return ElementTree.fromstring(normalized)
|
||||
@@ -444,7 +471,7 @@ def _register_windows_task(name: str, xml: str, *, expected_logon_type: str) ->
|
||||
|
||||
try:
|
||||
task = _parse_windows_task_xml(query.stdout or b"")
|
||||
except ElementTree.ParseError as exc:
|
||||
except (ElementTree.ParseError, ValueError, LookupError) as exc:
|
||||
raise _WindowsTaskRegistrationError(
|
||||
f"Could not parse registered Windows task {name!r}", task_created=True
|
||||
) from exc
|
||||
|
||||
@@ -147,6 +147,112 @@ def test_parse_windows_task_xml_accepts_valid_bytes() -> None:
|
||||
assert task.find("LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_accepts_utf16_encoded_bytes() -> None:
|
||||
xml = (
|
||||
'<?xml version="1.0" encoding="UTF-16"?>\n'
|
||||
'<Task xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">'
|
||||
"<Principals><Principal><LogonType>S4U</LogonType></Principal></Principals>"
|
||||
"</Task>"
|
||||
)
|
||||
task = _parse_windows_task_xml(xml.encode("utf-16"))
|
||||
assert task is not None
|
||||
assert task.find(".//{*}LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_accepts_utf16be_without_bom() -> None:
|
||||
xml = (
|
||||
'<?xml version="1.0" encoding="UTF-16"?>\n'
|
||||
'<Task xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">'
|
||||
"<Principals><Principal><LogonType>S4U</LogonType></Principal></Principals>"
|
||||
"</Task>"
|
||||
)
|
||||
task = _parse_windows_task_xml(xml.encode("utf-16-be"))
|
||||
assert task is not None
|
||||
assert task.find(".//{*}LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_preserves_declared_windows_codepage() -> None:
|
||||
xml = (
|
||||
'<?xml version="1.0" encoding="windows-1252"?>\n'
|
||||
'<Task xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">'
|
||||
"<Name>Caf\xe9</Name>"
|
||||
"<Principals><Principal><LogonType>S4U</LogonType></Principal></Principals>"
|
||||
"</Task>"
|
||||
)
|
||||
task = _parse_windows_task_xml(xml.encode("windows-1252"))
|
||||
assert task is not None
|
||||
assert task.find(".//{*}Name").text == "Caf\xe9"
|
||||
assert task.find(".//{*}LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_accepts_declared_multibyte_encoding() -> None:
|
||||
xml = (
|
||||
'<?xml version="1.0" encoding="shift_jis"?>\n'
|
||||
'<Task xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">'
|
||||
"<Name>\u30c6\u30b9\u30c8</Name>"
|
||||
"<Principals><Principal><LogonType>S4U</LogonType></Principal></Principals>"
|
||||
"</Task>"
|
||||
)
|
||||
task = _parse_windows_task_xml(xml.encode("shift_jis"))
|
||||
assert task is not None
|
||||
assert task.find(".//{*}Name").text == "\u30c6\u30b9\u30c8"
|
||||
assert task.find(".//{*}LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_accepts_ansi_with_invalid_utf8() -> None:
|
||||
xml = (
|
||||
b'<Task xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">'
|
||||
b"<Name>Caf\xe9</Name>"
|
||||
b"<Principals><Principal><LogonType>S4U</LogonType></Principal></Principals>"
|
||||
b"</Task>"
|
||||
)
|
||||
task = _parse_windows_task_xml(xml)
|
||||
assert task is not None
|
||||
assert task.find(".//{*}LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_register_windows_task_cleans_up_on_unparseable_xml(monkeypatch) -> None:
|
||||
def fake_run(command: list[str], **kwargs):
|
||||
if command[1] == "/Query":
|
||||
return _LaunchctlResult(stdout=b"<<<not-xml>>>")
|
||||
return _LaunchctlResult()
|
||||
|
||||
monkeypatch.setattr("headroom.install.supervisors.subprocess.run", fake_run)
|
||||
|
||||
with pytest.raises(_WindowsTaskRegistrationError) as exc_info:
|
||||
_register_windows_task("headroom-test-task", "<Task />", expected_logon_type="S4U")
|
||||
|
||||
assert exc_info.value.task_created is True
|
||||
|
||||
|
||||
def test_register_windows_task_cleans_up_on_unsupported_multibyte_error(monkeypatch) -> None:
|
||||
def fake_run(command: list[str], **kwargs):
|
||||
if command[1] == "/Query":
|
||||
return _LaunchctlResult(
|
||||
stdout=b'<?xml version="1.0" encoding="unsupported-codec"?><Task>'
|
||||
)
|
||||
return _LaunchctlResult()
|
||||
|
||||
monkeypatch.setattr("headroom.install.supervisors.subprocess.run", fake_run)
|
||||
|
||||
with pytest.raises(_WindowsTaskRegistrationError) as exc_info:
|
||||
_register_windows_task("headroom-test-task", "<Task />", expected_logon_type="S4U")
|
||||
|
||||
assert exc_info.value.task_created is True
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_accepts_utf8_bom_bytes() -> None:
|
||||
xml = (
|
||||
'<?xml version="1.0" encoding="UTF-16"?>\n'
|
||||
'<Task xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">'
|
||||
"<Principals><Principal><LogonType>S4U</LogonType></Principal></Principals>"
|
||||
"</Task>"
|
||||
)
|
||||
task = _parse_windows_task_xml(xml.encode("utf-8-sig"))
|
||||
assert task is not None
|
||||
assert task.find(".//{*}LogonType").text == "S4U"
|
||||
|
||||
|
||||
def test_parse_windows_task_xml_accepts_text() -> None:
|
||||
task = _parse_windows_task_xml("<Task><LogonType>InteractiveToken</LogonType></Task>")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user