Files
Octop/tests/unit/infra/utils/test_json_file.py
HUANG Cheng c736e50685 fix(config): never rewrite an unparseable config.json as empty (#730) (#738)
* fix(config): never rewrite an unparseable config.json as empty (#730)

Four call sites read config.json, treated a parse failure as an empty dict and
wrote that back, destroying every other key. One trailing comma plus one
`octop run --port 8088` wiped 17 of 18 keys including the whole `database`
section; the next boot silently came up as a greenfield SQLite instance (fresh
setup-wizard password, zero warnings) while the PostgreSQL data sat unreachable.
Losing `bind_host` the same way is what makes #730's reporter unable to reach
the server from the LAN.

A torn write creates the same precondition with no user error: three of the four
sites wrote config.json with a bare write_text, so a crash / OOM / full disk
mid-write left invalid JSON that the next plugin toggle then wiped.

One shared policy in a new leaf helper, infra/utils/json_file.py:

  absent file            -> {} and write (first run, unchanged)
  valid JSON object      -> merge and write (unchanged)
  unparseable/non-object -> raise CONFIG_FILE_CORRUPT, write nothing

The error names the file and the parser line/column but never the contents,
which hold database credentials. Writes now go through write_json_atomic (temp +
os.replace, fd closed on every path, temp removed on failure) so no reader can
observe a torn file.

Sites: cli/commands/run.py (_save_configfile_overrides, plus
_load_configfile_overrides which used to fall back to launch defaults),
infra/agents/plugins/manager.py (_write_global_plugin_enabled = the dashboard
toggle), infra/setup/service.py (persist_bind_options),
infra/agents/plugins/seed.py. `octop run` / `octop service start` report the
error and exit 1 via fail_octop (now NoReturn) instead of a traceback.

Supersedes the pinned "treat a malformed file as empty so the next run rewrites
it cleanly" contract: its fixture (`{not valid json`) had nothing to lose, so
the test could not observe the destruction. Rewritten as
test_run_reports_corrupt_config_cleanly and
test_run_does_not_overwrite_corrupt_config.

Verified end-to-end against a real OctopServer + real PostgreSQL. Before: one
`octop run --port` flipped driver postgresql -> sqlite, the instance reverted to
the first-run wizard, and users=1/agents=1 were orphaned in PG with no warning.
After: exit 1, config.json byte-identical, boot fails loudly instead of
switching backends. Mutation check: restoring the lenient policy in the helper
reddens 9 tests spanning all four sites.

make all green (3299 passed, 16 skipped); mypy --strict clean; dashboard
`npx tsc --noEmit` clean.

* fix(config): name the file when config.json is corrupt (#730)

`load_config` let a bare `json.JSONDecodeError` escape: it reports a line and
column but never says *which* file failed, and this entry point is reached from
boot, backup and several CLI commands — not only `octop run`. Raise a ValueError
naming the path and the parser position, refuse a non-object document, and never
echo file contents (config.json holds database credentials).

Same policy as infra/utils/json_file.py, duplicated here because octop.config
must stay free of infra imports (AGENTS.md §5).

`octop run` already exits cleanly on a corrupt config through resolve_bind; this
covers the remaining entry points.

make all green (3301 passed, 16 skipped); mypy --strict clean.

* fix(config): preserve config.json permissions in atomic writes

Self-review against CONTRIBUTING.md / AGENTS.md §7 caught a regression in the
atomic writer added earlier in this branch: tempfile.mkstemp creates the temp
file 0600, so replacing config.json silently tightened an existing 0644 file to
0600. Under `octop service start --scope system` that runs as root and writes
into the service user's home, a root-owned 0600 config.json is unreadable by the
service user and the unit cannot start; the bare write_text it replaced produced
the umask default.

write_json_atomic now chmods the temp file to the mode of the file being
replaced, or to `0o666 & ~umask` for a new file — matching the two pre-existing
config.json writers (infra/db/rebind.py, infra/backup/auto.py; both
write_text + replace) rather than mkstemp's 0600. POSIX-guarded, so Windows
behaviour is unchanged and no test asserts mode bits off POSIX (AGENTS.md §7).

Also adds the missing CHANGELOG line for the load_config error-message change.

Verified: new file -> 0644, existing 0644 -> 0644, existing 0600 -> 0600.
make all green (3303 passed, 16 skipped); make typecheck-frontend green;
eslint 0 errors (67 pre-existing react-hooks warnings).

Note: `make check-all` is red on develop itself — prettier --check fails on
dashboard/src/pages/Agent/Skills/components/SkillsTabs.tsx exactly as committed
by 83378876. Left untouched here; confirmed pre-existing by checking develop's
version of that file in isolation.
2026-09-17 15:36:40 +08:00

108 lines
3.8 KiB
Python

"""Unit tests for the shared JSON config helpers (issue #730)."""
from __future__ import annotations
import json
import os
import stat
from pathlib import Path
import pytest
from octop.infra.utils.json_file import (
JsonFileCorruptError,
read_json_object,
write_json_atomic,
)
posix_only = pytest.mark.skipif(os.name != "posix", reason="POSIX file modes only")
def test_read_absent_file_returns_none(tmp_path: Path) -> None:
assert read_json_object(tmp_path / "missing.json") is None
def test_read_valid_object(tmp_path: Path) -> None:
path = tmp_path / "config.json"
path.write_text(json.dumps({"bind_host": "0.0.0.0", "port": 8088}), encoding="utf-8")
assert read_json_object(path) == {"bind_host": "0.0.0.0", "port": 8088}
def test_read_corrupt_raises_with_position_and_never_leaks_contents(tmp_path: Path) -> None:
path = tmp_path / "config.json"
# Trailing comma — the classic hand-edit slip. The password must not surface.
path.write_text('{"database": {"password": "s3cret"},}', encoding="utf-8")
with pytest.raises(JsonFileCorruptError) as excinfo:
read_json_object(path)
err = excinfo.value
assert err.path == path
assert "line" in err.detail and "column" in err.detail
assert "s3cret" not in err.detail
assert "s3cret" not in str(err)
def test_read_non_object_raises(tmp_path: Path) -> None:
path = tmp_path / "config.json"
path.write_text("[1, 2]", encoding="utf-8")
with pytest.raises(JsonFileCorruptError) as excinfo:
read_json_object(path)
assert "list" in excinfo.value.detail
def test_write_is_pretty_newline_terminated_and_leaves_no_temp_files(tmp_path: Path) -> None:
path = tmp_path / "config.json"
write_json_atomic(path, {"a": 1})
text = path.read_text(encoding="utf-8")
assert json.loads(text) == {"a": 1}
assert text.endswith("\n")
assert [p.name for p in tmp_path.iterdir()] == ["config.json"]
def test_write_creates_missing_parent(tmp_path: Path) -> None:
path = tmp_path / "nested" / "config.json"
write_json_atomic(path, {"a": 1})
assert json.loads(path.read_text(encoding="utf-8")) == {"a": 1}
def test_failed_replace_leaves_original_intact_and_cleans_temp(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""A torn write must not be observable: the old file survives byte-for-byte."""
path = tmp_path / "config.json"
original = '{"bind_host": "0.0.0.0", "database": {"driver": "postgresql"}}'
path.write_text(original, encoding="utf-8")
def boom(*_args: object, **_kwargs: object) -> None:
raise OSError("disk full")
monkeypatch.setattr("octop.infra.utils.json_file.os.replace", boom)
with pytest.raises(OSError, match="disk full"):
write_json_atomic(path, {"port": 1})
assert path.read_text(encoding="utf-8") == original
assert [p.name for p in tmp_path.iterdir()] == ["config.json"]
@posix_only
def test_write_preserves_existing_file_mode(tmp_path: Path) -> None:
"""A config write must never silently change an existing file's mode."""
path = tmp_path / "config.json"
path.write_text("{}", encoding="utf-8")
os.chmod(path, 0o640)
write_json_atomic(path, {"a": 1})
assert stat.S_IMODE(path.stat().st_mode) == 0o640
@posix_only
def test_write_new_file_uses_umask_default_not_mkstemp_0600(tmp_path: Path) -> None:
"""mkstemp's 0600 must not leak into a new config.json.
A root-created 0600 file would be unreadable to the service user after
``octop service start --scope system``; the two pre-existing config.json
writers (``db/rebind.py``, ``backup/auto.py``) produce the umask default.
"""
mask = os.umask(0o022)
os.umask(mask)
path = tmp_path / "fresh.json"
write_json_atomic(path, {"a": 1})
assert stat.S_IMODE(path.stat().st_mode) == 0o666 & ~mask