Files
OpenMAIC/packages
wyuc cfae106b89 fix(storage): keep jsonb writes valid when model output contains NUL or lone surrogates (#1499)
* fix(storage): keep jsonb writes valid when model output contains NUL or lone surrogates

PostgreSQL jsonb rejects the `\u0000` and lone UTF-16 surrogate escape
sequences that JSON.stringify emits verbatim, failing the enclosing
statement with SQLSTATE 22P05/22P02. In the agent-session store the failed
tree-entry/event write is treated as critical, so the whole run aborts and
all work in that run is lost.

Add a shared encodeJson at @openmaic/storage/src/pg-json.ts that replaces
U+0000 and unpaired surrogates with U+FFFD while preserving valid surrogate
pairs (emoji), sanitizing in-memory values and object keys before
JSON.stringify. Route every jsonb parameter through it: the agent-session,
document, runtime, asset, and material backends, plus owner-material
registration in lib/persistence.

The document/runtime/asset backends keep their existing assertJsonValue
guard, which rejects these code points with a readable error before the
shared serializer runs; the agent-session and material paths had no guard
and were the live 22P05 exposure. Literal backslash-u text is untouched.

* chore(storage): bump @openmaic/storage to 0.29.2

* fix(storage): keep colliding sanitized keys and own __proto__ members when encoding jsonb

encodeJson replaces NUL and lone surrogates in object keys, but two distinct
keys can sanitize to the same string: "a\u0000" and "a\uFFFD" both become
"a\uFFFD". The rebuild assigned each member by its sanitized key, so a later
member silently overwrote an earlier one and that earlier value was lost.

Keep the first member under the sanitized key and give each later colliding
member a deterministic suffix (#2, #3, ... appended until the key is unique
among the keys emitted so far, whether sanitized or original). Member order is
preserved. The suffix is ASCII and contains no NUL or surrogate, and the
sanitizer only ever emits U+FFFD, so a suffixed key can never be confused with
a bare replacement result.

The rebuild also used a plain {} object, so assigning an own "__proto__" member
invoked the prototype setter and dropped it from the output whenever a sibling
key needed sanitizing. Build the rebuilt object with a null prototype so
"__proto__" stays an own data property; JSON.stringify then emits it and
PostgreSQL round-trips it. Arrays and the no-op fast path (return the original
reference when nothing needs sanitizing) are unchanged.
2026-09-14 18:29:55 +02:00
..