docs: reorder durable harness milestones

packages/durable/docs/pico-v5-handoff.md

@@ -265,201 +265,256 @@ commits; delivery Context value preservation; retained earlier revision
 stability; trusted mutation footguns; retirement before start and while active;
 recreation; idempotent stop; second-start rejection; cancellation during
 acquisition; cancellation/close during a callback without aborting or joining it;
-listener-error settlement; and invocation-owned cleanup in package 15.
+listener-error settlement; and invocation-owned cleanup in package 14.
 
-## 13. Conversations and entries
+Packages 13–20 are vertical milestones. Each must leave one real public path
+working end to end; do not defer all integration to the last package.
 
-Implement conversation history/ownership records, entry creation,
-conversation-bound cursor-based fork-aware scans, head lookup, and entry edits.
-Expose public history pagination through `Conversation.entries()`, never through
-`Harness`.
+Cross-cutting constraints for these milestones:
 
-Test conversation creation and actual forks, deep ancestor caps, same-commit
-entry prefixes, newest-edit wins, self-head resolution, raw head-to-tail
-transcript, and ownership traversal.
+- Introduce each persisted built-in document kind once, with its final schema,
+  history/fork policy, migration, checkpoint predicate, and public mount path.
+  Record that concrete protocol in the normative specification when the kind
+  lands. Do not add temporary built-in kinds or later replace a fake schema.
+- Keep all visible progress durable. Tests may use faux models and fake effects,
+  but production code must use pi-ai's exported `Models` interface and the real
+  task chain; do not add a Pico model adapter or production fake successor.
+- Prepare every loaded `ConversationView` revision from the complete candidate
+  Session commit before Storage admission. Never rebuild a view by subscribing
+  to already-committed table/document publications.
+- A milestone may leave later operations unimplemented, but it must not expose a
+  temporary public facade. Extend the final §2.2 objects as later behavior lands.
 
-## 14. Context derivation and system messages
+## 13. Openable Harness
 
-Implement model-context reduction, PR #9548 positional `SystemMessage` replay,
-tool-result ordering, and missing post-fork tool results. Replay `content`,
-ordered named `sections` with `null` removal, then tool removals/additions.
+Implement the first usable slice of the final §2.2 surface:
+`Harness.open/close`, stable root creation and lookup, conversation lookup and
+creation, concrete-entry forks, `onConversation`, conversation-bound `commit()`
+and `entries()`, and generic inherited Session document APIs. Implement the
+public `Entry` definition token and `Conversation` handles rather than adding an
+intermediate capability facade.
 
-Test model-less and excluded-stop-reason entries, replacements/omissions,
-multiple heads, section replacement/removal/re-addition order, order-only
-configuration changes between separate request preparations, rejection of
-integer-like section keys, tool addition/removal/replacement order, and raw-view
-versus model context.
+Bring conversation and entry semantics up to the normative contract: explicit
+ownership, fork-aware cursor pagination, head lookup, entry edits, and model
+context derivation. Context reduction includes newest-edit wins, positional PR
+#9548 `SystemMessage` replay, tool-result ordering, missing post-fork tool
+results, excluded stop reasons, and separate raw-history/model-context results.
 
-## 15. Task definitions and invocations
+Define the final rewindable/as-of conversation configuration document now. It
+contains model selection, default `"off"` thinking, ordered section values, and
+active tool names. Implement every configuration getter/setter and atomic
+configuration seeding/override needed by root, independent conversation, and
+fork creation. Definitions supplied through final Harness options may be used
+for section/tool identity validation; dynamic execution and hooks arrive later.
+Do not create temporary fixed document accessors.
+
+Use an internal final-form bootstrap transaction for reserved root ID `1`; do not
+expose a temporary root-creation API or split empty-storage root/config creation
+across commits. `options.root` applies only to empty storage. For this milestone,
+Harness open/reopen acceptance is explicitly limited to storage with no live
+tasks. Package 14 removes that limitation by adding complete open-time task
+reconciliation; Package 13 must not invent partial reconciliation behavior.
+
+Acceptance: open persistent storage, obtain the root, mutate configuration and
+history through public handles, create and fork conversations, close, reopen,
+and verify stable root/conversation identity and state. Test atomic root and
+conversation/config creation; actual forks; deep ancestor caps; same-commit
+entry prefixes; cursor boundaries; self-head resolution; newest-edit wins; raw
+head-to-tail transcript versus model context; model-less and excluded assistant
+entries; replacements/omissions; multiple heads; section replacement/removal/
+re-addition order; integer-like section-key rejection; positional tool changes;
+missing post-fork results; every configuration getter/setter; explicit active-
+tool duplicate/unregistered rejection; default active registry snapshot; as-of
+configuration inheritance including unavailable historical names; seed
+overrides; listener initial/future delivery and isolation; and reopen.
+
+## 14. Durable task runtime
 
 Implement `defineTask`, exhaustive phase maps, full checkpoint replacement,
-kind migration, runtime commits, memos, and invocation close gates.
+kind migration, runtime commits and memos, invocation lifetime gates, scheduler
+reservation, dependencies, terminal outcomes, typed waits, holds, quiescence,
+joins, and orphaning. Complete the task-facing §2.2 methods: `resume`, `suspend`,
+`hold`, task-kind registration, `getTask`, `waitForTask`, `markTask`,
+`abortTask`, and the task-aware portion of idle waits.
 
-Use a fake two-phase effect. Test intent/effect/outcome recovery,
-unchanged-checkpoint faulting, same-phase checkpoint progress, cancellation
-precedence, thrown-handler faulting, close/reopen without abort marks, outcomes,
-or task-document retirement, no fresh phase/abort dispatch while closing,
-first-writer-wins memos, and automatic watch cleanup.
+Include the execution-critical abort core: durable direct-task marks,
+signal-and-join of an active run, fresh abort invocation, run-commit rejection
+after a mark, and close precedence. Deep owned-subtree cascading and background
+boundaries remain Package 18. Open now reconciles every surviving `running` task
+to `pending`, migrates registered kinds, and atomically orphans unknown or
+unmigratable live kinds as required by the normative specification. No handler
+dispatches during open, and dynamic registration does not resurrect a task
+settled by that pass.
 
-## 16. Scheduler and terminal tasks
+Use a fake two-phase external effect to test the real runtime. Acceptance is an
+intent/effect/outcome task interrupted after intent, closed, reopened on the same
+storage, and safely resumed to a durable terminal receipt. Also test unchanged-
+checkpoint faulting, same-phase progress, cancellation precedence, thrown
+handlers, dependencies, result values and entry IDs, first-writer-wins memos,
+terminal checkpoint/memo removal, task-document retirement, close/reopen without
+abort marks or fabricated outcomes, no fresh phase/abort dispatch while closing,
+watch cleanup, holds and quiescence with eligible work, unknown kinds, mark-only
+versus signalling abort, and crashes at every direct-task abort stage.
 
-Implement reservation, running-task reopen reconciliation, dependencies,
-terminal outcomes, waits, holds, joins, and orphaning.
+## 15. First runnable no-tool chat turn
 
-Test result values and entry IDs, terminal records after reopen, dependency
-eligibility, unknown kinds, and terminal removal of checkpoints/memos.
+Implement the smallest real input-to-answer vertical path. Define the final
+inbox, turn-control, and generation presentation documents needed by this path;
+do not use provisional kinds or schemas. Add input `Submission` admission,
+request-ID deduplication, reacquisition and waiting, idle placement, active-turn
+ownership, successful answer settlement, and terminal failure cleanup. Expose
+the final `SubmissionDraft` union rather than an interim input-only API. Complete
+the optional initial input path on conversation creation so conversation,
+configuration, sections, and input admission commit atomically. Busy steer/
+follow-up behavior, passive writes, and reset remain Package 17.
 
-## 17. Abort and owned conversations
+Implement section registration and no-tool request preparation, including exact
+persisted rendered strings, positional system baselines/deltas, head-cut
+rebaselining with `ContextEdit` omissions, and preparation revision checks.
+Implement the ordinary generation phases needed for one response: preparation,
+request intent, durable throttled partials, attempts/retry classification,
+deferred handle polling/cancellation, assistant entry settlement, and input-
+submission completion.
 
-Implement durable abort marks, signal/join/fresh-abort invocation, owned
-conversation creation, subtree traversal, background behavior, and idle waits.
+Call pi-ai only through its exported `Models` methods: `getModel()`,
+`streamSimple()`, `fetchDeferred()`, and `cancelDeferred()`. The test double must
+implement that same interface; production code gets no adapter. A missing configured model produces
+the durable `no_model` failure. All progress exposed to observers is committed
+state, never raw provider frames.
 
-Test commit rejection after a run task is marked, crashes at every abort stage,
-close precedence over a previously marked task, deep ownership trees, atomic
-retirement of task-scoped documents, default non-inheritance, inheritance from
-the current committed tail, an empty source conversation, document fork
-policies, and explicit model/section seed overrides.
+Acceptance: `Harness.open → root → resume → submit(input) → Submission.wait →
+durable assistant answer → close/reopen`, using faux Models in tests. Exercise
+interruption and reopen before/after every implemented generation phase,
+aborted-partial conversion, deferred polling/cancellation, retryable and
+terminal model errors, no-visible-undurable updates, exact section order/value
+patches, complete post-head baselines, retained system deltas on both sides of a
+head marker, atomic input/
+configuration/section creation, and durable submission settlement. This is also
+the first print-mode smoke path: print awaits its own input submission rather
+than global idle.
 
-## 18. Submissions and positional inbox
+## 16. First coding-agent tool turn
 
-Define the initial inbox and turn-control documents, then implement strict input/
-write `SubmissionRecord` variants, `Conversation.submit()`, request-ID
-deduplication, awaitable/reacquirable submissions, busy admission, withdrawal, queue modes,
-and `postTools`/`final` boundaries. Successful input settlement requires an
-answer; write settlement means entry placement and never starts a turn. Use a fake successor
-task.
+Implement task/tool registries and Session/owned-subtree hooks, then wire the
+real generation → tool tasks → post-tools → generation chain. Implement offered-
+set checks, declaration and argument validation, hook composition, durable
+execution intent, stored replay policy, bounded stream/progress documents,
+result entries, post-tools joining, controls, and `postTools`/`final` boundaries.
+The generation task now classifies tool calls and continues through the real
+built-in task chain; neither side uses a production fake successor.
+
+Implement runtime registration lifetimes and preparation behavior for tool
+loadout additions/removals, same-name replacement ordering, complete baseline
+tool declarations, hook memos, and configuration/registry revision retries.
+Do not implement in-process replacement of executing Session-side extension
+code; the normative v1 close/reopen boundary remains Package 20.
+
+Acceptance: input → model tool call → registered local read/bash/edit operation →
+tool result → model answer → durable submission settlement. Run that path once
+normally and once interrupted/reopened. Test recovery from every tool and post-
+tools phase; offered-history enforcement; before/after hook rules; both stored/
+current replay-policy directions; default and overridden output bounds; streamed
+content fallback; progress replacement and coalesced commit settlement; drain-
+before-terminal ordering; abort/close with buffered output; invocation-bound
+handles and watches; `missing_active_tool` settlement; atomic assistant/tool/
+post-tools commits; and all registry lifetime and positional tool-history cases.
+
+## 17. Live UI and product state
+
+Complete submissions and inbox behavior: busy `steer`/`followUp`/`reject`,
+passive writes, withdrawal, ordered `postTools`/`final` selection, stale targets,
+self-head cuts, successor turns, queued reset/handoff, and every terminal cleanup.
+Successful inputs still require an answer; writes settle on placement and never
+start generation.
+
+Define any remaining built-in preference/presentation documents once with final
+schemas. Implement the structural `{ conversation, entries, docs }`
+`ConversationView`, `viewState()`, and `watch()`. Build the first revision lazily
+on the Session line. For each affected commit, derive and prepare one mounted
+operation batch from the complete candidate transaction before Storage
+admission; after success, only install prepared pointers/cursors and enqueue the
+exact immutable frame. Do not derive the mount through `subscribeCommits()`.
+
+Add the §9.4 notification adapter directly from uncoalesced committed
+publication, without another tracker or persistence authority. Wire TUI
+hydration to structural state/watch, print to its own `Submission`, and JSON/RPC
+to correlated commands plus ordered committed notifications. Transport
+backpressure and disconnect policy stay in the mode adapter.
 
 Table-test every submission transition, cross-type request-ID conflicts,
-interleaved steer/follow-up/write selection, self-head cuts, stale targets,
-successor triggers, reopen waits, writes pending without a later boundary,
-compact large-payload removals, abort results for queued/placed/terminal
-submissions, and orphan/fault cleanup of active turn control.
+interleaved queue selection, compact positional removal of large payloads,
+abort results, reopen waits, writes pending without a later boundary, busy reset
+placement, and orphan/fault cleanup. Test one view publication per Session
+commit; atomic entry/preview settlement; parent-linked active entries and heads;
+mounted create/recreate/retire; preparation rollback before Storage; empty-batch
+suppression and redundant nonempty revisions; contiguous delivery; stable public
+paths; O(1) immutable acquisition; structural sharing; serialized consumers;
+bounded reset behind an in-flight callback; durable retry/tool/collapse status;
+output truncation metadata; the documented placement of diagnostics in entries,
+terminal details, or bounded state; asynchronous consumer initialization; and
+absence of semantic projection. Verify watch overflow cannot erase a separately
+subscribed notification lifecycle, late clients hydrate structurally, and
+notifications expose committed throttled
+progress rather than raw provider frames.
 
-## 19. Remaining built-in documents and view
+## 18. Ownership and subagents
 
-Define the concrete configuration, preference, and live presentation documents;
-reuse the approved inbox/turn definitions. Record all IDs, fields, history,
-fork settings, migration, and checkpoint predicates in the normative
-specification.
+Complete the remaining owned-conversation and abort semantics: durable foreground
+subtree cascades, signal/join/fresh-abort across ownership edges, background
+boundaries, ordinary and full traversal, conversation abort, and exact idle
+waits. Finish invocation-bound owned APIs used by tools and the foreground and
+background subagent provisioning patterns, including atomic task/conversation/
+registry creation and request-ID-safe submission recovery.
 
-Implement `{ conversation, entries, docs }` as immutable structurally shared
-revisions produced by the optimized immutable applier. Build the first revision
-lazily on the Session line. For every later affected Session commit, derive the
-mounted operation batch and prepare its next revision before Storage admission;
-a failure rolls back normally. After Storage succeeds, finalization only installs
-prepared pointers/cursors and enqueues publication. Conversation views expose a read-only Chord state directly; their watches use
-Package 12's O(1) acquisition and bounded exact-frame buffering.
+Test deep ownership trees, owner edges after terminal settlement, nested
+background boundaries, conversation abort/join with surviving passive writes and
+background tasks, cancellation of waiters without cancellation of work, and
+atomic cancellation intent. Test default non-inheritance, inheritance from the
+current committed tail, empty source conversations, document fork policies,
+explicit model/section/tool seed overrides, foreground subagent cascade, and
+background supervisor recovery before and after submission admission.
 
-Test direct task writes, one publication per Session commit, atomic
-entry/preview settlement, parent-linked active-entry reconstruction, head
-changes, mounted create/recreate/retire transitions, preparation failure before
-Storage, empty mounted-batch suppression, redundant nonempty mounted revisions,
-contiguous revisions, stable public paths, immutable O(1) acquisition,
-asynchronous consumer
-initialization, serialized updates, revision/payload structural sharing,
-bounded buffering and reset behind an in-flight callback, retry/collapse late-join status,
-bounded-output truncation metadata, and absence of semantic projection. Specify
-which diagnostics become entries, terminal details, or bounded document state.
+## 19. Collapse and overflow
 
-## 20. Registries, hooks, and sections
+Implement manual, threshold, and generation-overflow collapse; exchange-boundary
+range selection; summarization; retry policy; staleness checks; and headed
+summary entries. Wire generation's real overflow path directly to the collapse
+task, and complete `Conversation.collapse()` so it returns the admitted task ID.
 
-Implement task/tool/section registries, Session and owned-subtree hooks,
-positional PR #9548 section/tool updates, complete baselines after a head cut,
-and preparation revision checks. Do not add a Session-kernel semantic event
-journal or extension-state router; package 24 adds the thin product notification
-adapter from specification §9.4.
+Test model context before and after collapse, raw history preservation, provider
+failure, declined and stale work, manual/threshold/overflow admission, late-join
+presentation state, and reopen from every phase. Rerun generation overflow
+integration without a fake collapse kind.
 
-Test registration lifetimes, hook replay with memos, exact persisted rendered
-section strings, minimal section patches and `null` removals, complete baseline
-tool declarations, and a head cut that retains earlier system messages. Verify
-the new baseline entry omits those messages through `ContextEdit` before replay,
-including retained `content`, section order, and tool changes. Include a retained
-delta whose ID precedes the head-carrying entry: select omissions by retained
-context membership, not an ID comparison with the head entry. Test tool loadout
-additions/removals and preparation retry after registry movement. Do
-not implement in-process replacement of Session-side extension code; a host extension
-change uses the Harness close/reopen boundary.
+## 20. Reload and final conformance
 
-## 21. Tool and post-tools tasks
+Complete any remaining §2.2 surface and lifecycle gates, then implement the
+normative v1 host-extension reload path: stop admission/reservation, close and
+join, dispose registrations/facets, rebuild over the same storage with new
+definition tokens and task/tool/section definitions, reopen/migrate live tasks,
+and resume. Ordinary documents migrate on later typed access. The separate live-
+registries proposal remains non-normative unless it is first merged into
+`pico-v5.md`; do not silently substitute it for §7.4.
 
-Implement offered-set checks, argument validation, tool hooks, durable bounded
-progress, owned APIs, interrupted/replay-safe recovery, result entries,
-post-tools joining, controls, and boundaries using fake tools.
+Test that close seals commit and mutation admission, lets already-admitted
+storage settlement finish despite caller cancellation, stops future state/watch
+delivery, joins task/tool/hook invocations outside the Session line, writes no
+abort or terminal outcome, starts no fresh abort invocation, and never runs old
+and new Harness generations concurrently. Include cancellation during watch
+acquisition and an already-running callback that remains caller-owned across
+shutdown. Verify service withdrawal and client detach.
 
-Test recovery from every phase, both stored/current replay-policy directions,
-default and overridden bounds, streamed-content fallback, progress replacement
-and coalesced commit settlement, drain-before-terminal ordering,
-abort/close with buffered output, invocation-bound owned handles, and atomic
-assistant/tool/post-tools settlement. Use a fake
-generation successor; package 22 replaces it and reruns integration.
+Run the exhaustive public conformance matrix: stable persisted root identity;
+all root/create/lookup/fork/reset/collapse/abort/idle paths; every configuration
+getter/setter and fork override; typed input/write submissions; task wait/abort;
+generic document access; task/tool/section registration between open and resume;
+conversation listener isolation; structural watches; and no resurrection of a
+task settled during open. Compile-test every §2.2 and §3 owner/key/seed overload,
+the normative usage sequences, and the Chord guide. The erased registry test must
+use a concrete narrowed-input task with multiple checkpoint phases and custom
+hooks. Verify that a Chord root-replacement delta remains distinct from a
+Session-selected storage checkpoint.
 
-## 22. Generation task
-
-Implement preparation, request intent, durable throttled partials, attempts,
-retry policy, response classification, continuation, and deferred polling/
-cancellation through pi-ai's exported `Models` interface. Do not add a Pico
-model adapter. The faux test double implements that same interface.
-
-Test every phase before and after reopen, aborted partial conversion, overflow
-through a fake collapse kind, input-submission settlement, and no visible-undurable update.
-Replace the fake tool successor and rerun the package 21 integration tests.
-
-## 23. Collapse task
-
-Implement manual, threshold, and overflow collapse; exchange-boundary range
-selection; summarization; retries; staleness; and headed summary entries.
-
-Test context before/after collapse, provider failure, declined/stale work, and
-reopen from every phase. Replace generation's fake overflow target and rerun its
-overflow integration test.
-
-## 24. Harness integration
-
-Implement the exact public surface in specification §2.2:
-`Harness.open/resume/suspend/close`, lifecycle gates, root/create/lookup
-`Conversation` objects, typed input/write `submit()` and `Submission` objects,
-conversation-bound commits and history pagination, fork/collapse/reset/abort/idle,
-typed task wait/abort, generic document access, task/tool/section registries, and
-structural conversation watches. Do not restore Pico3's
-namespace router, fixed document accessors, semantic view events, or manual Chord
-view bridge.
-
-Expose service withdrawal/client detach and product wiring. Implement the §9.4
-agent-mode notification adapter directly from uncoalesced committed publication,
-without another tracker or persistence authority. Migrate TUI hydration to the
-structural conversation watch, make print await its own input `Submission`, and expose
-JSON/RPC correlated commands plus ordered committed notifications. Test that
-watch overflow resets cannot erase a separately subscribed notification
-lifecycle, late clients use structural hydration rather than event
-replay, progress notifications
-reflect durable throttled state rather than every provider frame, and stdout
-backpressure/disconnect policy stays in the mode adapter.
-
-Implement the v1 host-extension reload path as stop admission, close/join, dispose,
-rebuild with new document tokens and registered task/tool/section definitions, reopen/migrate live tasks, and resume. Ordinary documents migrate
-on later typed access. Test that closing seals commit and
-mutation admission, lets storage settlement for already-prepared admitted
-commits finish despite caller cancellation, stops future watch deliveries, joins
-task/tool/hook invocations outside the Session line, writes no abort or terminal
-outcome, starts no fresh abort invocation, and does not run old and new
-generations concurrently. Include cancellation during watch acquisition; an
-already-running watch callback remains caller-owned across shutdown.
-
-Test stable persisted root identity; atomic conversation/config/section/input-
-submission creation; default `"off"` thinking; every configuration getter/setter; explicit
-active-tool seed duplicate/unregistered rejection; default active registry
-snapshot; as-of fork inheritance including unavailable historical names; durable
-`missing_active_tool` settlement; fork seed overrides; concrete-entry forks;
-collapse task-ID return;
-busy reset admission and later placement; mark-only versus signalling abort;
-conversation abort/join with surviving passive writes and background tasks;
-quiescence with eligible work; listener initial/future delivery and isolation;
-and runtime registration between open and resume without resurrection of a task
-settled during open.
-
-Compile-test every §2.2 and §3 owner/key/seed overload plus the usage sequences
-in the normative specification and Chord guide. Verify that a Chord root
-replacement delta remains distinct from a Session-selected storage checkpoint. The erased registry test must include a concrete task with narrowed
-input, multiple checkpoint phases, and custom hooks. Run all package-specific
-tests and the repository check. Verify a local
-coding-agent turn and a reopened interrupted turn, then stop for final review.
+Run all package-specific tests and the repository check. Finish with a local
+coding-agent turn and a reopened interrupted turn through the public Harness,
+then stop for final review.