DottaandPaperclip
18e8c121d9
## Thinking Path > - Paperclip manages agents through a shared native runner. > - Built-in harness support should ship with Paperclip's public distribution. > - Grok already speaks ACP; it does not require a new public bridge package. > - Sandbox provisioning owns the native executable and its pinned version. > - The runner must verify that prerequisite without downloading it during npm installation. > - This change separates built-in launcher identity from external runtime identity. > - Clean npm installation and live staging checks verify the distribution boundary. ## Linked Issues or Issue Description Refs #13882, #13973, #13977, #13979. This follow-up now targets master after #13882 was squash-merged. It replaces the private `@paperclipai/grok-acp` workspace package with runner-owned assets. Current master is included so the branch also contains the merged scheduler, complete-event capture, and durable cleanup fixes. ## What Changed - Ship Grok launcher and qualification metadata inside the runner's compiled output and the public server's vendored runner tree. - Remove the separate Grok npm package and all package-manager install hooks for this runtime. - Require the checksum-verified Grok Build 1.0.13 binary at `/opt/paperclip/providers/grok/1.0.13/grok` in the selected execution environment. Provision it explicitly in the Daytona image and CI setup. - Keep native binaries outside the provider pack. Bind the built-in launcher into the pack manifest. - Preserve executable leases, descriptor-backed startup, credential fences, permissions, and exact ACP model admission. - Use `builtin:grok-acp` and `native:grok` as profile identities. Historical package-profile sessions fail closed on resume rather than being silently reinterpreted. - Resolve built-in assets from the authenticated sidecar location, including public server npm layouts. Keep the controller path out of provider environments. - Add clean npm tarball installation verification to the existing trusted canary CI job and the admitted manual EC2 verification path. It stages a unified release version and runs npm lifecycle scripts, then verifies missing-prerequisite rejection and admission after separate provisioning without credentials or inference. - Include the controller-owned provider pack in stamped Cloud images. Unstamped local images omit the pack and remain usable; remote ACPX requires full source provenance. - Correct CLI approval-page metadata for an already authenticated Cloud board user; approval authorization remains unchanged. - Honor explicit native-runner enablement in the Cloud agent picker and direct setup page, keeping the flag disabled by default. - Allow selecting the execution environment before connecting credentials. Include Grok in the existing authenticated hello-probe flow, targeting its pinned native prerequisite for runner setup. - Recover an existing subscription sign-in conflict through an explicit cancel-and-retry action, serialized after cancellation succeeds. - Preserve the selected ACPX harness before normalizing config fields, so new Grok agents use the Grok default model. - Keep the credential-free Cloud provider pack root-owned and readable after runtime UID remapping; verify manifest and referenced asset access under an unrelated unprivileged UID during image builds. - Archive prior failover backups alongside explicitly replaced harness state, preserving evidence while preventing stale backups from blocking a fresh replacement. - Update Daytona image content inputs and contract tests for the built-in assets and explicit provisioner. - Document and regression-test the shared `approve-all` default for Grok setup, saved configuration, and native execution. Explicitly saved restrictions remain unchanged. ## Verification Current merge-repair head `df09eb3e1a619430ad8419a0ee9aedd486689b05` incorporates master `f1a394bd30cb56fb9e479f98b9f50176fe921858` after the base PR was squash-merged. All 12 conflicts came from incoming files identical to the tested pre-squash base. The final tree exactly matches a three-way merge using that original base, preserving built-in Grok distribution and removal of the obsolete private package. All 252 focused runner/UI tests, six npm-isolation tests, and token gates pass. Fresh exact-head Greptile review is 5/5 with no outstanding findings; security scans and EC2 native compilation pass. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([run 36468768035](https://github.com/paperclipai/paperclip/actions/runs/36468768035)). The repository owner explicitly authorized bypassing code-owner approval after all checks passed; no CI checks or repository protection settings are bypassed or changed. The only remaining PR was removed from the completed stack metadata to permit native auto-merge. Earlier integration head `78cb306ecc41b5c96577c26c1d89153b0ef865a1` includes master `3447609d2247e75e55d91493dda91a608364f672` (2026-09-28). Two master advances during verification overlapped the eval catalog; the final merge preserves Grok qualification, completion updates, and bounded API-response reading in all 348 cells. All 77 focused catalog/eval/workflow tests pass. Both native stack layers (#14397) are mergeable, and both exact-head Greptile reviews are 5/5 with successful security scans and no unresolved review threads. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([CI attempts](https://github.com/paperclipai/paperclip/actions/runs/36447124691)). The initial attempt lost two EC2 runners to shutdown signals and stalled a third shard during dependency preparation; all three passed the same-commit failed-job-only retry. Trunk code-owner requirements remain enforced. The review summary’s non-blocking saved-asset offset classification note concerns code already merged in #14301; those runtime files are identical to master and outside this stack’s diff. Historical live evidence below retains its original source revisions. [Final public npm verification](https://github.com/paperclipai/paperclip/actions/runs/36445542764) passed on `76ea70cd4d13786a042af9df82f0fd7a8c85ae30`: 17 public packages, an executed offline lifecycle sentinel, unchanged consumer lock, built-in launcher, missing-prerequisite rejection, and verified separately provisioned binary/command lease. Provisioning and cleanup require no host privilege elevation; only the positive probe mounts the temporary native binary read-only. The verifier is unchanged by the final master merge. All six isolation tests and an offline npm smoke test pass. The prior head had 56 green CI checks and a 5/5 review after two unchanged tests timed out and passed a failed-job-only retry ([CI attempts](https://github.com/paperclipai/paperclip/actions/runs/36444597313)). All 56 recovery-display/lineage tests pass; re-review cleared the already-covered missed-retry concern. Earlier EC2 failures remain retained: [npm lockfile rejection](https://github.com/paperclipai/paperclip/actions/runs/36436311203), [missing compiler in the slim image](https://github.com/paperclipai/paperclip/actions/runs/36440210984), and the aggregate 15-minute test timeouts in those broad runs. Both broad attempts passed typecheck, token gates, Product E2E type/unit checks and build. The focused EC2 lane preserves the existing trusted-actor and immutable-source gates. Earlier documentation/test checkpoint `ff244c4fd78a7ede5a3e00efe09f475f133ef33e` leaves runtime behavior unchanged. 154 focused tests pass across configuration building, native provider resolution, permission policy, credentials, UI configuration, and new-agent setup (including both Grok auth modes); token gates pass. All fresh CI is green for this head: 56 successful checks/statuses and two intentional skips ([run 36367065119](https://github.com/paperclipai/paperclip/actions/runs/36367065119)). Greptile is 5/5 with no new findings. Grok already inherits the shared `approve-all` default, so unattended setup requires no manual permission change. Runtime head `bb5a9307991f1ac567b781970ef11b39d518e19b` fixes a final staging continuation failure before provider startup: explicit replacement archived the old harness but left its failover backups active, which caused `runner_harness_state_mismatch`. The regression fails before the fix and passes after it; all eight adjacent recovery-safety cases also pass. Old backups remain inspectable inside the continuity archive. All fresh CI is green at this head ([run 36360839248](https://github.com/paperclipai/paperclip/actions/runs/36360839248)), with a 5/5 review. One unrelated Cursor test timed out in the initial server shard; the same-commit failed-job rerun passed, and both attempts are retained. Staging deployment is confirmed healthy on this revision. The controller image is `ghcr.io/paperclipai/paperclip@sha256:6ad91c487910ccd2596ff7aed0a3a3ea5233d12b51b83cd6e1402237749b9673`. The final browser-created staging task passed on this exact revision with API authentication: context read → structured human question → controller restart → answer submission → same native provider session resumed → document saved → task Done. The two turns took approximately 119s and 77s. The actual write receipt was applied, and the saved document has exactly one revision containing the selected answer and requested marker. Usage and cost were not reported. [Controller image build](https://github.com/paperclipai/paperclip/actions/runs/36360889243). - Previous integration head `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`: all CI green (53 successful checks/statuses, two intentional skips), including repository typecheck/build/tests, native Runner tests, browser shards, and canary installation checks. [CI run 36358672529](https://github.com/paperclipai/paperclip/actions/runs/36358672529). Greptile is 5/5 with no unresolved findings. - Focused checks cover Grok credentials, executable admission, launcher assets, provider-pack paths/permissions, workflow contracts, setup defaults, CLI authorization, and subscription conflict recovery. All 39 protocol definitions validate. Final integration checks pass 124 catalog/evidence/cache tests and nine project-form tests; token gates pass. Some local dependency checks could not load the stale installed dependency tree; the corresponding fresh EC2 checks pass. - Clean public npm installation passed on EC2 at `8b172ebcf8e02e30662d830c00f3961e3bd459ec` ([run 36164964900](https://github.com/paperclipai/paperclip/actions/runs/36164964900)): 17 unified-version packages, lifecycle scripts enabled, built-in launcher present, no separate Grok package or npm-downloaded binary, missing prerequisite rejected, separately provisioned native executable and command lease verified. No credentials or inference were used. Subsequent changes preserve this npm asset layout. - The immutable Daytona prerequisite image is `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:98957d5be0ac774d086b6402b5849e8e6356fec70fb8c09fca6eb4ed6de918e0`, built from `5a2db471f3ddabe77f9f80e76ed27f996cb97fba`. The previous Cloud controller image was `ghcr.io/paperclipai/paperclip@sha256:fd914e1ab1e45f741e8e078ff452d16f082d7ac05f9b4b3506d3a3c64150d204`, built from `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`; it is superseded by the latest image above. Its EC2 build verified provider-pack access under an unrelated unprivileged UID. - Browser staging at `40f898bc4cba73c1dff4e6344a3983ba0fb247ef` passed full Grok onboarding with the correct `grok-4.7` model, saved credential delivery, and pinned Daytona execution. A browser-created task read context and asked the structured human question. After a controller restart, answering the persisted question resumed the same native provider session, saved the requested document, and completed the task. Actual tool outcomes and durable state agree: one question and one document revision. The two successful turns took 42.7s and 63.1s; usage and cost were not reported. - Restricted policy returned the expected `approval_required` outcome. Functional staging tests explicitly selected `approve-all`; controller authorization and governed approvals remain enforced. Temporary board CLI access was revoked and verified rejected (HTTP 401), and the disposable onboarding agent was paused. Failures remain retained: the pre-fix continuation failure (its task remains blocked; the passing final task is fresh), the original Cloud provider-pack permission failure, the expected restricted-policy denial, the superseded npm staging failure, and an earlier monolithic CI infrastructure timeout. Browser CI exposed a project alias/form race; the final stack uses master's stronger draft-preservation fix and all browser shards pass. Historical full subscription/API protocol and Product rosters retain their original source revisions and do not qualify this packaging revision. No local Docker or Rust build was used. ## Risks The branch includes master’s draft-preservation fix for project URL aliases. It keeps the same project’s edit form mounted and clears prior data when the project or company changes. Custom sandboxes and local execution hosts must provision the pinned binary before Grok starts. Missing, changed, unsupported-platform, and symlinked executables fail admission. The new builtin profile cannot resume sessions created with the former private-package profile. Existing Claude/Codex npm bridge profiles retain their package pins. Grok restricted modes preserve the selected policy but cannot automatically admit Paperclip calls: ACP permission metadata does not independently bind tool authority, so those calls stop with `approval_required`. New Grok configurations default to `approve-all`, including API configurations that omit the mode. Existing explicitly restricted configurations remain restricted; controller authorization and governed approvals remain enforced. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
8.2 KiB
Grok Build native runner
Select Grok Build in the native runner provider selector. The stored contract is
adapterType: "paperclip_runner" with provider: "acpx", acpxAgent: "grok",
and model: "grok-4.7". Existing grok_local agents keep their legacy adapter.
New Grok runner agents default to Full auto (approve all)
(acpxPermissionMode: "approve-all") in setup and the configuration form.
API configurations that omit the permission mode use the same default. No
additional permission setting is needed for unattended execution. Explicitly
saved restrictions remain unchanged.
On Cloud, an operator must enable enableNativeRunner for the instance before
the new-agent picker or direct setup page offers the native runner.
Grok Build speaks ACP over stdio.
The runner owns grok agent --no-leader stdio through ACPX, including session
identity, cancellation, recovery and the authenticated Paperclip MCP bridge.
ACP permission requests are approved by the runner under the default full-auto
policy. It does not add --always-approve: permission decisions remain under
the selected ACPX policy. Grok's ACP metadata cannot independently establish
Paperclip tool authority, so explicitly selecting approve-paperclip or
approve-reads returns the approval-required outcome; deny-all rejects requests.
Full auto does not bypass Paperclip's company permissions, governed approvals,
or execution-environment boundaries. Isolated ask
rules override project allow rules, and compatible always-approve settings are
locked off. Compatible hook/MCP discovery and shell login capture are disabled.
Installation and identity
Grok support ships inside the native runner and the public Paperclip server npm
artifact. There is no separate Grok npm package, binary payload, or npm lifecycle
download. The built-in launcher is identified as builtin:grok-acp version 1;
its native runtime identity is native:grok version 1.0.13. The historical
agentServerPackage/agentRuntimePackage wire fields carry these identities,
not npm dependencies. Existing npm-backed ACP bridges retain their package pins.
Provision Grok Build 1.0.13 at
/opt/paperclip/providers/grok/1.0.13/grok in the selected execution environment.
The sandbox provisioning helper is explicit and is never run by npm:
sudo node packages/paperclip-runner/scripts/provision-grok.mjs /opt/paperclip/providers/grok/1.0.13/grok
The standard Daytona image provisions it separately from the provider pack.
Custom images and local execution hosts must provide the same prerequisite.
The runner verifies the native executable checksum before credential refresh or
ACP startup; a missing prerequisite reports the required path and version.
Only macOS arm64 and Linux x64 are qualified. No ambient grok from PATH is used.
ACP must report the requested exact model; mismatches fail closed.
The distribution identity change deliberately rejects resume bindings from the former private-package profile. Start a fresh session after upgrading that unreleased profile; do not silently reinterpret its saved identity.
Instructions use Grok ACP session rules. Assigned skills live in the isolated Grok home. Steering and goals are unsupported. Token and cost values remain unknown when Grok does not report them; missing usage is not zero usage.
Authentication
Use the existing Grok company connection/login flow for subscription execution.
An explicitly selected company-secret XAI_API_KEY selects paid API execution.
There is no automatic subscription-to-API fallback. Remote execution cannot
borrow the operator's home credentials. Local execution can use the operator's
existing Grok login when no company login has been selected.
Only the selected credential is staged in the private runtime home. An ownership
lease fences concurrent processes. Before ACP startup, an expiring subscription
credential is refreshed through the verified Grok executable’s non-inference
models command. This bounded step suppresses output and prevents Grok 1.0.13
from caching a pre-refresh model list. Exact model verification still precedes
any prompt; refresh failure requires reconnecting Grok Build. After the provider exits, refreshed credentials
are copied back through Grok's existing identity and refresh checks. Runtime
credentials, refresh handoffs, and diagnostic logs are excluded from workspace
backups and removed on close. Session history remains available for resume. Host
configuration, other provider credentials and unselected keys are not forwarded
to Grok.
Evaluation
The private paperclip-evals repository maintains live-acpx-grok-4.7.json and
rosters/live-acpx-grok.json. The roster covers all 39 protocol cases. Its campaign
lane remains disabled pending complete live qualification.
Product E2E exposes runner-acpx-grok in core local/Daytona compatibility and
local session integrity. The explicit grok-qualification suite covers replies,
planning approvals, structured questions, downloadable project revisions, stop/resume
and continuation after controller restart
in both environments. Run with --suite grok-qualification; it is excluded from
scheduled --all. Use the canonical Product E2E dashboard and Evalbook reports.
The separate grok-subscription-qualification suite covers the same workflows
with the explicit GROK_AUTH_JSON fixture credential. It seeds only the disposable
company's private login home and supplies no API key. It does not exercise the
interactive sign-in UI. Authentication mode remains part of the profile identity.
After a subscription upgrade, a fresh grok login may be needed if the existing
login still reports the previous entitlement through ACP.
packages/paperclip-runner/scripts/grok-native-smoke.mjs records explicit auth,
model/binary identity, MCP outcome, durable resume and restrictive permissions.
Pass --auth subscription --auth-file /private/path/auth.json --output /private/report.json
or --auth api --output /private/report.json with an explicitly supplied key.
Each attempt is retained; unknown usage and cost are null.
Qualification requires the full protocol roster, selected product workflows, subscription and API execution locally and on Daytona, and three successful repetitions of core tool, approval and resume cases. Deterministic tests or a single successful browser task do not establish that qualification.
Remote verification
Do not run Docker on a developer laptop when using remote verification. The
Docker Runner check workflow offers maintainer-authorized manual EC2 image
builds and broad source checks without provider credentials. It records the
source revision, resolved lock digest and immutable image reference. Paid
Product E2E remains behind the protected default-branch workflow and environment.
The public npm consumer check uses a digest-pinned, unprivileged container with
no checkout or credentials mounted. It downloads dependencies with lifecycle
scripts disabled and freezes the resulting consumer lockfile. It completes the
clean install with offline npm rebuild, running the deferred lifecycle hooks
without re-resolving bundled optional dependencies. Networking stays disabled,
the lockfile must remain unchanged, and a sentinel proves scripts actually ran.
The pinned image includes native build tools and local Node headers so dependency
hooks can compile without network access. Both executable admission probes run
in the same isolation. The verification user provisions Grok inside the disposable
test directory without privilege elevation or host /opt changes. Only the
positive probe mounts that binary read-only at the canonical sandbox path.
The Cloud application image also carries the controller-owned provider pack and
sets PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH. Remote ACPX execution verifies
the sandbox against that pack before using it, or stages the matching pack when
needed. The Cloud controller image does not install the native Grok executable;
the selected sandbox image must provide the prerequisite above.
Cloud builds must supply the full source SHA through PAPERCLIP_BUILD_COMMIT
to produce that verified pack. Unstamped local Cloud builds still work for other
features, but omit the pack and cannot start remote ACPX sessions.