trending/paperclip
1
0
Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-02 02:07:25 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
18e8c121d99fee1d4038610531ab57ae3f824fdd
paperclip/doc/grok-native-runner.md
T
DottaandPaperclip 18e8c121d9 fix(runner): include Grok support in public installs with sandbox prerequisites (#14024) 
## Thinking Path

> - Paperclip manages agents through a shared native runner.
> - Built-in harness support should ship with Paperclip's public
distribution.
> - Grok already speaks ACP; it does not require a new public bridge
package.
> - Sandbox provisioning owns the native executable and its pinned
version.
> - The runner must verify that prerequisite without downloading it
during npm installation.
> - This change separates built-in launcher identity from external
runtime identity.
> - Clean npm installation and live staging checks verify the
distribution boundary.

## Linked Issues or Issue Description

Refs #13882, #13973, #13977, #13979.

This follow-up now targets master after #13882 was squash-merged. It
replaces the private `@paperclipai/grok-acp` workspace package with
runner-owned assets. Current master is included so the branch also
contains the merged scheduler, complete-event capture, and durable
cleanup fixes.

## What Changed

- Ship Grok launcher and qualification metadata inside the runner's
compiled output and the public server's vendored runner tree.
- Remove the separate Grok npm package and all package-manager install
hooks for this runtime.
- Require the checksum-verified Grok Build 1.0.13 binary at
`/opt/paperclip/providers/grok/1.0.13/grok` in the selected execution
environment. Provision it explicitly in the Daytona image and CI setup.
- Keep native binaries outside the provider pack. Bind the built-in
launcher into the pack manifest.
- Preserve executable leases, descriptor-backed startup, credential
fences, permissions, and exact ACP model admission.
- Use `builtin:grok-acp` and `native:grok` as profile identities.
Historical package-profile sessions fail closed on resume rather than
being silently reinterpreted.
- Resolve built-in assets from the authenticated sidecar location,
including public server npm layouts. Keep the controller path out of
provider environments.
- Add clean npm tarball installation verification to the existing
trusted canary CI job and the admitted manual EC2 verification path. It
stages a unified release version and runs npm lifecycle scripts, then
verifies missing-prerequisite rejection and admission after separate
provisioning without credentials or inference.
- Include the controller-owned provider pack in stamped Cloud images.
Unstamped local images omit the pack and remain usable; remote ACPX
requires full source provenance.
- Correct CLI approval-page metadata for an already authenticated Cloud
board user; approval authorization remains unchanged.
- Honor explicit native-runner enablement in the Cloud agent picker and
direct setup page, keeping the flag disabled by default.
- Allow selecting the execution environment before connecting
credentials. Include Grok in the existing authenticated hello-probe
flow, targeting its pinned native prerequisite for runner setup.
- Recover an existing subscription sign-in conflict through an explicit
cancel-and-retry action, serialized after cancellation succeeds.
- Preserve the selected ACPX harness before normalizing config fields,
so new Grok agents use the Grok default model.
- Keep the credential-free Cloud provider pack root-owned and readable
after runtime UID remapping; verify manifest and referenced asset access
under an unrelated unprivileged UID during image builds.
- Archive prior failover backups alongside explicitly replaced harness
state, preserving evidence while preventing stale backups from blocking
a fresh replacement.
- Update Daytona image content inputs and contract tests for the
built-in assets and explicit provisioner.
- Document and regression-test the shared `approve-all` default for Grok
setup, saved configuration, and native execution. Explicitly saved
restrictions remain unchanged.

## Verification

Current merge-repair head `df09eb3e1a619430ad8419a0ee9aedd486689b05`
incorporates master `f1a394bd30cb56fb9e479f98b9f50176fe921858` after the
base PR was squash-merged. All 12 conflicts came from incoming files
identical to the tested pre-squash base. The final tree exactly matches
a three-way merge using that original base, preserving built-in Grok
distribution and removal of the obsolete private package. All 252
focused runner/UI tests, six npm-isolation tests, and token gates pass.
Fresh exact-head Greptile review is 5/5 with no outstanding findings;
security scans and EC2 native compilation pass. All current-head CI is
green: 56 successful checks/statuses and four intentional skips ([run
36468768035](https://github.com/paperclipai/paperclip/actions/runs/36468768035)).
The repository owner explicitly authorized bypassing code-owner approval
after all checks passed; no CI checks or repository protection settings
are bypassed or changed. The only remaining PR was removed from the
completed stack metadata to permit native auto-merge.

Earlier integration head `78cb306ecc41b5c96577c26c1d89153b0ef865a1`
includes master `3447609d2247e75e55d91493dda91a608364f672` (2026-09-28).
Two master advances during verification overlapped the eval catalog; the
final merge preserves Grok qualification, completion updates, and
bounded API-response reading in all 348 cells. All 77 focused
catalog/eval/workflow tests pass. Both native stack layers (#14397) are
mergeable, and both exact-head Greptile reviews are 5/5 with successful
security scans and no unresolved review threads. All current-head CI is
green: 56 successful checks/statuses and four intentional skips ([CI
attempts](https://github.com/paperclipai/paperclip/actions/runs/36447124691)).
The initial attempt lost two EC2 runners to shutdown signals and stalled
a third shard during dependency preparation; all three passed the
same-commit failed-job-only retry. Trunk code-owner requirements remain
enforced. The review summary’s non-blocking saved-asset offset
classification note concerns code already merged in #14301; those
runtime files are identical to master and outside this stack’s diff.
Historical live evidence below retains its original source revisions.
[Final public npm
verification](https://github.com/paperclipai/paperclip/actions/runs/36445542764)
passed on `76ea70cd4d13786a042af9df82f0fd7a8c85ae30`: 17 public
packages, an executed offline lifecycle sentinel, unchanged consumer
lock, built-in launcher, missing-prerequisite rejection, and verified
separately provisioned binary/command lease. Provisioning and cleanup
require no host privilege elevation; only the positive probe mounts the
temporary native binary read-only. The verifier is unchanged by the
final master merge. All six isolation tests and an offline npm smoke
test pass. The prior head had 56 green CI checks and a 5/5 review after
two unchanged tests timed out and passed a failed-job-only retry ([CI
attempts](https://github.com/paperclipai/paperclip/actions/runs/36444597313)).
All 56 recovery-display/lineage tests pass; re-review cleared the
already-covered missed-retry concern. Earlier EC2 failures remain
retained: [npm lockfile
rejection](https://github.com/paperclipai/paperclip/actions/runs/36436311203),
[missing compiler in the slim
image](https://github.com/paperclipai/paperclip/actions/runs/36440210984),
and the aggregate 15-minute test timeouts in those broad runs. Both
broad attempts passed typecheck, token gates, Product E2E type/unit
checks and build. The focused EC2 lane preserves the existing
trusted-actor and immutable-source gates.


Earlier documentation/test checkpoint
`ff244c4fd78a7ede5a3e00efe09f475f133ef33e` leaves runtime behavior
unchanged. 154 focused tests pass across configuration building, native
provider resolution, permission policy, credentials, UI configuration,
and new-agent setup (including both Grok auth modes); token gates pass.
All fresh CI is green for this head: 56 successful checks/statuses and
two intentional skips ([run
36367065119](https://github.com/paperclipai/paperclip/actions/runs/36367065119)).
Greptile is 5/5 with no new findings. Grok already inherits the shared
`approve-all` default, so unattended setup requires no manual permission
change.

Runtime head `bb5a9307991f1ac567b781970ef11b39d518e19b` fixes a final
staging continuation failure before provider startup: explicit
replacement archived the old harness but left its failover backups
active, which caused `runner_harness_state_mismatch`. The regression
fails before the fix and passes after it; all eight adjacent
recovery-safety cases also pass. Old backups remain inspectable inside
the continuity archive. All fresh CI is green at this head ([run
36360839248](https://github.com/paperclipai/paperclip/actions/runs/36360839248)),
with a 5/5 review. One unrelated Cursor test timed out in the initial
server shard; the same-commit failed-job rerun passed, and both attempts
are retained. Staging deployment is confirmed healthy on this revision.
The controller image is
`ghcr.io/paperclipai/paperclip@sha256:6ad91c487910ccd2596ff7aed0a3a3ea5233d12b51b83cd6e1402237749b9673`.
The final browser-created staging task passed on this exact revision
with API authentication: context read → structured human question →
controller restart → answer submission → same native provider session
resumed → document saved → task Done. The two turns took approximately
119s and 77s. The actual write receipt was applied, and the saved
document has exactly one revision containing the selected answer and
requested marker. Usage and cost were not reported. [Controller image
build](https://github.com/paperclipai/paperclip/actions/runs/36360889243).

- Previous integration head `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`:
all CI green (53 successful checks/statuses, two intentional skips),
including repository typecheck/build/tests, native Runner tests, browser
shards, and canary installation checks. [CI run
36358672529](https://github.com/paperclipai/paperclip/actions/runs/36358672529).
Greptile is 5/5 with no unresolved findings.
- Focused checks cover Grok credentials, executable admission, launcher
assets, provider-pack paths/permissions, workflow contracts, setup
defaults, CLI authorization, and subscription conflict recovery. All 39
protocol definitions validate. Final integration checks pass 124
catalog/evidence/cache tests and nine project-form tests; token gates
pass. Some local dependency checks could not load the stale installed
dependency tree; the corresponding fresh EC2 checks pass.
- Clean public npm installation passed on EC2 at
`8b172ebcf8e02e30662d830c00f3961e3bd459ec` ([run
36164964900](https://github.com/paperclipai/paperclip/actions/runs/36164964900)):
17 unified-version packages, lifecycle scripts enabled, built-in
launcher present, no separate Grok package or npm-downloaded binary,
missing prerequisite rejected, separately provisioned native executable
and command lease verified. No credentials or inference were used.
Subsequent changes preserve this npm asset layout.
- The immutable Daytona prerequisite image is
`ghcr.io/paperclipai/paperclip-daytona-runner@sha256:98957d5be0ac774d086b6402b5849e8e6356fec70fb8c09fca6eb4ed6de918e0`,
built from `5a2db471f3ddabe77f9f80e76ed27f996cb97fba`. The previous
Cloud controller image was
`ghcr.io/paperclipai/paperclip@sha256:fd914e1ab1e45f741e8e078ff452d16f082d7ac05f9b4b3506d3a3c64150d204`,
built from `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`; it is superseded
by the latest image above. Its EC2 build verified provider-pack access
under an unrelated unprivileged UID.
- Browser staging at `40f898bc4cba73c1dff4e6344a3983ba0fb247ef` passed
full Grok onboarding with the correct `grok-4.7` model, saved credential
delivery, and pinned Daytona execution. A browser-created task read
context and asked the structured human question. After a controller
restart, answering the persisted question resumed the same native
provider session, saved the requested document, and completed the task.
Actual tool outcomes and durable state agree: one question and one
document revision. The two successful turns took 42.7s and 63.1s; usage
and cost were not reported.
- Restricted policy returned the expected `approval_required` outcome.
Functional staging tests explicitly selected `approve-all`; controller
authorization and governed approvals remain enforced. Temporary board
CLI access was revoked and verified rejected (HTTP 401), and the
disposable onboarding agent was paused.

Failures remain retained: the pre-fix continuation failure (its task
remains blocked; the passing final task is fresh), the original Cloud
provider-pack permission failure, the expected restricted-policy denial,
the superseded npm staging failure, and an earlier monolithic CI
infrastructure timeout. Browser CI exposed a project alias/form race;
the final stack uses master's stronger draft-preservation fix and all
browser shards pass. Historical full subscription/API protocol and
Product rosters retain their original source revisions and do not
qualify this packaging revision. No local Docker or Rust build was used.

## Risks

The branch includes master’s draft-preservation fix for project URL
aliases. It keeps the same project’s edit form mounted and clears prior
data when the project or company changes.

Custom sandboxes and local execution hosts must provision the pinned
binary before Grok starts. Missing, changed, unsupported-platform, and
symlinked executables fail admission. The new builtin profile cannot
resume sessions created with the former private-package profile.
Existing Claude/Codex npm bridge profiles retain their package pins.
Grok restricted modes preserve the selected policy but cannot
automatically admit Paperclip calls: ACP permission metadata does not
independently bind tool authority, so those calls stop with
`approval_required`. New Grok configurations default to `approve-all`,
including API configurations that omit the mode. Existing explicitly
restricted configurations remain restricted; controller authorization
and governed approvals remain enforced.

## Model Used

OpenAI GPT-6 through Codex, with tool use and code execution. The exact
serving model identifier and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-28 14:54:44 -05:00

139 lines
8.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Grok Build native runner
Select **Grok Build** in the native runner provider selector. The stored contract is
`adapterType: "paperclip_runner"` with `provider: "acpx"`, `acpxAgent: "grok"`,
and `model: "grok-4.7"`. Existing `grok_local` agents keep their legacy adapter.
New Grok runner agents default to **Full auto (approve all)**
(`acpxPermissionMode: "approve-all"`) in setup and the configuration form.
API configurations that omit the permission mode use the same default. No
additional permission setting is needed for unattended execution. Explicitly
saved restrictions remain unchanged.
On Cloud, an operator must enable `enableNativeRunner` for the instance before
the new-agent picker or direct setup page offers the native runner.
Grok Build speaks [ACP over stdio](https://github.com/xai-org/grok-build/blob/main/crates/codegen/xai-grok-pager/docs/user-guide/15-agent-mode.md).
The runner owns `grok agent --no-leader stdio` through ACPX, including session
identity, cancellation, recovery and the authenticated Paperclip MCP bridge.
ACP permission requests are approved by the runner under the default full-auto
policy. It does not add `--always-approve`: permission decisions remain under
the selected ACPX policy. Grok's ACP metadata cannot independently establish
Paperclip tool authority, so explicitly selecting `approve-paperclip` or
`approve-reads` returns the approval-required outcome; `deny-all` rejects requests.
Full auto does not bypass Paperclip's company permissions, governed approvals,
or execution-environment boundaries. Isolated ask
rules override project allow rules, and compatible always-approve settings are
locked off. Compatible hook/MCP discovery and shell login capture are disabled.
## Installation and identity
Grok support ships inside the native runner and the public Paperclip server npm
artifact. There is no separate Grok npm package, binary payload, or npm lifecycle
download. The built-in launcher is identified as `builtin:grok-acp` version 1;
its native runtime identity is `native:grok` version 1.0.13. The historical
`agentServerPackage`/`agentRuntimePackage` wire fields carry these identities,
not npm dependencies. Existing npm-backed ACP bridges retain their package pins.
Provision Grok Build 1.0.13 at
`/opt/paperclip/providers/grok/1.0.13/grok` in the selected execution environment.
The sandbox provisioning helper is explicit and is never run by npm:
```sh
sudo node packages/paperclip-runner/scripts/provision-grok.mjs /opt/paperclip/providers/grok/1.0.13/grok
```
The standard Daytona image provisions it separately from the provider pack.
Custom images and local execution hosts must provide the same prerequisite.
The runner verifies the native executable checksum before credential refresh or
ACP startup; a missing prerequisite reports the required path and version.
Only macOS arm64 and Linux x64 are qualified. No ambient `grok` from PATH is used.
ACP must report the requested exact model; mismatches fail closed.
The distribution identity change deliberately rejects resume bindings from the
former private-package profile. Start a fresh session after upgrading that
unreleased profile; do not silently reinterpret its saved identity.
Instructions use Grok ACP session rules. Assigned skills live in the isolated
Grok home. Steering and goals are unsupported. Token and cost values remain
unknown when Grok does not report them; missing usage is not zero usage.
## Authentication
Use the existing Grok company connection/login flow for subscription execution.
An explicitly selected company-secret `XAI_API_KEY` selects paid API execution.
There is no automatic subscription-to-API fallback. Remote execution cannot
borrow the operator's home credentials. Local execution can use the operator's
existing Grok login when no company login has been selected.
Only the selected credential is staged in the private runtime home. An ownership
lease fences concurrent processes. Before ACP startup, an expiring subscription
credential is refreshed through the verified Grok executable’s non-inference
`models` command. This bounded step suppresses output and prevents Grok 1.0.13
from caching a pre-refresh model list. Exact model verification still precedes
any prompt; refresh failure requires reconnecting Grok Build. After the provider exits, refreshed credentials
are copied back through Grok's existing identity and refresh checks. Runtime
credentials, refresh handoffs, and diagnostic logs are excluded from workspace
backups and removed on close. Session history remains available for resume. Host
configuration, other provider credentials and unselected keys are not forwarded
to Grok.
## Evaluation
The private `paperclip-evals` repository maintains `live-acpx-grok-4.7.json` and
`rosters/live-acpx-grok.json`. The roster covers all 39 protocol cases. Its campaign
lane remains disabled pending complete live qualification.
Product E2E exposes `runner-acpx-grok` in core local/Daytona compatibility and
local session integrity. The explicit `grok-qualification` suite covers replies,
planning approvals, structured questions, downloadable project revisions, stop/resume
and continuation after controller restart
in both environments. Run with `--suite grok-qualification`; it is excluded from
scheduled `--all`. Use the canonical Product E2E dashboard and Evalbook reports.
The separate `grok-subscription-qualification` suite covers the same workflows
with the explicit `GROK_AUTH_JSON` fixture credential. It seeds only the disposable
company's private login home and supplies no API key. It does not exercise the
interactive sign-in UI. Authentication mode remains part of the profile identity.
After a subscription upgrade, a fresh `grok login` may be needed if the existing
login still reports the previous entitlement through ACP.
`packages/paperclip-runner/scripts/grok-native-smoke.mjs` records explicit auth,
model/binary identity, MCP outcome, durable resume and restrictive permissions.
Pass `--auth subscription --auth-file /private/path/auth.json --output /private/report.json`
or `--auth api --output /private/report.json` with an explicitly supplied key.
Each attempt is retained; unknown usage and cost are null.
Qualification requires the full protocol roster, selected product workflows,
subscription and API execution locally and on Daytona, and three successful
repetitions of core tool, approval and resume cases. Deterministic tests or a
single successful browser task do not establish that qualification.
## Remote verification
Do not run Docker on a developer laptop when using remote verification. The
`Docker Runner check` workflow offers maintainer-authorized manual EC2 image
builds and broad source checks without provider credentials. It records the
source revision, resolved lock digest and immutable image reference. Paid
Product E2E remains behind the protected default-branch workflow and environment.
The public npm consumer check uses a digest-pinned, unprivileged container with
no checkout or credentials mounted. It downloads dependencies with lifecycle
scripts disabled and freezes the resulting consumer lockfile. It completes the
clean install with offline `npm rebuild`, running the deferred lifecycle hooks
without re-resolving bundled optional dependencies. Networking stays disabled,
the lockfile must remain unchanged, and a sentinel proves scripts actually ran.
The pinned image includes native build tools and local Node headers so dependency
hooks can compile without network access. Both executable admission probes run
in the same isolation. The verification user provisions Grok inside the disposable
test directory without privilege elevation or host `/opt` changes. Only the
positive probe mounts that binary read-only at the canonical sandbox path.
The Cloud application image also carries the controller-owned provider pack and
sets `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH`. Remote ACPX execution verifies
the sandbox against that pack before using it, or stages the matching pack when
needed. The Cloud controller image does not install the native Grok executable;
the selected sandbox image must provide the prerequisite above.
Cloud builds must supply the full source SHA through `PAPERCLIP_BUILD_COMMIT`
to produce that verified pack. Unstamped local Cloud builds still work for other
features, but omit the pack and cannot start remote ACPX sessions.
Reference in New Issue View Git Blame Copy Permalink