mirror of
https://github.com/alphaXiv/OpenResearch.git
synced 2026-11-11 09:59:34 +08:00
Daniel Kim
874e84080c
* Plan gate: read-only git verbs + pipelines into pure consumers + 2>&1
Planning agents read experiment code that lives only on git branches
(git show <ref>:<file>, git ls-tree) and habitually pipe output through
head/grep with a stderr-merge — all of which the gate rejected outright
via its whole-command metachar blacklist. Rework classification to be
per-pipeline-stage:
- segments (;, &&) split into pipe stages; stage 0 must be a read-only
producer (orx read, git read, glue, or pure consumer), later stages
pure consumers (head/tail/grep/wc/cat/sort/uniq/cut/tr/nl/column —
tee/xargs/sed/awk deliberately excluded)
- standalone 2>&1 tokens dropped; any other token carrying > or & (or
backticks/$(/</newlines anywhere) still gates the whole line
- read-only git allowlist: whole-verb reads (status/log/show/diff/grep/
ls-tree/...) with --output and -O/--open-files-in-pager escape-hatch
guards; conditional forms for branch/tag (list-query only), stash
(list/show), remote (bare/show/get-url), worktree (list), reflog;
global flags limited to -P/--no-pager/-C (rejecting -c config
injection); git config excluded wholesale
- command_is_readonly exported for reuse by the upcoming MCP permission
bridge auto-policy
Fail-safe allowlist property preserved: unknown programs, verbs, or
unparseable forms gate the entire command.
* Plan mode: synthesize a plan card when ExitPlanMode never fires + accept-edits option
Observed failure: a plan-mode session ended with the plan as plain chat
text and no ExitPlanMode call — and since only a plan-card answer
switches the resume mode, the user's chat 'yes' resumed still in plan
mode, stranding the session read-only. Root cause of the missing call:
the shared playbook still said 'Never call an interactive
question/elicitation tool — it will hang', guidance that predates the
prompt-card work in #37/#40.
- run_turn tracks (plan_mode, saw_prompt, turn_errored, last_text) and
synthesizes a WirePrompt{kind:plan, synthesized:true} from the final
text when a plan-mode turn ends card-less and error-free; the pure
condition fn is unit-tested
- WirePrompt/ChatPrompt gain ; the UI softens the header to
'Plan mode — ready to proceed?' for synthesized cards
- playbook: interactive tools surface as cards now; in plan mode, always
present the finished plan via ExitPlanMode
- plan card gains 'Approve & accept edits' (resumeMode accept-edits —
synthesize_resume already honors it)
* chore: drop stray npm lockfile (ui uses pnpm)
* Plan mode: MCP permission bridge — desktop-style mid-turn approvals
Headless plan mode had no approval loop: gray-area tools hard-denied
with nobody to ask, WebFetch was denied (verified on claude 2.1.197),
and — worst — ExitPlanMode SELF-APPROVED headless ("User has approved
exiting plan mode", nobody asked), letting the model exit plan mode and
edit files without the user's say.
The fix is a permission bridge on Claude Code's (hidden but functional)
--permission-prompt-tool flag:
claude ── --mcp-config ──▶ orx mcp-gate (stdio MCP server, hidden cmd)
│ HTTP long-poll, 127.0.0.1:<up port>
▼
orx up: POST /api/internal/permissions ─ oneshot ─▶ card (SSE)
user answers /respond ─▶ ResumeAction::Handled ─▶ claude unblocks
- src/commands/mcp_gate.rs: newline JSON-RPC stdio server (initialize /
tools/list / tools/call), concurrent calls, single stdout writer; any
transport failure answers deny — never hangs claude
- ChatHost: request_permission (token gate -> tier-1 auto-policy ->
card + held oneshot, 55-min timeout under MCP_TOOL_TIMEOUT=60min),
settle_permission, per-session drain on turn end/interrupt, drop-guard
so an abandoned request always resolves its card
- auto-policy (pure fn, tested): read-only Bash allow (shared
plan_gate classifier), WebFetch/WebSearch allow, file edits DENY —
with a permission tool configured the CLI delegates plan mode's edit
block to it (verified: an allow creates files mid-plan), so that deny
IS the safety
- plan_gate hook now also matches ExitPlanMode and answers "ask",
routing plan approval to the bridge card instead of self-approval;
without a bridge, ask denies headless — still better than self-approve
- claude.rs: write_mcp_config + flags wired per plan-mode turn
(best-effort; older CLIs silently ignore the flag and degrade to
today's gating), ExitPlanMode tool_use renders as a plain tool row
while the bridge card is authoritative, bridge cards count as
"saw a prompt" for the synthesized-card fallback
- resume_from_prompt: native (mid-turn) branch — permission allow/deny
and plan keep-planning settle the held request in place (Handled);
plan approve interrupts the paused plan turn and resumes via the
proven --resume + mode-flip machinery (respond()'s SendMessage arm
now interrupts first for native cards only)
Empirical grounding (claude 2.1.197 spikes): denials don't fail headless
runs (permission_denials + subtype success); WebFetch denied natively in
plan mode; hook allow bypasses the permission tool (no card spam);
bridge tool timeout = tool error, turn continues; hook "ask" on
ExitPlanMode routes it through the bridge and an allow resumes the same
turn.
* Plan UI: docked approval strip + right-pane plan tab
The plan card scrolls away with the transcript and renders the whole
plan inline. Desktop-style flow instead:
- PlanStrip (new): docked between the thread and the composer while a
plan awaits a decision — "View plan" / "Keep planning" / "Approve &
run" with a split-menu for accept-edits / bypass. Keep-planning
consumes the composer draft as the refinement note, so typing
feedback then clicking the button is the natural gesture. Solid card
floated over the composer's gradient fade; disappears on resolve via
the SSE re-emit.
- Plan tab: "View plan" opens the markdown as a right-pane ClosableTab
(PlanViewDef in the rightTab union, one tab per prompt id, no fetch —
the text is already client-side); file links inside the plan resolve
against the plan's session worktree.
- The inline plan card goes compact while a strip host exists: clamped
to ~9.5em with a fade, "View full plan" link, no buttons (the strip
owns the actions; single source of truth). Without a strip host the
card keeps its full button row.
* Store: stop normalizing claude-code plan sessions to auto
The retired-modes migration runs on every Store::open and still listed
'plan' for claude-code — written when plan mode was dropped (pre-#75).
Re-adding plan to options() without updating it meant picking Plan
silently degraded to Auto on the next request; the composer masked it by
re-sending the mode with every message, but any turn without an override
(API callers, resumes) ran in auto. Caught live in the plan-mode E2E.
* chore: bump version to 0.1.60
OpenResearch CLI (orx)
Run autoresearch on your machine.
orx up starts a local dashboard where you give each research direction its
own agent. Agents work in parallel in isolated git worktrees, propose and edit
experiments, launch runs on GPU compute, and analyze the results — using your
existing tools, accounts, and compute.
Quick start
curl -LsSf https://openresearch.sh/install.sh | sh
orx up
The dashboard opens at http://127.0.0.1:4791. Give the agent a goal — for
example, ask it to reproduce a paper:
/reproduce-paper <paper URL or title> on <compute>
or turn one into an interactive marimo notebook:
/paper-to-marimo <paper URL or title> on <compute>
The dashboard
orx up runs a single local process on 127.0.0.1 — an embedded web UI plus a
JSON/SSE API over a local SQLite store. From there you get:
- Agent chat — a research assistant with full project context, backed by your locally installed harness: Claude Code, Codex, or OpenCode (pick the harness and model in the UI). Ask it to analyze runs, dig into results, edit code, and spin up new experiments.
- The experiment tree — every experiment is a git branch: a runnable snapshot of your code. The root is your baseline; children are variants measured against it, so lineage stays explicit.
- Runs — launch on your compute (Modal, Hugging Face Jobs, Kubernetes, Slurm, any SSH box, your own machine) or OpenResearch managed GPUs, and watch live logs, statuses, git diffs, files, and linked W&B runs stream in.
- Autoresearch — describe a goal and let the agent run autonomously toward it: proposing, launching, and analyzing experiments.
Everything binds to loopback only; nothing on the dashboard's paths leaves your machine except the compute and paper-search calls you initiate.
On a remote machine
Develop from your laptop while the dashboard runs next to your GPUs:
orx up --remote user@host # or an ~/.ssh/config alias; append :PORT for a custom SSH port
This starts orx up on the remote box over SSH, tunnels the port back, and
opens your browser locally. Note the remote server is unauthenticated on that
host's loopback, so other users on the same box can reach it.
Commands
Run orx --help (or orx <command> --help) for full usage. The highlights:
| Area | Commands |
|---|---|
| Dashboard | up |
| Auth | login, logout |
| Projects | projects, explore, project, create-project, env |
| Experiments | experiments, create-experiment, exp status/cmd/run/cancel |
| Runs & evidence | runs, logs, search-logs, artifacts, artifact, wandb, query, chart, report |
| Compute | compute, instance create |
| Literature | lit, paper (alphaXiv full-text search — no login required) |
| Agent integration | install-skills, skill |
| Maintenance | version, update, telemetry |
orx install-skills drops the OpenResearch skill into your local coding agents
(Claude Code, Codex, OpenCode, Cursor) so they can drive orx themselves —
orx login offers this too.
Installing
The install script above fetches the latest prebuilt release (macOS and Linux, x86_64 and arm64) and is the same as:
curl -LsSf https://github.com/alphaXiv/openresearch-cli/releases/latest/download/openresearch-cli-installer.sh | sh
orx update keeps script-installed binaries current; interactive terminals
also get a once-a-day background check with a one-line stderr notice (silence
it with ORX_NO_UPDATE_CHECK=1).
From source
Requires Rust (stable) via rustup. The prebuilt dashboard
UI is committed at ui/dist, so a plain build works:
cargo build --release # binary at target/release/orx
cargo install --path . # or install onto your PATH (~/.cargo/bin)
To hack on the dashboard UI itself (Vite + React, embedded into the binary at build time):
cd ui && pnpm install && pnpm build
Run the tests with cargo test.
Configuration
- API URL — defaults to production (
https://api.openresearch.sh); override with--api-urlorOPENRESEARCH_API_URL. - Credentials —
orx loginopens your browser, mints a personal access token, and stores it at${XDG_CONFIG_HOME:-~/.config}/openresearch/credentials.json(mode0600). Sent asAuthorization: Bearer …on every request.
Usage analytics
orx sends anonymous usage analytics to help prioritize features. It's
opt-out, and the orx up onboarding surfaces the choice on first run.
- Collected: command name, a random per-install UUID, CLI version, OS/arch,
a CI flag, and coarse event labels (e.g. "a run launched on
modal"). - Never collected: code, prompts, file contents or paths, project or experiment ids/names, repo names, tokens, emails — nothing identifying. The install UUID is not tied to your account.
orx telemetry off # persistent, per-machine
orx telemetry status # current state + the anonymous install id
orx <cmd> --no-telemetry # per-run
Events are fire-and-forget on a background task and never block a command.