mirror of
https://github.com/alphaXiv/OpenResearch.git
synced 2026-11-11 09:59:34 +08:00
* fix(app): detect harnesses in the macOS app by adopting the shell PATH Finder launches OpenResearch.app through launchd, so the bundle starts with PATH=/usr/bin:/bin:/usr/sbin:/sbin and no shell rc ever sourced. Harness detection resolves `claude`/`codex`/`opencode` off that PATH, so a DMG user with a Homebrew, nvm, or npm-global install saw every harness missing -- including an already-signed-in Claude Code session -- while a terminal `orx up` on the same machine detected them all. App mode now probes the user's shell once at startup and installs the result via `local::search_path`, which harness lookup and harness children consult instead of the process PATH. The probe is `$SHELL -ilc`: interactive, because zsh reads .zshrc only for interactive shells and that is where PATH edits live -- a login-only probe missed the ~/.local/bin holding `claude`. It runs an inner `sh -c` so fish reports a colon-separated PATH rather than its own list-valued $PATH, fences the value between per-probe nonce markers so rc-file chatter cannot forge one, and requires an absolute directory in the result. Best-effort throughout: a 5s cap, and every outcome logged. The five detection probes that spawned a harness binary with no env prep (`bin_version`, `claude_accepts_ultracode`, `claude_list_models`, `codex_model_list`, `run_models`) now go through `prepare_env` like `probe_auth` already did. Without that, a node-shebang install resolves via the hydrated PATH but its `--version` fails for want of `node`, and `gate_oauth_version` turns the missing version into `Unknown` -- telling a signed-in user to sign in. Verified end to end: a bundle run under launchd's PATH with an isolated ORX_DATA_DIR reports Claude Code agentReady, and finds a `codex` reachable only through the hydrated PATH -- codex has no fallback drop-location, so that resolution can only have come from the override. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(app): stop the bundle and a CLI install from conflicting Three ways the DMG app and a curl-installed `orx` stepped on each other. `orx delete` takes an exclusive lifecycle lock and refuses when another OpenResearch process holds it -- "Close `orx up`, `orx serve`, and any active runs before trying again." CLI `orx up` holds the matching read lock via `dispatch`, but app mode returns from `main` before `dispatch` runs, so it held nothing: `orx delete` from a CLI install would wipe the store out from under a live app. App mode now takes the same read lock for the life of the process. Agents shell out to `orx`, and `chat::prepare_env` prepends the running executable's directory so they get THIS build. In the bundle that directory held only `OpenResearch`, so `orx` fell through to whatever CLI the user had -- a different version, or nothing at all for a DMG-only user. The bundle now ships `Contents/MacOS/orx` as a symlink to the executable. Verified the symlink survives `codesign --options runtime` + `--verify --strict`. That alias would otherwise open a second dashboard, since a bare `orx` reaches app mode's no-arguments condition. `launched_as_app_bundle` now also requires that the process was invoked under the bundle executable's own name, which keeps the alias a plain CLI while leaving a direct run of the executable path -- the documented way to watch the app's logs -- in GUI mode. macOS `current_exe` reports the launch path symlink and all, so it is canonicalized before the comparison. Verified safe already, and left alone: the app binds an ephemeral port rather than 4791; the store is WAL with a 5s busy timeout; and run supervisors hold a per-run exclusive lock, so a second server recovering the same active run exits instead of double-driving it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(app): address review of the PATH and coexistence work Blocker: `launched_as_app_bundle` had grown real logic -- four distinct outcomes turning on a subtle platform fact -- with no coverage, and being macOS-gated it could not get any, since CI runs ubuntu only. The decision is now the pure `is_bundle_exe_launch(exe, argv0)`, compiled everywhere and table-tested for all four cases; only the process-state read stays gated. Same seam `extract_path` uses. The lifecycle lock moves ahead of the PATH probe. Taking it afterwards left the store unprotected for however long the probe ran -- up to 5s -- which is exactly when an `orx delete` racing app startup would land. A failed `read()` is now logged too; it was the one outcome in this module that could pass silently, and silently is how the app would then run unlocked. `orx delete` blocks on a running app now, so its message says so instead of naming three things the user isn't running. Probe diagnostics split the timeout from the spawn failure and name the shell, since "which shell, and did it hang or fail to start" is the whole question a report needs to answer. `bin_version` gets the `NO_COLOR` guard its siblings already carry: it feeds `parse_version`, an escape-laden version line parses as `None`, and `gate_oauth_version` turns that into `Unknown` -- reproducing the bug this branch exists to fix, via the synced env the probe now inherits. `find_opencode` skips empty PATH components, matching `find_on_path`. Packaging asserts the `orx` symlink survived staging. Every step preserves it today, but a dereferencing copy would fail silently -- the app builds, signs, notarizes and runs, and agents just quietly fall back to the user's own CLI. Docs: corrected the same overstatement in `search_path`'s module doc that DISTRIBUTION.md already fixed, and recorded the known gap that the probe imports only PATH, so `ORX_DATA_DIR`/`XDG_*` from a shell rc reach the CLI but not a Finder-launched app. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(app): import the shell's data and config dirs, not just PATH `ORX_DATA_DIR`, `XDG_DATA_HOME`, and `XDG_CONFIG_HOME` exported from a shell rc reached the CLI but never a Finder-launched app, so the two resolved different directories: the dashboard opened the default database while `orx` on the same machine used the user's, and the lifecycle lock added last commit guarded a file neither of them shared. Same root cause as the PATH bug, one layer down. `search_path` generalizes to `shell_env`: an allowlist (`IMPORTED`) the startup probe reads, NUL-separated so a directory can contain anything but NUL, behind `var()` -- process environment when no override is installed, so nothing changes for a terminal `orx`. `store::env_path` and `config::config_dir` are the only two places that resolve these, and both now go through it. The allowlist stays short on purpose: these are the variables whose divergence makes the app and the CLI act like separate installs, while credentials already reach harness children via `prepare_env`. This moves the lifecycle lock back after the probe, undoing last commit's reordering. The reason is now structural rather than incidental: the lock path is derived from `config_dir()`, so taking it before the probe would lock the default path while the CLI locks the user's -- protecting nothing. A correct lock a few hundred milliseconds late beats an immediate lock on the wrong file. Verified with the app's process environment stripped of both variables while the probe shell exported them: `orx.db` lands in the shell's data dir and `orx.lifecycle.lock` in the shell's config dir, and the HOME-derived defaults are never created. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(app): carry the imported environment into orx child processes Importing the shell's directories fixed the app's own resolution but not its children's, which made things worse rather than better for anyone exporting `ORX_DATA_DIR`. `spawn_detached_supervise` and the harness adapters spawn plain CLI processes; those never probe, so they re-resolved to the default store while the app read the user's. A run launched from the app would have been supervised against a different database than the dashboard was showing — before this branch both landed on the default and at least agreed. `shell_env::export_to` hands the imported variables to a child. `prepare_env` applies it, so an agent's `orx exp run` shares the dashboard's store, and `spawn_detached_supervise` applies it plus PATH, which also reaches the run payload it launches (`localbox` spawns `bash run.sh` with an inherited env, so it was losing the user's PATH entirely — no `python`, no `uv`, no `conda`). `opencode::spawn_agent` had hand-rolled a copy of `prepare_env`; it now calls it rather than growing a third copy of the same fix. Verified with the app's process environment stripped of both variables while the probe shell exported them: a harness child is handed ORX_DATA_DIR and XDG_CONFIG_HOME, and an `orx` run with that environment creates no default store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(cli): print a runnable command in the hints the bundle breaks A DMG user has no `orx` anywhere on PATH, so every hint telling them to run `orx exp wait …` names a command their shell cannot find. Agents are already covered -- `prepare_env` puts the bundle's `orx` first on their PATH -- but a human driving the bundle's CLI is not. `invocation::orx()` resolves the spelling once: plain `orx` when it is on PATH, which is every CLI install and every agent, otherwise this binary's own path, shell-quoted when it needs it. Nothing changes for anyone who installed the CLI. Applied to the three hints the report named, each of which had been copied per backend: the post-launch "Follow it with …" line (nine copies), the missing run-command error (eight), and the outdated-version warning. Both hint bodies move into `invocation`, so the eight backends now share one line each -- a net deletion, and one place to fix the next time the wording moves. `warning_for` takes the spelling as an argument rather than reading process state, keeping its tests deterministic. Not covered: the long tail of `orx login`-style hints in error strings, which still print a literal `orx`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(app): close the environment gaps review found A dashboard-synced `ORX_DATA_DIR`/`XDG_*` silently beat the shell-imported one for harness children: `prepare_env` exported the imported vars, then the synced loop overwrote any whose key was absent from the *process* env -- which in app mode is exactly these. The app read one store while claude and opencode wrote another. The guard now consults `shell_env::var`, so the imported value counts as present. codex escaped this only because it re-pins `ORX_DATA_DIR` after `prepare_env`; claude and opencode have no such pin. Local run payloads were never actually fixed. The previous commit claimed the supervisor passed its environment to `bash run.sh`, but the supervisor does not spawn it -- `localbox::run_job` does, in the launching process, from `LocalJobSpec::env`. A run started from the app still executed the user's script under launchd's PATH, with no python, uv, or conda. The env now carries the imported PATH and directories, and the supervisor comment drops the wrong claim. Three more readers of these same variables were still going to the process env, and two of them made harness detection itself diverge between app and terminal: `harness::xdg_config_home` (whose doc claims to mirror `config::config_dir`) and `opencode_auth_path`, which would report opencode signed out in the app while the CLI saw it signed in. `invocation::resolves_on_path` was the third -- it decides whether a hint can say plain `orx`, and the reader pastes that into the shell whose PATH we imported. Also: `orx login` -- the first hint a fresh DMG user meets -- now resolves like the rest; `export_to` iterates in `IMPORTED` order so the startup log is diffable; and the doc drift is repaired, including the octal-escape invariant that makes the nonce's leading underscore load-bearing, the reopened `orx delete` window, and the scope note listing what still inherits the process environment (`git`, `gh`, `kubectl`, `ssh`, `publish-branch`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(app): allow dead_code for the launch check off macOS `is_bundle_exe_launch` is deliberately un-gated so its tests run on CI's Linux runner, but its only non-test caller is `#[cfg(target_os = "macos")]`, so the Linux bin target sees it as dead. `mod local` carries a blanket `#[allow(dead_code)]`, which is why `shell_env::parse_probe` -- same shape -- did not trip; `mod commands` does not. Verified by rebuilding with every `target_os = "macos"` in `src/` swapped to `"linux"`, which reproduces exactly what the runner cfgs out. Clean, and no other item on this branch is dead off macOS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
783 lines
32 KiB
Rust
783 lines
32 KiB
Rust
//! opencode bootstrap for `orx up` — binary discovery, per-session config +
|
|
//! playbook written into the session's worktree, spawn + health check, and the
|
|
//! shared `AgentHost` handle the axum server holds (`Arc<AgentHost>` in state).
|
|
//!
|
|
//! One opencode serve child **per chat session**, cwd = that session's private
|
|
//! worktree (see `git::ensure_session_worktree`) so parallel agents never share
|
|
//! a checkout. Env is inherited (that's where ANTHROPIC_API_KEY /
|
|
//! OPENROUTER_API_KEY live — opencode auto-detects providers from env).
|
|
//! Children die with `orx up` via `kill_on_drop`.
|
|
|
|
use std::collections::HashMap;
|
|
use std::path::{Path, PathBuf};
|
|
use std::process::Stdio;
|
|
use std::time::{Duration, Instant};
|
|
|
|
use serde::Serialize;
|
|
use serde_json::json;
|
|
use tokio::process::{Child, Command};
|
|
use tokio::sync::Mutex;
|
|
|
|
use crate::error::{anyhow, Result};
|
|
use crate::local::git;
|
|
use crate::local::model::LocalProject;
|
|
use crate::store;
|
|
|
|
/// Playbook path inside the session worktree; opencode re-reads it every turn,
|
|
/// so rewriting the file retargets a running server without a restart.
|
|
const PLAYBOOK_REL: &str = ".openresearch/agent/autoresearch-local.md";
|
|
|
|
const HEALTH_TIMEOUT: Duration = Duration::from_secs(30);
|
|
|
|
/// `opencode` on PATH, else the installer's default drop location.
|
|
pub fn find_opencode() -> Result<PathBuf> {
|
|
if let Some(paths) = crate::local::shell_env::search_path() {
|
|
// An empty component (`PATH=":/usr/bin"`) means cwd — never a place to
|
|
// pick up a binary we are about to execute. Mirrors `find_on_path`.
|
|
for dir in std::env::split_paths(&paths).filter(|dir| !dir.as_os_str().is_empty()) {
|
|
let candidate = dir.join("opencode");
|
|
if candidate.is_file() {
|
|
return Ok(candidate);
|
|
}
|
|
}
|
|
}
|
|
if let Some(home) = dirs::home_dir() {
|
|
let fallback = home.join(".opencode").join("bin").join("opencode");
|
|
if fallback.is_file() {
|
|
return Ok(fallback);
|
|
}
|
|
}
|
|
Err(anyhow!(
|
|
"opencode not found (checked PATH and ~/.opencode/bin/opencode).\n\
|
|
Install it with: curl -fsSL https://opencode.ai/install | bash"
|
|
))
|
|
}
|
|
|
|
/// Ask the OS for a free loopback port (bind :0, read it back, release).
|
|
fn free_port() -> Result<u16> {
|
|
let listener = std::net::TcpListener::bind(("127.0.0.1", 0))
|
|
.map_err(|e| anyhow!("Could not pick a free port: {}", e))?;
|
|
Ok(listener.local_addr()?.port())
|
|
}
|
|
|
|
/// Where the spawned server's stdout/stderr land (startup diagnostics).
|
|
pub fn agent_log_path() -> PathBuf {
|
|
store::data_dir().join("agent-opencode.log")
|
|
}
|
|
|
|
/// Project-local opencode config. Every real permission is pre-approved so a
|
|
/// headless turn never stalls on a TUI prompt; the interactive `question` tool
|
|
/// is denied AND disabled (it would deadlock serve mode — nothing can answer
|
|
/// it), repeated on the default `build` agent because the tool filter is
|
|
/// agent-scoped. `model` only when the user passed `orx up --model`.
|
|
fn opencode_config_json(model: Option<&str>, instructions: &str) -> String {
|
|
let mut cfg = json!({
|
|
"$schema": "https://opencode.ai/config.json",
|
|
"permission": {
|
|
"edit": "allow",
|
|
"bash": "allow",
|
|
"webfetch": "allow",
|
|
"websearch": "allow",
|
|
"read": "allow",
|
|
"glob": "allow",
|
|
"grep": "allow",
|
|
"task": "allow",
|
|
"skill": "allow",
|
|
"lsp": "allow",
|
|
"doom_loop": "allow",
|
|
"external_directory": "allow",
|
|
"question": "deny",
|
|
},
|
|
"tools": { "question": false },
|
|
"agent": {
|
|
"build": { "tools": { "question": false }, "permission": { "question": "deny" } }
|
|
},
|
|
"instructions": [instructions],
|
|
});
|
|
if let Some(model) = model {
|
|
cfg["model"] = json!(model);
|
|
}
|
|
serde_json::to_string_pretty(&cfg).unwrap_or_else(|_| "{}".to_string())
|
|
}
|
|
|
|
/// The local-mode autoresearch playbook: project context + cardinal rules +
|
|
/// the v1 local command surface. Ported from the cloud agent's
|
|
/// `autoresearchMd()`/`projectContextMd()` prompts, adapted for `orx up`
|
|
/// (external backends via `--backend`, analysis via `orx logs`, no
|
|
/// artifacts/query/chart).
|
|
/// The playbook template — a literal, GitHub-readable markdown file. Rendered
|
|
/// by [`playbook_md`]: the leading HTML comment is stripped and `{token}`
|
|
/// placeholders are substituted (project facts, the compute default, and the
|
|
/// skills index).
|
|
const SYSTEM_PROMPT: &str = include_str!("../../SYSTEM_PROMPT.md");
|
|
|
|
fn playbook_md(project: &LocalProject) -> String {
|
|
let all_backends_enabled = true;
|
|
let id = &project.id;
|
|
let name = &project.name;
|
|
let publication_line = if project.github_enabled() {
|
|
"- Source: immutable snapshots from committed local Git revisions\n- GitHub publication: enabled for experiment visibility; never used for compute transport"
|
|
} else {
|
|
"- Source: immutable snapshots from committed local Git revisions\n- GitHub publication: disabled"
|
|
};
|
|
let experiment_publish_clause = "created locally";
|
|
let edit_step = "2. **Edit** in this worktree: check out `<branch>`, change the code, and commit. Do not push just to launch compute; `orx` snapshots the committed revision directly.";
|
|
let compute_contract = "Every backend receives the same immutable archive of the recorded local Git revision; no backend clones a repository or needs GitHub credentials.";
|
|
let baseline = &project.baseline_branch;
|
|
let artifacts = super::files::files_dir(project)
|
|
.to_string_lossy()
|
|
.into_owned();
|
|
let paper_line = project.paper_id.as_deref().map_or(String::new(), |p| {
|
|
format!(
|
|
"- Paper: arXiv {p} (https://arxiv.org/abs/{p}) — the paper this project starts \
|
|
from; `orx paper {p}` fetches its report\n"
|
|
)
|
|
});
|
|
// The default compute target is read fresh on every
|
|
// playbook rewrite, but how soon a rewrite reaches a live agent varies:
|
|
// claude reads it at child spawn, so a rewrite reaches the agent on the next
|
|
// respawn (config change / interrupt / crash), not every turn; codex only on
|
|
// thread start/resume; a live opencode server keeps its playbook until
|
|
// respawn (`AgentHost::ensure` early-returns for a running child). Launch-time
|
|
// resolution in `exp run` stays authoritative either way: the agent is
|
|
// told to OMIT `--backend`, never to echo the default back, so even a
|
|
// stale prompt launches on the current default.
|
|
let configured_compute_default = crate::config::compute_default();
|
|
let compute_default = if all_backends_enabled {
|
|
Some(
|
|
configured_compute_default
|
|
.clone()
|
|
.unwrap_or_else(|| ("local".to_string(), None)),
|
|
)
|
|
} else {
|
|
Some(("local".to_string(), None))
|
|
};
|
|
let compute_default_source = if configured_compute_default.is_some() {
|
|
"the user's configured default"
|
|
} else {
|
|
"the local fallback"
|
|
};
|
|
let compute_bullet = if !all_backends_enabled {
|
|
"- Compute: run immutable snapshots of recorded commits on configured backends.".to_string()
|
|
} else {
|
|
match &compute_default {
|
|
Some((b, f)) => {
|
|
let flavor_part = f
|
|
.as_ref()
|
|
.map_or(String::new(), |f| format!(" (`--flavor {f}`)"));
|
|
format!(
|
|
"- Compute: default target **{b}**{flavor_part} — {compute_default_source}; omit `--backend` \
|
|
on `orx exp run` to launch there. \
|
|
Use another backend only when the user names one (see \"Compute backends\")"
|
|
)
|
|
}
|
|
None => {
|
|
"- Compute: backends — `hf`, `modal`, `k8s`, `ssh`, `slurm`, `ray`, or `local` —\n \
|
|
chosen by the user per run; **there is no default backend** (see \"Compute\n \
|
|
backends\")"
|
|
.to_string()
|
|
}
|
|
}
|
|
};
|
|
let backends_intro = if !all_backends_enabled {
|
|
"`orx exp run` transfers the recorded source snapshot to the selected backend.".to_string()
|
|
} else {
|
|
match &compute_default {
|
|
Some((b, f)) => {
|
|
let flavor_part = f
|
|
.as_ref()
|
|
.map_or(String::new(), |f| format!(" --flavor {f}"));
|
|
// The omit-instruction must match what a bare launch actually
|
|
// needs: with a saved flavor both flags can go; a flavor-required
|
|
// backend without one still needs --flavor; ssh always needs
|
|
// --host. Contradicting the launch validation here sends the
|
|
// agent into a guaranteed-failing command.
|
|
let omit_hint = if f.is_some() {
|
|
"Omit it (and `--flavor`) to use the default.".to_string()
|
|
} else if super::FLAVOR_REQUIRED_BACKENDS.contains(&b.as_str()) {
|
|
"Omit `--backend` to use the default, but still pass `--flavor` — no default \
|
|
flavor is saved."
|
|
.to_string()
|
|
} else {
|
|
"Omit it to use the default.".to_string()
|
|
};
|
|
let mut s = format!(
|
|
"`orx exp run` launches on {compute_default_source} — **{b}{flavor_part}** \
|
|
— when you omit `--backend`. {omit_hint} \
|
|
Deviate only when the user names another backend for the task or this \
|
|
conversation — a connected token for some other backend is NOT a signal to \
|
|
switch."
|
|
);
|
|
if b == "ssh" {
|
|
s.push_str(" (`--host <alias>` is still required on every launch.)");
|
|
}
|
|
s
|
|
}
|
|
None => "`orx exp run` requires an explicit `--backend` — **there is no default**.\n\
|
|
Which backend to use is the user's decision: if the task doesn't name one and\n\
|
|
the user hasn't already picked one in this conversation, ask before launching."
|
|
.to_string(),
|
|
}
|
|
};
|
|
let (run_invocation, run_guidance, compute_guidance) = if all_backends_enabled {
|
|
(
|
|
"`orx exp run <expId> [--backend <hf|modal|k8s|ssh|slurm|openresearch|local>] [flags]`",
|
|
"Launch the node's run. Backend flags, flavors, and sizing: **`orx-compute` skill** (k8s manifest: **`orx-compute-k8s`**).",
|
|
"Before launching on a backend you have not used this session, load the `orx-compute` skill; k8s additionally needs `orx-compute-k8s`.",
|
|
)
|
|
} else {
|
|
(
|
|
"`orx exp run <expId> [--backend <name>] [flags]`",
|
|
"Launch the recorded commit on a configured backend.",
|
|
"Load `orx-compute` for the launch and wait contract.",
|
|
)
|
|
};
|
|
let skills_scope = if all_backends_enabled {
|
|
"backend flags and sizing, the k8s manifest, tree shaping, git recipes, log analysis, and artifact naming"
|
|
} else {
|
|
"local runs, tree shaping, local git recipes, log analysis, and artifact naming"
|
|
};
|
|
let launch_step = if !all_backends_enabled {
|
|
"3. **Launch**: `orx exp run <expId>` using the configured backend."
|
|
} else if compute_default.is_some() {
|
|
"3. **Launch**: `orx exp run <expId>` — omitting `--backend` uses the default\n \
|
|
target (flags the default still needs are listed under \"Compute backends\") —\n \
|
|
or name one explicitly (`--flavor` for hf/modal/ray, `--host` for ssh/slurm; k8s\n \
|
|
reads the committed manifest; local takes no flags)."
|
|
} else {
|
|
"3. **Launch**: `orx exp run <expId> --backend <backend>` (`--flavor` for\n \
|
|
hf/modal/ray, `--host` for ssh/slurm; k8s reads the committed manifest; local\n \
|
|
takes no flags)."
|
|
};
|
|
// The modular skills installed into this session's worktree (see
|
|
// `agent_skills::ensure_session_skills`). Generated from the Local set so
|
|
// the playbook index and the files on disk can never drift.
|
|
let skills_list = super::agent_skills::skills(super::agent_skills::SkillSet::Local)
|
|
.iter()
|
|
.map(|s| format!("- **{}** — {}", s.name, s.description))
|
|
.collect::<Vec<_>>()
|
|
.join("\n");
|
|
let template = SYSTEM_PROMPT
|
|
.split_once("-->\n\n")
|
|
.map(|(_, rest)| rest)
|
|
.unwrap_or(SYSTEM_PROMPT);
|
|
template
|
|
.replace("{name}", name)
|
|
.replace("{id}", id)
|
|
.replace("{publication_line}", publication_line)
|
|
.replace("{experiment_publish_clause}", experiment_publish_clause)
|
|
.replace("{edit_step}", edit_step)
|
|
.replace("{compute_contract}", compute_contract)
|
|
.replace("{baseline}", baseline)
|
|
.replace("{paper_line}", &paper_line)
|
|
.replace("{compute_bullet}", &compute_bullet)
|
|
.replace("{artifacts}", &artifacts)
|
|
.replace("{skills_list}", &skills_list)
|
|
.replace("{launch_step}", launch_step)
|
|
.replace("{backends_intro}", &backends_intro)
|
|
.replace("{run_invocation}", run_invocation)
|
|
.replace("{run_guidance}", run_guidance)
|
|
.replace("{compute_guidance}", compute_guidance)
|
|
.replace("{skills_scope}", skills_scope)
|
|
}
|
|
|
|
/// Keep the files we drop into the checkout out of `git status` / accidental
|
|
/// commits via the local-only `.git/info/exclude` (never touches tracked
|
|
/// files or the repo's own `.gitignore`). Takes the **hub clone** path — its
|
|
/// `.git/info/exclude` is shared by every session worktree (a worktree's own
|
|
/// `.git` is just a pointer file). Best-effort.
|
|
fn exclude_agent_files(hub: &Path) {
|
|
let Ok(git_dir) = git::common_git_dir(hub) else {
|
|
return;
|
|
};
|
|
let path = git_dir.join("info").join("exclude");
|
|
let existing = std::fs::read_to_string(&path).unwrap_or_default();
|
|
let missing: Vec<&str> = [
|
|
"opencode.json",
|
|
".openresearch/",
|
|
".claude/skills/",
|
|
".opencode/skills/",
|
|
".agents/skills/",
|
|
]
|
|
.into_iter()
|
|
.filter(|entry| !existing.lines().any(|l| l.trim() == *entry))
|
|
.collect();
|
|
if missing.is_empty() {
|
|
return;
|
|
}
|
|
let mut block = String::new();
|
|
if !existing.is_empty() && !existing.ends_with('\n') {
|
|
block.push('\n');
|
|
}
|
|
for entry in missing {
|
|
block.push_str(entry);
|
|
block.push('\n');
|
|
}
|
|
let _ = std::fs::OpenOptions::new()
|
|
.create(true)
|
|
.append(true)
|
|
.open(&path)
|
|
.and_then(|mut f| std::io::Write::write_all(&mut f, block.as_bytes()));
|
|
}
|
|
|
|
/// Ensure the project's hub clone and this session's private worktree exist,
|
|
/// and write the autoresearch playbook into the worktree. Every harness
|
|
/// adapter injects this same file (opencode via config `instructions`, Claude
|
|
/// Code via `--append-system-prompt`, Codex via `developerInstructions` —
|
|
/// legacy exec: first-turn context). Returns
|
|
/// `(workdir, playbook)` — the worktree the harness runs in and the playbook
|
|
/// path inside it.
|
|
///
|
|
/// `session_skills_dir` is the harness's worktree-relative native-skills dir
|
|
/// (`.claude/skills`, `.opencode/skills`, `.agents/skills`); when `Some`, the
|
|
/// modular `orx` skills are written there too, fresh alongside the playbook, so
|
|
/// the session's own agent auto-loads them with zero drift.
|
|
pub fn ensure_playbook(
|
|
project: &LocalProject,
|
|
session_id: &str,
|
|
session_skills_dir: Option<&str>,
|
|
) -> Result<(PathBuf, PathBuf)> {
|
|
let workdir = git::ensure_session_worktree(project, session_id)?;
|
|
let playbook = workdir.join(PLAYBOOK_REL);
|
|
if let Some(parent) = playbook.parent() {
|
|
std::fs::create_dir_all(parent)
|
|
.map_err(|e| anyhow!("Could not create {}: {}", parent.display(), e))?;
|
|
}
|
|
std::fs::write(&playbook, playbook_md(project))
|
|
.map_err(|e| anyhow!("Could not write {}: {}", playbook.display(), e))?;
|
|
// Modular skills, written fresh beside the playbook (same freshness
|
|
// semantics) so this session's agent discovers them natively.
|
|
if let Some(dir) = session_skills_dir {
|
|
super::agent_skills::ensure_session_skills(&workdir, dir, project.github_enabled())?;
|
|
// User-uploaded skills land beside the built-ins, same freshness.
|
|
super::user_skills::write_into_session(&workdir, dir, &project.id)?;
|
|
}
|
|
// One shared exclude covers every worktree.
|
|
exclude_agent_files(Path::new(&project.repo_path));
|
|
// The playbook points the agent at the artifacts dir — make sure it exists.
|
|
let _ = super::files::ensure_dir(project);
|
|
Ok((workdir, playbook))
|
|
}
|
|
|
|
/// Write the opencode config + the playbook into the session's worktree
|
|
/// (self-healing via `ensure_session_worktree` if the cache was wiped).
|
|
/// Returns the worktree path plus, when the repo tracks its own
|
|
/// `opencode.json` (which we must never clobber — the agent commits and
|
|
/// pushes from this worktree), the path of our config to pass via
|
|
/// `OPENCODE_CONFIG` instead.
|
|
fn write_agent_files(
|
|
project: &LocalProject,
|
|
model: Option<&str>,
|
|
session_id: &str,
|
|
) -> Result<(PathBuf, Option<PathBuf>)> {
|
|
// Source of truth for the session-skills dir is the harness trait.
|
|
use crate::local::harness::Harness;
|
|
let skills_dir = crate::local::harness::opencode::OpenCode.session_skills_dir();
|
|
let (repo, playbook) = ensure_playbook(project, session_id, skills_dir)?;
|
|
let config_override = if git::is_tracked(&repo, "opencode.json") {
|
|
// Out-of-root config: absolute instructions path (no root to anchor it).
|
|
let path = repo
|
|
.join(".openresearch")
|
|
.join("agent")
|
|
.join("opencode.json");
|
|
std::fs::write(
|
|
&path,
|
|
opencode_config_json(model, &playbook.to_string_lossy()),
|
|
)
|
|
.map_err(|e| anyhow!("Could not write {}: {}", path.display(), e))?;
|
|
Some(path)
|
|
} else {
|
|
std::fs::write(
|
|
repo.join("opencode.json"),
|
|
opencode_config_json(model, PLAYBOOK_REL),
|
|
)
|
|
.map_err(|e| anyhow!("Could not write opencode.json: {}", e))?;
|
|
None
|
|
};
|
|
Ok((repo, config_override))
|
|
}
|
|
|
|
/// Wire status of one serve child for `GET /api/agent/status`.
|
|
#[derive(Debug, Clone, Serialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct AgentStatus {
|
|
pub running: bool,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub port: Option<u16>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub project_id: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub session_id: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub model: Option<String>,
|
|
}
|
|
|
|
struct AgentChild {
|
|
child: Child,
|
|
port: u16,
|
|
project_id: String,
|
|
session_id: String,
|
|
model: Option<String>,
|
|
}
|
|
|
|
impl AgentChild {
|
|
fn status(&self) -> AgentStatus {
|
|
AgentStatus {
|
|
running: true,
|
|
port: Some(self.port),
|
|
project_id: Some(self.project_id.clone()),
|
|
session_id: Some(self.session_id.clone()),
|
|
model: self.model.clone(),
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Poll `/global/health` until opencode answers, watching for early exit.
|
|
async fn wait_healthy(child: &mut Child, port: u16) -> Result<()> {
|
|
let client = reqwest::Client::builder()
|
|
.timeout(Duration::from_secs(2))
|
|
.build()?;
|
|
let url = format!("http://127.0.0.1:{port}/global/health");
|
|
let deadline = Instant::now() + HEALTH_TIMEOUT;
|
|
loop {
|
|
if let Some(status) = child.try_wait()? {
|
|
return Err(anyhow!(
|
|
"opencode exited during startup ({status}); see {}",
|
|
agent_log_path().display()
|
|
));
|
|
}
|
|
if let Ok(resp) = client.get(&url).send().await {
|
|
if resp.status().is_success() {
|
|
return Ok(());
|
|
}
|
|
}
|
|
if Instant::now() >= deadline {
|
|
return Err(anyhow!(
|
|
"opencode did not become healthy on 127.0.0.1:{port} within {}s; see {}",
|
|
HEALTH_TIMEOUT.as_secs(),
|
|
agent_log_path().display()
|
|
));
|
|
}
|
|
tokio::time::sleep(Duration::from_millis(400)).await;
|
|
}
|
|
}
|
|
|
|
/// Spawn `opencode serve` in the session's worktree and wait for it to come
|
|
/// up healthy.
|
|
async fn spawn_agent(
|
|
project: &LocalProject,
|
|
model: Option<&str>,
|
|
session_id: &str,
|
|
) -> Result<AgentChild> {
|
|
let bin = find_opencode()?;
|
|
// The clone/worktree setup inside can hit the network; keep it off the
|
|
// async workers.
|
|
let (repo, config_override) = {
|
|
let (project, model) = (project.clone(), model.map(str::to_string));
|
|
let session = session_id.to_string();
|
|
tokio::task::spawn_blocking(move || write_agent_files(&project, model.as_deref(), &session))
|
|
.await
|
|
.map_err(|e| anyhow!("agent file task failed: {e}"))??
|
|
};
|
|
// Best-effort: the playbook is the real guide; the shim just lets
|
|
// opencode's skill tool surface `orx skill` too.
|
|
if let Err(err) = crate::commands::install_skills::install_opencode_shim().await {
|
|
eprintln!("warning: could not install the orx opencode skill: {err}");
|
|
}
|
|
let port = free_port()?;
|
|
// The data dir may not exist yet (fresh machine, no Store::open before us).
|
|
if let Some(parent) = agent_log_path().parent() {
|
|
std::fs::create_dir_all(parent)
|
|
.map_err(|e| anyhow!("Could not create {}: {}", parent.display(), e))?;
|
|
}
|
|
let log = std::fs::OpenOptions::new()
|
|
.create(true)
|
|
.append(true)
|
|
.open(agent_log_path())
|
|
.map_err(|e| anyhow!("Could not open {}: {}", agent_log_path().display(), e))?;
|
|
|
|
let mut cmd = Command::new(&bin);
|
|
cmd.arg("serve")
|
|
.arg("--port")
|
|
.arg(port.to_string())
|
|
.arg("--hostname")
|
|
.arg("127.0.0.1")
|
|
// Without --print-logs the log file stays empty and startup failures
|
|
// are undiagnosable.
|
|
.arg("--print-logs")
|
|
.current_dir(&repo)
|
|
.stdin(Stdio::null())
|
|
.stdout(Stdio::from(log.try_clone().map_err(|e| anyhow!("{e}"))?))
|
|
.stderr(Stdio::from(log))
|
|
// Dies with `orx up` when the runtime drops the handle (Ctrl-C, exit).
|
|
.kill_on_drop(true);
|
|
// This orx first on PATH (the agent shells out to plain `orx`), the imported
|
|
// shell environment, and the dashboard's Environment tab vars.
|
|
crate::local::chat::prepare_env(&mut cmd);
|
|
// Tag runs the agent launches (`orx exp run`) with this session so they can
|
|
// be explicitly subscribed to. One serve child per session; set after the
|
|
// synced-env loop so it isn't shadowed.
|
|
crate::local::chat::set_chat_session_env(&mut cmd, session_id);
|
|
if let Some(config) = &config_override {
|
|
// The repo tracks its own opencode.json; ours rides OPENCODE_CONFIG.
|
|
// Project configs load after OPENCODE_CONFIG and would override our
|
|
// headless permission grants, so they are disabled for this child.
|
|
cmd.env("OPENCODE_CONFIG", config)
|
|
.env("OPENCODE_DISABLE_PROJECT_CONFIG", "1");
|
|
}
|
|
// Own process group: a terminal SIGINT reaches orx up alone, which then
|
|
// tears the child down deliberately (kill_on_drop / shutdown()).
|
|
#[cfg(unix)]
|
|
cmd.process_group(0);
|
|
|
|
let mut child = cmd
|
|
.spawn()
|
|
.map_err(|e| anyhow!("Could not spawn {}: {}", bin.display(), e))?;
|
|
if let Err(err) = wait_healthy(&mut child, port).await {
|
|
let _ = child.kill().await;
|
|
return Err(err);
|
|
}
|
|
Ok(AgentChild {
|
|
child,
|
|
port,
|
|
project_id: project.id.clone(),
|
|
session_id: session_id.to_string(),
|
|
model: model.map(str::to_string),
|
|
})
|
|
}
|
|
|
|
/// The `orx up` opencode host: one serve child per chat session, keyed by the
|
|
/// orx session id, each running in that session's worktree. Share as
|
|
/// `Arc<AgentHost>` in axum state.
|
|
pub struct AgentHost {
|
|
/// `orx up --model` override, applied to every spawn.
|
|
model_override: Option<String>,
|
|
/// Serializes ensure() spawns (across all sessions — a spawn is seconds,
|
|
/// and one at a time keeps clone/fetch traffic sane). Never taken by
|
|
/// status()/port_for(), and `inner` is never held across a spawn — a slow
|
|
/// clone or health poll must not block status reads or turn replies.
|
|
spawn_lock: Mutex<()>,
|
|
inner: Mutex<HashMap<String, AgentChild>>,
|
|
}
|
|
|
|
impl AgentHost {
|
|
pub fn new(model_override: Option<String>) -> Self {
|
|
Self {
|
|
model_override,
|
|
spawn_lock: Mutex::new(()),
|
|
inner: Mutex::new(HashMap::new()),
|
|
}
|
|
}
|
|
|
|
/// Status of every live child; reaps children that died behind our back.
|
|
pub async fn status(&self) -> Vec<AgentStatus> {
|
|
let mut guard = self.inner.lock().await;
|
|
guard.retain(|_, agent| matches!(agent.child.try_wait(), Ok(None)));
|
|
guard.values().map(AgentChild::status).collect()
|
|
}
|
|
|
|
/// Loopback port of the session's live server (for inline replies/aborts).
|
|
pub async fn port_for(&self, session_id: &str) -> Option<u16> {
|
|
let mut guard = self.inner.lock().await;
|
|
let agent = guard.get_mut(session_id)?;
|
|
if matches!(agent.child.try_wait(), Ok(None)) {
|
|
Some(agent.port)
|
|
} else {
|
|
guard.remove(session_id);
|
|
None
|
|
}
|
|
}
|
|
|
|
/// Spawn (or reuse) the opencode server for this session. Idempotent when
|
|
/// the session's server is already alive; a dead child is replaced.
|
|
pub async fn ensure(&self, project: &LocalProject, session_id: &str) -> Result<AgentStatus> {
|
|
let _spawning = self.spawn_lock.lock().await;
|
|
{
|
|
let mut guard = self.inner.lock().await;
|
|
if let Some(agent) = guard.get_mut(session_id) {
|
|
if agent.project_id == project.id && matches!(agent.child.try_wait(), Ok(None)) {
|
|
return Ok(agent.status());
|
|
}
|
|
}
|
|
if let Some(mut old) = guard.remove(session_id) {
|
|
let _ = old.child.kill().await; // kill() also reaps
|
|
}
|
|
}
|
|
// inner released: status()/port reads keep answering while the spawn
|
|
// (clone/fetch + health poll) is in flight instead of hanging.
|
|
let agent = spawn_agent(project, self.model_override.as_deref(), session_id).await?;
|
|
let status = agent.status();
|
|
self.inner
|
|
.lock()
|
|
.await
|
|
.insert(session_id.to_string(), agent);
|
|
Ok(status)
|
|
}
|
|
|
|
/// Kill and reap one session's child (on session delete). No-op when the
|
|
/// session has none.
|
|
pub async fn kill_session(&self, session_id: &str) {
|
|
if let Some(mut agent) = self.inner.lock().await.remove(session_id) {
|
|
let _ = agent.child.kill().await;
|
|
}
|
|
}
|
|
|
|
/// Kill and reap every child (also happens via kill_on_drop on exit).
|
|
pub async fn shutdown(&self) {
|
|
for (_, mut agent) in self.inner.lock().await.drain() {
|
|
let _ = agent.child.kill().await;
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use crate::local::agent_skills::{self, SkillSet};
|
|
|
|
fn sample_project() -> LocalProject {
|
|
LocalProject {
|
|
id: "proj_test".into(),
|
|
name: "Test Project".into(),
|
|
slug: "test-project".into(),
|
|
github_owner: "acme".into(),
|
|
github_repo: "widget".into(),
|
|
github_sync_enabled: true,
|
|
baseline_branch: "main".into(),
|
|
repo_path: "/tmp/nonexistent".into(),
|
|
run_command: None,
|
|
paper_id: None,
|
|
created_at: 0,
|
|
updated_at: 0,
|
|
}
|
|
}
|
|
|
|
fn sample_playbook() -> String {
|
|
playbook_md(&sample_project())
|
|
}
|
|
|
|
/// The playbook's "## Skills" index must list exactly the Local-set skills,
|
|
/// in order — regenerate-and-compare so it can never freeze out of sync with
|
|
/// `agent_skills::skills` (the same set written into the session worktree).
|
|
#[test]
|
|
fn playbook_skills_index_matches_local_set() {
|
|
let md = sample_playbook();
|
|
let expected: Vec<String> = agent_skills::skills(SkillSet::Local)
|
|
.iter()
|
|
.map(|s| format!("- **{}** — {}", s.name, s.description))
|
|
.collect();
|
|
|
|
// The "## Skills" section body: between the heading and the next `## `.
|
|
let after = md
|
|
.split("## Skills\n")
|
|
.nth(1)
|
|
.expect("no ## Skills section");
|
|
let section = after.split("\n## ").next().unwrap();
|
|
|
|
let listed: Vec<String> = section
|
|
.lines()
|
|
.filter(|l| l.starts_with("- **"))
|
|
.map(str::to_string)
|
|
.collect();
|
|
assert_eq!(
|
|
listed, expected,
|
|
"playbook Skills index drifted from Local set"
|
|
);
|
|
}
|
|
|
|
/// The slimmed playbook keeps its templated conditional logic — the
|
|
/// compute-default branch's placeholders must still resolve (no leftover
|
|
/// `{...}` braces from a botched edit).
|
|
#[test]
|
|
fn playbook_has_no_unresolved_placeholders() {
|
|
let md = sample_playbook();
|
|
// Every current token must be substituted — a typo'd or newly added
|
|
// token that playbook_md doesn't know about fails here.
|
|
for token in [
|
|
"{name}",
|
|
"{id}",
|
|
"{repo}",
|
|
"{baseline}",
|
|
"{paper_line}",
|
|
"{compute_bullet}",
|
|
"{artifacts}",
|
|
"{skills_list}",
|
|
"{launch_step}",
|
|
"{backends_intro}",
|
|
"{run_invocation}",
|
|
"{run_guidance}",
|
|
"{compute_guidance}",
|
|
"{skills_scope}",
|
|
] {
|
|
assert!(!md.contains(token), "unresolved placeholder {token}");
|
|
}
|
|
for retired in ["{files}", "{memory}"] {
|
|
assert!(!md.contains(retired), "retired placeholder {retired}");
|
|
}
|
|
// The template's leading HTML comment (repo-reader documentation) must
|
|
// be stripped — the prompt starts at the title.
|
|
assert!(
|
|
md.starts_with("# OpenResearch local agent"),
|
|
"template comment not stripped"
|
|
);
|
|
assert!(!md.contains("<!--"), "HTML comment leaked into the prompt");
|
|
// Sanity: the slimmed pointers to the modules survived.
|
|
assert!(md.contains("orx-compute"));
|
|
assert!(md.contains("orx-reports"));
|
|
assert!(md.contains("orx-evidence"));
|
|
assert!(!md.contains("## Memory"));
|
|
assert!(!md.contains("User memory"));
|
|
assert!(!md.contains("Project memory"));
|
|
}
|
|
|
|
#[test]
|
|
fn playbook_requires_commits_without_pushes() {
|
|
let mut project = sample_project();
|
|
project.github_owner.clear();
|
|
project.github_repo.clear();
|
|
let md = playbook_md(&project);
|
|
assert!(md.contains("immutable archive"));
|
|
assert!(md.contains("Do not push just to launch compute"));
|
|
assert!(md.contains("orx-compute-k8s"));
|
|
assert!(md.contains("--backend <hf|modal"));
|
|
}
|
|
|
|
#[test]
|
|
fn playbook_explains_opt_in_run_wakeups() {
|
|
let md = sample_playbook();
|
|
assert!(md.contains("`orx exp wake <expId>`"));
|
|
assert!(md.contains("Unregistered runs never wake the chat"));
|
|
assert!(md.contains("cancelled runs never send a wake-up"));
|
|
assert!(!md.contains("OpenResearch injects an `[orx]` message"));
|
|
}
|
|
|
|
#[test]
|
|
fn playbook_requires_clickable_file_references() {
|
|
let md = sample_playbook();
|
|
assert!(md.contains("chip renders as a small pill"));
|
|
assert!(md.contains("every file or"));
|
|
assert!(md.contains("artifact mentioned in prose"));
|
|
assert!(md.contains("never a bare or backticked path"));
|
|
assert!(md.contains("Use repo-relative paths"));
|
|
assert!(md.contains("committed version"));
|
|
assert!(md.contains("Logs are the only evidence channel"));
|
|
assert!(md.contains("run ids come from `orx runs`"));
|
|
assert!(md.contains("Add a short label to name"));
|
|
assert!(md.contains("the claim when useful"));
|
|
assert!(md.contains("<file path=\"artifacts/<relative-path>\" />"));
|
|
assert!(md.contains("Wrong: Saved as"));
|
|
assert!(md.contains("<file path=\"artifacts/figures/result.png\" />"));
|
|
}
|
|
|
|
#[test]
|
|
fn playbook_avoids_ui_navigation() {
|
|
let mut local_only = sample_project();
|
|
local_only.github_owner.clear();
|
|
local_only.github_repo.clear();
|
|
|
|
for md in [sample_playbook(), playbook_md(&local_only)] {
|
|
crate::local::assert_agent_guidance_is_ui_agnostic("playbook", &md);
|
|
}
|
|
}
|
|
}
|