Onboarding: research-background step; drop telemetry step (#152)

* Remove usage-analytics onboarding step

The first-run walkthrough is now two steps — coding agents → GitHub —
instead of three. GitHub becomes the final step and finishes onboarding
directly via onDone; the data-location disclosure moves onto it.

Telemetry stays opt-out and is controlled from the CLI (`orx telemetry
off` / `--no-telemetry`); it no longer prompts during onboarding. Removes
the now-dead telemetry UI API client exports and rebuilds the embedded
ui/dist bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Add research-background onboarding step

Third onboarding step (optional): a free-text background blurb plus
alphaXiv papers linked via title search (reusing the paper-search infra).
Prefills from any saved profile and saves best-effort on finish, so an
empty or failed save never blocks completing onboarding.

Persisted locally in settings.json via a new GET/POST /api/settings/profile
(`background` + `linkedPapers`), sharing the same locked read-modify-write
as the other settings so it can't clobber install_id/telemetry_disabled.
The blurb is user free-text kept on local disk — never sent in telemetry
and outside the snapshotted data dir. Not yet consumed by the agent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Myles Anderson <mylesanderson@Myless-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Myles Anderson
2026-08-04 16:32:31 -07:00
committed by GitHub
co-authored by Claude Opus 4.8 Myles Anderson
parent c70e387ade
commit 601a201d0e
8 changed files with 480 additions and 254 deletions
+34
View File
@@ -286,6 +286,10 @@ fn router(state: AppState) -> Router {
"/api/settings/telemetry/consent",
post(record_telemetry_consent),
)
.route(
"/api/settings/profile",
get(profile_settings).post(set_profile_settings),
)
.route("/api/settings/ssh", get(ssh_settings))
.route("/api/settings/ssh/preflight", post(ssh_preflight))
.route(
@@ -2110,6 +2114,36 @@ async fn set_telemetry_settings(Json(req): Json<SetTelemetryReq>) -> ApiResult {
.map_err(|e| ApiError::from(anyhow!("telemetry task failed: {e}")))?
}
fn profile_settings_json() -> Value {
let (background, papers) = crate::telemetry::load_profile();
json!({ "background": background, "papers": papers })
}
async fn profile_settings() -> ApiResult {
tokio::task::spawn_blocking(|| Ok(Json(profile_settings_json())))
.await
.map_err(|e| ApiError::from(anyhow!("profile task failed: {e}")))?
}
#[derive(Deserialize)]
struct SetProfileReq {
#[serde(default)]
background: Option<String>,
#[serde(default)]
papers: Vec<crate::telemetry::ProfilePaper>,
}
async fn set_profile_settings(Json(req): Json<SetProfileReq>) -> ApiResult {
tokio::task::spawn_blocking(move || {
let background = req.background.map(|b| b.trim().to_string());
crate::telemetry::set_profile(background, req.papers)
.map_err(|e| ApiError::from(anyhow!("could not save profile: {e}")))?;
Ok(Json(profile_settings_json()))
})
.await
.map_err(|e| ApiError::from(anyhow!("profile task failed: {e}")))?
}
/// Record the consent decision (agree/reject) for the analytics choice — fired
/// once when the user leaves the onboarding step, so every user who sees it is
/// counted, including those who accept the default. Unconditional by design (see
+92 -2
View File
@@ -74,8 +74,12 @@ fn flush_window() -> Duration {
/// Machine-local CLI settings. Lives at `$XDG_CONFIG_HOME/openresearch/
/// settings.json` (the config dir, NOT the R2-snapshotted data dir) so the
/// anonymous install id stays per-install rather than travelling with an agent
/// box's data snapshot. Modeled on `K8sSettings`. Unknown fields on older files
/// parse fine and are dropped on the next save.
/// box's data snapshot. Beyond telemetry, this is the one file for user-level
/// config: the data-dir/compute defaults and the onboarding researcher profile
/// (`background`/`linked_papers`) — the latter user free-text that, like
/// everything here, stays on local disk and is never sent in any event.
/// Modeled on `K8sSettings`. Unknown fields on older files parse fine and are
/// dropped on the next save.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct Settings {
@@ -107,6 +111,22 @@ pub(crate) struct Settings {
/// applied when a launch resolves to that same backend without a flavor.
#[serde(default)]
pub default_flavor: Option<String>,
/// Free-text researcher background captured in onboarding (Settings →
/// profile). Optional; absent/empty = never provided.
#[serde(default)]
pub background: Option<String>,
/// alphaXiv/arXiv papers the user linked to their profile in onboarding.
#[serde(default)]
pub linked_papers: Vec<ProfilePaper>,
}
/// A paper the user linked to their researcher profile.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct ProfilePaper {
pub paper_id: String,
#[serde(default)]
pub title: Option<String>,
}
/// The persisted data-dir choice, if any (non-empty). Read by `store::data_dir()`
@@ -155,6 +175,29 @@ pub(crate) fn set_compute_default(
})
}
/// The persisted researcher profile: the background blurb (non-empty) and the
/// linked papers. Read by the profile settings endpoint.
pub(crate) fn load_profile() -> (Option<String>, Vec<ProfilePaper>) {
let s = load_settings().unwrap_or_default();
// Trim on read too, so a hand-edited whitespace-only blurb reads as absent.
(
s.background.filter(|b| !b.trim().is_empty()),
s.linked_papers,
)
}
/// Set the researcher profile, preserving every other settings field (same
/// `mutate_settings` guarantees as the data dir).
pub(crate) fn set_profile(
background: Option<String>,
papers: Vec<ProfilePaper>,
) -> std::io::Result<()> {
mutate_settings(|s| {
s.background = background.filter(|b| !b.is_empty());
s.linked_papers = papers;
})
}
fn settings_path() -> PathBuf {
crate::config::config_dir().join("settings.json")
}
@@ -875,6 +918,53 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn profile_fields_dont_clobber_siblings() {
// Same single-writer contract: the onboarding profile persists in the
// telemetry-owned settings.json, so a profile write must preserve
// install_id/telemetry_disabled and vice-versa.
let _g = EnvGuard::new(OPT_VARS);
let dir = std::env::temp_dir().join(format!("orx-tel-profile-{}", uuid::Uuid::new_v4()));
std::env::set_var("XDG_CONFIG_HOME", &dir);
set_persisted_disabled(true).unwrap();
let id = install_id().expect("install id");
set_profile(
Some("I study RL".into()),
vec![ProfilePaper {
paper_id: "1706.03762".into(),
title: Some("Attention Is All You Need".into()),
}],
)
.unwrap();
let (background, papers) = load_profile();
assert_eq!(background.as_deref(), Some("I study RL"));
assert_eq!(papers.len(), 1);
assert_eq!(papers[0].paper_id, "1706.03762");
let s = load_settings().expect("settings present");
assert_eq!(
s.telemetry_disabled,
Some(true),
"opt-out survived profile write"
);
assert_eq!(
s.install_id.as_deref(),
Some(id.as_str()),
"install id survived"
);
// A whitespace-only background clears to None; papers can be emptied.
set_profile(Some(" ".into()), vec![]).unwrap();
let (background, papers) = load_profile();
assert!(background.is_none(), "whitespace background cleared");
assert!(papers.is_empty(), "papers cleared");
let s = load_settings().expect("settings present");
assert_eq!(s.telemetry_disabled, Some(true), "opt-out still intact");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn compute_default_roundtrip_preserves_siblings() {
// Same single-writer contract as data_dir: the Compute settings persist
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -9,8 +9,8 @@
html { background: #ffffff; }
@media (prefers-color-scheme: dark) { html { background: #0e0c0c; } }
</style>
<script type="module" crossorigin src="/assets/index-DW2mH5mM.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-DRGpJwBi.css">
<script type="module" crossorigin src="/assets/index-BM9o2ZyK.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-B2G-OkZC.css">
</head>
<body>
<div id="root"></div>
+11 -12
View File
@@ -633,22 +633,21 @@ export const saveGitToken = (token: string) =>
export const removeGitToken = () =>
fetch("/api/settings/git/token", { method: "DELETE" }).then((r) => json<GitSettings>(r));
export interface TelemetrySettings {
/** Whether anonymous usage analytics is currently on. */
enabled: boolean;
/** When off, a short human reason (e.g. "--no-telemetry flag"); null when on. */
reason: string | null;
/** A paper linked to the researcher profile during onboarding. */
export interface LinkedPaper {
paperId: string;
title: string | null;
}
export const getTelemetry = () => get<TelemetrySettings>("/api/settings/telemetry");
/** The local researcher profile captured in onboarding (settings.json). */
export interface Profile {
background: string | null;
papers: LinkedPaper[];
}
export const setTelemetry = (enabled: boolean) =>
post<TelemetrySettings>("/api/settings/telemetry", { enabled });
export const getProfile = () => get<Profile>("/api/settings/profile");
/** Record the consent decision (agree/reject) once, when the user leaves the
* onboarding step — fires unconditionally so opt-outs are counted too. */
export const recordTelemetryConsent = (enabled: boolean) =>
post<{ ok: boolean }>("/api/settings/telemetry/consent", { enabled });
export const setProfile = (body: Profile) => post<Profile>("/api/settings/profile", body);
export type HarnessId = "claude-code" | "codex" | "opencode";
+125 -120
View File
@@ -1,16 +1,17 @@
import { ArrowLeft, ArrowRight, Check, Copy, RefreshCw } from "lucide-react";
import { ArrowLeft, ArrowRight, Check, Copy, RefreshCw, X } from "lucide-react";
import { Wordmark } from "./Wordmark";
import { useEffect, useRef, useState } from "react";
import {
getGitSettings,
getHarnesses,
getTelemetry,
getProfile,
harnessModelLabel,
recordTelemetryConsent,
setTelemetry,
searchPapers,
setProfile,
type GitSettings,
type Harness,
type TelemetrySettings,
type LinkedPaper,
type PaperHit,
} from "../api";
import { GitTokenForm } from "./GitTokenForm";
import { renderNote } from "./agentNote";
@@ -18,23 +19,30 @@ import { onHarnessAuth } from "../events";
const RETRY_COPY = "Couldn't reach orx. Check it's still running, then re-check.";
/** First-run walkthrough: the detected coding agents, then GitHub access, then
* the usage-analytics choice, then hand off to the (empty) projects page.
* Step 1 gates — orx can't chat without a signed-in agent. Step 2 doesn't:
* cloning and pushing work over SSH keys, so a missing token is a hint, not a
* wall. The data-dir choice deliberately lives in Settings → Storage instead:
* the default suits almost everyone, and Settings can also *move* existing
* data, which this flow never could. */
/** First-run walkthrough: the detected coding agents, then GitHub access, then a
* short research-background prompt, then hand off to the (empty) projects page.
* Step 1 gates — orx can't chat without a signed-in agent. Steps 2 and 3 don't:
* cloning/pushing work over SSH keys, and the background (a blurb + linked
* papers) is optional, saved best-effort so it never blocks finishing. The
* data-dir choice lives in Settings → Storage (which can also *move* existing
* data); usage analytics is opt-out via the CLI (`orx telemetry off`). */
export function Onboarding({ onDone }: { onDone: () => void }) {
const [step, setStep] = useState<0 | 1 | 2>(0);
const [harnesses, setHarnesses] = useState<Harness[] | null>(null);
const [git, setGit] = useState<GitSettings | null>(null);
const [telemetry, setTelemetryState] = useState<TelemetrySettings | null>(null);
const [telemetrySaving, setTelemetrySaving] = useState(false);
const [checking, setChecking] = useState(false);
// Per-probe, not one shared flag: a telemetry failure must not put a
// connectivity error on a gate it has nothing to do with — or worse, hide
// the actionable "sign in" hint behind it.
// Step 3 (optional): a free-text background plus any alphaXiv papers linked
// via title search. Prefilled from any saved profile so a replayed
// walkthrough doesn't look empty.
const [background, setBackground] = useState("");
const [papers, setPapers] = useState<LinkedPaper[]>([]);
const [paperQuery, setPaperQuery] = useState("");
const [paperHits, setPaperHits] = useState<PaperHit[]>([]);
const [searchingPapers, setSearchingPapers] = useState(false);
const paperSeq = useRef(0);
// Per-probe, not one shared flag: a git failure must not put a connectivity
// error on the harness gate it has nothing to do with — or worse, hide the
// actionable "sign in" hint behind it.
const [harnessError, setHarnessError] = useState(false);
const [gitError, setGitError] = useState(false);
@@ -60,7 +68,6 @@ export function Onboarding({ onDone }: { onDone: () => void }) {
void Promise.allSettled([
getHarnesses(refresh, retryRejected).then((h) => fresh() && setHarnesses(h)),
getGitSettings().then((g) => fresh() && setGit(g)),
getTelemetry().then((t) => fresh() && setTelemetryState(t)),
])
.then(([harness, git]) => {
if (!fresh()) return;
@@ -95,13 +102,50 @@ export function Onboarding({ onDone }: { onDone: () => void }) {
}),
[],
);
// Prefill from any saved profile — best-effort, never gates the step.
useEffect(() => {
void getProfile()
.then((p) => {
setBackground(p.background ?? "");
setPapers(p.papers);
})
.catch(() => {});
}, []);
// Leaving step 3 → record the final consent decision once (agree or reject),
// so every user who reaches the analytics step is counted, including those who
// accept the default. Default to enabled if the setting hasn't loaded yet
// (that's the default state shown). Best-effort — never block finishing.
const finishOnboarding = () => {
void recordTelemetryConsent(telemetry?.enabled ?? true).catch(() => {});
// Debounced title search; `paperSeq` drops superseded responses.
useEffect(() => {
const q = paperQuery.trim();
if (q.length < 3) {
setPaperHits([]);
setSearchingPapers(false);
return;
}
const seq = ++paperSeq.current;
setSearchingPapers(true);
const t = setTimeout(() => {
searchPapers(q)
.then((res) => seq === paperSeq.current && setPaperHits(res))
.catch(() => seq === paperSeq.current && setPaperHits([]))
.finally(() => seq === paperSeq.current && setSearchingPapers(false));
}, 350);
return () => clearTimeout(t);
}, [paperQuery]);
const addPaper = (h: PaperHit) => {
setPapers((cur) =>
cur.some((p) => p.paperId === h.paperId)
? cur
: [...cur, { paperId: h.paperId, title: cleanPaperTitle(h.title) }],
);
setPaperQuery("");
setPaperHits([]);
};
const removePaper = (id: string) => setPapers((cur) => cur.filter((p) => p.paperId !== id));
// Persist the profile, then finish. Best-effort — an empty or failed save
// must never trap the user on the last step.
const finish = () => {
void setProfile({ background: background.trim() || null, papers }).catch(() => {});
onDone();
};
@@ -202,38 +246,70 @@ export function Onboarding({ onDone }: { onDone: () => void }) {
<div className="onb-eyebrow">
<Wordmark /> · Step 3 of 3
</div>
<h2 className="onb-title">Usage analytics</h2>
<h2 className="onb-title">Tell us about your research</h2>
<p className="onb-sub">
orx can send anonymous usage analytics to help improve the tool. No code, prompts,
file contents, repo names, or project/session identifiers are ever sent — just a
random per-install id, CLI version, OS and architecture, CI flag, coarse install
type, and coarse events such as commands, onboarding completion, project creation,
and chat-session starts.
A sentence or two about what you work on helps orx tailor its research — and link any
alphaXiv papers it should know. All optional; you can skip and add this later.
</p>
<div className="onb-cards">
<TelemetryCard
telemetry={telemetry}
saving={telemetrySaving}
onSavingChange={setTelemetrySaving}
onUpdate={setTelemetryState}
/>
<div className="onb-card">
<textarea
className="onb-textarea"
value={background}
onChange={(e) => setBackground(e.target.value)}
rows={4}
placeholder="e.g. I work on sample-efficient RL for LLM post-training, focused on reward-model-free methods."
/>
<div className="onb-paper-search">
<input
value={paperQuery}
onChange={(e) => setPaperQuery(e.target.value)}
placeholder="Search alphaXiv by title to link a paper…"
/>
{searchingPapers ? (
<div className="onb-card-meta">Searching alphaXiv…</div>
) : paperHits.length > 0 ? (
<div className="onb-paper-results">
{paperHits.map((h) => (
<button key={h.paperId} type="button" onClick={() => addPaper(h)}>
<span className="title">{cleanPaperTitle(h.title)}</span>
<span className="id">{h.paperId}</span>
</button>
))}
</div>
) : null}
</div>
{papers.length > 0 && (
<div className="onb-paper-chips">
{papers.map((p) => (
<span key={p.paperId} className="onb-paper-chip">
<span className="title">{p.title || p.paperId}</span>
<span className="id">{p.paperId}</span>
<button
type="button"
aria-label={`Remove ${p.paperId}`}
onClick={() => removePaper(p.paperId)}
>
<X size={12} />
</button>
</span>
))}
</div>
)}
</div>
</div>
{/* The data dir moved to Settings → Storage; still disclose where
things land so the location isn't a surprise. */}
<p className="onb-aside-text" style={{ marginTop: 12 }}>
Your database, run logs, and artifacts stay on this machine — change where they live
any time in Settings → Storage.
Your background stays on this machine, alongside your database, run logs, and
artifacts — change where they live any time in Settings → Storage.
</p>
<div className="onb-actions">
<button className="btn ghost" onClick={() => setStep(1)}>
<ArrowLeft size={12} /> Back
</button>
<div style={{ flex: 1 }} />
<button
className="btn primary"
onClick={finishOnboarding}
disabled={telemetrySaving}
>
<button className="btn primary" onClick={finish}>
Create your first project <ArrowRight size={13} />
</button>
</div>
@@ -244,6 +320,12 @@ export function Onboarding({ onDone }: { onDone: () => void }) {
);
}
/** Fast-search titles carry scrape cruft: "[1706.03762] Title - arXiv".
* Kept in sync with NewProjectForm's cleanTitle. */
function cleanPaperTitle(title: string): string {
return title.replace(/^\[[^\]]*\]\s*/, "").replace(/\s*[-–|]\s*arXiv\s*$/i, "");
}
/** A shell command plus its own copy button, sharing one border so the button
* reads as part of the command. Each chip owns its "Copied" state. */
function CmdChip({ cmd }: { cmd: string }) {
@@ -397,80 +479,3 @@ function GitCard({
</div>
);
}
function TelemetryCard({
telemetry,
saving,
onSavingChange,
onUpdate,
}: {
telemetry: TelemetrySettings | null;
saving: boolean;
onSavingChange: (saving: boolean) => void;
onUpdate: (t: TelemetrySettings) => void;
}) {
if (telemetry === null) {
return (
<div className="onb-loading">
<span className="spinner" /> Checking analytics…
</div>
);
}
const on = telemetry.enabled;
// A per-run override (e.g. `--no-telemetry`) that isn't the persisted setting:
// the toggle writes the persisted flag, but this run stays off regardless.
const overridden = !on && telemetry.reason !== null && telemetry.reason !== "disabled via `orx telemetry off`";
const choose = (enabled: boolean) => {
if (saving || enabled === on) return;
onSavingChange(true);
void setTelemetry(enabled)
.then(onUpdate)
.catch(() => {})
.finally(() => onSavingChange(false));
};
return (
<div className="onb-card">
<div className="onb-card-head">
<div>
<div className="onb-card-name">Share anonymous usage analytics</div>
<div className="onb-card-meta" style={{ marginTop: 2 }}>
{on
? "On — helps prioritize what to build next."
: overridden
? `Off — ${telemetry.reason}.`
: "Off — you can turn it back on anytime."}
</div>
</div>
<div style={{ display: "flex", gap: 6, flex: "none" }}>
<button
className={`btn ${on ? "primary" : "ghost"}`}
onClick={() => choose(true)}
disabled={saving}
aria-pressed={on}
>
{on ? <Check size={12} strokeWidth={3} /> : null} On
</button>
<button
className={`btn ${!on ? "primary" : "ghost"}`}
onClick={() => choose(false)}
disabled={saving}
aria-pressed={!on}
>
{!on ? <Check size={12} strokeWidth={3} /> : null} Off
</button>
</div>
</div>
<div className="onb-card-meta" style={{ marginTop: 12 }}>
Sent: a random per-install id, CLI version, OS/architecture, CI flag, coarse install type,
and coarse usage events. Never sent: code, prompts, file contents, paths, repo names, or
project/session identifiers. Change anytime in Settings or with{" "}
<code>orx telemetry off</code>.
</div>
{overridden && (
<div className="onb-card-meta" style={{ marginTop: 8 }}>
Note: this run is off because of {telemetry.reason}, which overrides the saved choice.
</div>
)}
</div>
);
}
+98
View File
@@ -2676,6 +2676,104 @@ textarea::placeholder {
color: var(--muted);
line-height: 1.5;
}
/* Step 3 (research background): a free-text blurb plus alphaXiv paper links. */
.onb-textarea {
width: 100%;
resize: vertical;
min-height: 78px;
line-height: 1.5;
font-size: var(--fs-base);
}
.onb-paper-search {
display: flex;
flex-direction: column;
gap: 6px;
margin-top: 12px;
}
.onb-paper-search input {
width: 100%;
}
.onb-paper-results {
display: flex;
flex-direction: column;
border: 1px solid var(--border);
border-radius: var(--radius-md);
max-height: 200px;
overflow-y: auto;
}
.onb-paper-results button {
display: flex;
flex-direction: column;
align-items: flex-start;
gap: 2px;
padding: 8px 10px;
background: none;
border: none;
border-bottom: 1px solid var(--border-variant);
text-align: left;
font: inherit;
color: var(--text);
cursor: pointer;
}
.onb-paper-results button:last-child {
border-bottom: none;
}
.onb-paper-results button:hover {
background: var(--surface);
}
.onb-paper-results .title {
font-size: var(--fs-md);
font-weight: var(--fw-medium);
}
.onb-paper-results .id {
font-family: var(--mono);
font-size: var(--fs-xs);
color: var(--muted);
}
.onb-paper-chips {
display: flex;
flex-wrap: wrap;
gap: 6px;
margin-top: 10px;
}
.onb-paper-chip {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 4px 4px 4px 10px;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--surface);
font-size: var(--fs-sm);
max-width: 100%;
}
.onb-paper-chip .title {
font-weight: var(--fw-medium);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
max-width: 240px;
}
.onb-paper-chip .id {
font-family: var(--mono);
font-size: var(--fs-xs);
color: var(--muted);
}
.onb-paper-chip button {
display: inline-flex;
align-items: center;
justify-content: center;
padding: 2px;
border: none;
background: none;
color: var(--muted);
cursor: pointer;
border-radius: var(--radius-xs);
}
.onb-paper-chip button:hover {
color: var(--text);
background: var(--panel);
}
/* The identity command is long — let it wrap inside the chip rather than
overflow the card. */
.onb-aside .onb-cmd-chip {