Files
OpenShell/AGENTS.md
T
Max Dubrinsky e6f319c76d feat(sdk): add openshell-sdk crate (#1862)
* feat(sdk): add openshell-sdk crate

Additive extraction of the shared async gRPC client core (transport, TLS,
OIDC single-flight refresh, edge tunnel, high-level sandbox surface, raw
escape hatch) as a new workspace crate. No existing consumers yet; CLI/TUI
migration follows in a separate PR.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* refactor(sdk,core): reuse shared JWT exp decoder for refresh deadlines

Extract the signature-unverified JWT exp decode out of
openshell-core/grpc_client.rs into openshell_core::jwt::parse_exp_secs,
and have the openshell-sdk refresh path reuse it to derive a proactive
refresh deadline from a bearer JWT when the caller does not advertise
expires_at. Addresses review feedback to reuse pre-existing logic rather
than reimplement JWT expiry handling per client.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): harden refresh single-flight and redact tokens in Debug

Addresses review feedback on the openshell-sdk refresh path.

- Make the single-flight cleanup cancellation-safe. The in-flight slot is
  now cleared by the shared refresh computation itself (epoch-guarded)
  rather than the leader's post-await code. Previously, if the leader future
  was dropped (e.g. an FFI caller cancelling its promise) after a follower
  drove the refresh to completion, the completed future was stranded in the
  slot and later refresh_now() calls re-joined it, pinning the client to a
  stale or already-rejected token. Adds a regression test that cancels the
  leader and asserts the next refresh starts a fresh attempt.

- Redact bearer secrets from Debug. RefreshedToken and the oidc
  RefreshTokenInput/RefreshTokenOutput now use manual Debug impls that omit
  the access/refresh token fields via finish_non_exhaustive, matching the
  house style (e.g. SecretResolver, SandboxJwtIssuer). Prevents a stray
  {:?} or a containing struct's derived Debug from writing tokens to logs.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): fail refresh when the new token can't be encoded as metadata

store_bearer now returns an error instead of silently keeping the previous
bearer value. The TokenSource commits the refreshed token to its state
before the client writes it into the interceptor slot, so a silent drop
left the interceptor on the old (expiring) token with no path back to a
refresh. Surfacing the error fails the call loudly instead. Adds a unit
test covering a token that can't be encoded as gRPC metadata.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): refresh OIDC tokens on raw routes and harden rotation

Raw gRPC access never triggered OIDC refresh: a client that only used
raw_grpc/raw_inference kept sending the initial bearer until it expired,
with no proactive or reactive refresh. Add raw_grpc_fresh and
raw_inference_fresh accessors that refresh before returning the client,
plus force_refresh for reactive recovery after an Unauthenticated raw
RPC.

Guard the single-flight refresh commit against a concurrent replace().
The in-flight attempt now records the generation it started from and
skips its write when an external replace() has advanced it, so timer or
callback driven rotation is no longer clobbered by a slower refresh.

Remove TokenSource::snapshot(): it returned an empty string under write
contention and had no consumer on the CLI/TUI path. Tests read committed
state directly instead.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* docs(sdk): rewrite crate README for consumers

Recast the openshell-sdk README as a usable crate README rather than an
RFC excerpt. Drop the Responsibilities/Non-responsibilities/Consumers
scope-boundary sections and the mTLS migration rationale, folding the
useful facts (explicit token, no disk/name resolution, Refresh trait,
SdkError mapping) into the intro, a new Auth and refresh section, and
Public surface. Remove the dead relative RFC link and status-label
prose so the doc renders cleanly wherever it is published.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): address review feedback on OIDC refresh and transport

Apply Drew's review notes on PR #1862:

- Drop unused `rustls-pemfile` dependency and move `tokio-stream` to
  dev-dependencies (only used by tests).
- Guard OIDC `expires_at` against u64 overflow with `saturating_add`.
- Fix stale `#[non_exhaustive]` rationale in `AuthConfig` (the struct
  `Oidc` variant it described as future already ships).
- Stop double-wrapping refresh errors: store the bare refresh-error text
  so the single `SdkError::auth` wrap happens once at await.
- Strip stale CLI porting breadcrumbs from `build_channel` docs, keeping
  the branch table.
- Treat proactive token refresh as best-effort: a transient failure falls
  through to the request instead of failing an RPC whose current token is
  still valid, with a regression test.
- Collapse `exec`'s inline auth retry into the shared `unary` helper.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): preserve transient/terminal distinction in refresh errors

The refresh single-flight collapsed both `RefreshError::Transient` and
`RefreshError::Terminal` into a stringified `SdkError::Auth`, so consumers
(CLI, TUI, future language bindings) had no machine-readable way to tell a
retryable IdP blip from a dead session that needs re-authentication.

Carry the `RefreshError` through the shared outcome (kept `Clone` for
`Shared`) instead of its rendered text, and map it at the await site to a
new `retryable` flag on `SdkError::Auth`. Add `SdkError::auth_retryable`
and a `SdkError::retryable()` accessor; transient refresh failures report
`true`, every other error `false`. Add classification tests.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

---------

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
2026-07-14 22:35:46 -07:00

13 KiB

Agent Instructions

This file is the primary instruction surface for agents contributing to OpenShell. It is injected into your context on every interaction — keep that in mind when proposing changes to it.

See CONTRIBUTING.md for build instructions, task reference, project structure, and the full agent skills table.

Project Identity

OpenShell is built agent-first. We design systems and use agents to implement them — this is not vibe coding. The product provides safe, sandboxed runtimes for autonomous AI agents, and the project itself is built using the same agent-driven workflows it enables.

Skills

Agent skills live in .agents/skills/. Your harness can discover and load them natively — do not rely on this file for a full inventory. The detailed skills table is in CONTRIBUTING.md (for humans).

Workflow Chains

These pipelines connect skills into end-to-end workflows. Individual skill files don't describe these relationships.

  • Community inflow: triage-issue → create-spike → build-from-issue
    • Triage assesses and classifies community-filed issues. Spike investigates unknowns. Build implements.
  • Internal development: create-spike → build-from-issue
    • Spike explores feasibility, then build executes once state:agent-ready is applied by a human.
  • Security: review-security-issue → fix-security-issue
    • Review produces a severity assessment and remediation plan. Fix implements it. Both require the topic:security label; fix also requires state:agent-ready.
  • Policy iteration: openshell-cli → generate-sandbox-policy
    • CLI manages the sandbox lifecycle; policy generation authors the YAML constraints.

Architecture Overview

Path Components Purpose
crates/openshell-cli/ CLI binary User-facing command-line interface
crates/openshell-server/ Gateway server Control-plane API, sandbox lifecycle, auth boundary
crates/openshell-sandbox/ Sandbox runtime Container supervision, policy-enforced egress routing
crates/openshell-policy/ Policy engine Filesystem, network, process, and inference constraints
crates/openshell-router/ Privacy router Privacy-aware LLM routing
crates/openshell-bootstrap/ Gateway metadata Gateway registration metadata, auth token storage, mTLS bundle storage
crates/openshell-ocsf/ OCSF logging OCSF v1.7.0 event types, builders, shorthand/JSONL formatters, tracing layers
crates/openshell-core/ Shared core Common types, configuration, error handling
crates/openshell-sdk/ Shared client SDK Async Rust gateway client (gRPC transport, TLS, OIDC refresh, edge tunnel); consumed by CLI, TUI, and @openshell/sdk
crates/openshell-providers/ Provider management Credential provider backends
crates/openshell-tui/ Terminal UI Ratatui-based dashboard for monitoring
crates/openshell-driver-kubernetes/ Kubernetes compute driver In-process ComputeDriver backend for K8s sandbox pods
crates/openshell-driver-docker/ Docker compute driver In-process ComputeDriver backend for local Docker sandbox containers
crates/openshell-driver-vm/ VM compute driver Standalone libkrun-backed ComputeDriver subprocess (embeds its own rootfs + runtime)
python/openshell/ Python SDK Python bindings and CLI packaging
proto/ Protobuf definitions gRPC service contracts
deploy/ Docker, Helm, K8s Dockerfiles, Helm chart, manifests
docs/ Published docs MDX pages, navigation, and content assets
fern/ Docs site config Fern site config, components, and theme assets
.agents/skills/ Agent skills Workflow automation for development
.agents/agents/ Agent personas Sub-agent definitions (e.g., reviewer, doc writer)
architecture/ Architecture docs Design decisions and component documentation

Vouch System

  • First-time external contributors must be vouched before their PRs are accepted. The vouch-check workflow auto-closes PRs from unvouched users.
  • Org members and collaborators bypass the vouch gate automatically.
  • Maintainers vouch users by commenting /vouch on a Vouch Request discussion. The vouch-command workflow appends the username to .github/VOUCHED.td.
  • Skills that create PRs (create-github-pr, build-from-issue) should note this requirement when operating on behalf of external contributors.

Issue and PR Conventions

  • Bug reports must include an agent diagnostic section — proof that the reporter's agent investigated the issue before filing. See the issue template.
  • Feature requests must include a design proposal, not just a "please build this" request. See the issue template.
  • New features must start as GitHub issues using the feature request template. Open an RFC only after an issue exists; maintainers decide when one is needed and assign RFC numbers from the issue.
  • PRs must follow the PR template structure: Summary, Related Issue, Changes, Testing, Checklist.
  • PRs from unvouched external contributors are automatically closed. See the Vouch System section above.
  • Security vulnerabilities must NOT be filed as GitHub issues. Follow SECURITY.md.
  • Skills that create issues or PRs (create-github-issue, create-github-pr, build-from-issue) should produce output conforming to these templates.

Plans

  • Store plan documents in architecture/plans. This is git ignored so its for easier access for humans. When asked to create Spikes or issues, you can skip to GitHub issues. Only use the plans dir when you aren't writing data somewhere else specific.
  • When asked to write a plan, write it there without asking for the location.

Sandbox Logging (OCSF)

When adding or modifying log emissions in openshell-sandbox, determine whether the event should use OCSF structured logging or plain tracing.

When to use OCSF

Use an OCSF builder + ocsf_emit!() for events that represent observable sandbox behavior visible to operators, security teams, or agents monitoring the sandbox:

  • Network decisions (allow, deny, bypass detection)
  • HTTP/L7 enforcement decisions
  • SSH authentication (accepted, denied, nonce replay)
  • Process lifecycle (start, exit, timeout, signal failure)
  • Security findings (unsafe policy, unavailable controls, replay attacks)
  • Configuration changes (policy load/reload, TLS setup, inference routes, settings)
  • Application lifecycle (supervisor start, SSH server ready)

When to use plain tracing

Use info!(), debug!(), warn!() for internal operational plumbing that doesn't represent a security decision or observable state change:

  • gRPC connection attempts and retries
  • "About to do X" events where the result is logged separately
  • Internal SSH channel state (unknown channel, PTY resize)
  • Zombie process reaping, denial flush telemetry
  • DEBUG/TRACE level diagnostics

Choosing the OCSF event class

Event type Builder When to use
TCP connections, proxy tunnels, bypass NetworkActivityBuilder L4 network decisions, proxy operational events
HTTP requests, L7 enforcement HttpActivityBuilder Per-request method/path decisions
SSH sessions SshActivityBuilder Authentication, channel operations
Process start/stop ProcessActivityBuilder Entrypoint lifecycle, signal failures
Security alerts DetectionFindingBuilder Nonce replay, bypass detection, unsafe policy. Dual-emit with the domain event.
Policy/config changes ConfigStateChangeBuilder Policy load, Landlock apply, TLS setup, inference routes, settings
Supervisor lifecycle AppLifecycleBuilder Sandbox start, SSH server ready/failed

Severity guidelines

Severity When
Informational Allowed connections, successful operations, config loaded
Low DNS failures, non-fatal operational warnings, LOG rule failures
Medium Denied connections, policy violations, deprecated config
High Security findings (nonce replay, Landlock unavailable)
Critical Process timeout kills

Example: adding a new network event

use openshell_ocsf::{
    ocsf_emit, NetworkActivityBuilder, ActivityId, ActionId,
    DispositionId, Endpoint, Process, SeverityId, StatusId,
};

let event = NetworkActivityBuilder::new(crate::ocsf_ctx())
    .activity(ActivityId::Open)
    .action(ActionId::Denied)
    .disposition(DispositionId::Blocked)
    .severity(SeverityId::Medium)
    .status(StatusId::Failure)
    .dst_endpoint(Endpoint::from_domain(&host, port))
    .actor_process(Process::new(&binary, pid))
    .firewall_rule(&policy_name, &engine_type)
    .message(format!("CONNECT denied {host}:{port}"))
    .build();
ocsf_emit!(event);

Key points

  • crate::ocsf_ctx() returns the process-wide SandboxContext. It is always available (falls back to defaults in tests).
  • ocsf_emit!() is non-blocking and cannot panic. It stores the event in a thread-local and emits via tracing::info!().
  • The shorthand layer and JSONL layer extract the event from the thread-local. The shorthand format is derived automatically from the builder fields.
  • For security findings, dual-emit: one domain event (e.g., SshActivityBuilder) AND one DetectionFindingBuilder for the same incident.
  • Never log secrets, credentials, or query parameters in OCSF messages. The OCSF JSONL file may be shipped to external systems.
  • The message field should be a concise, grep-friendly summary. Details go in builder fields (dst_endpoint, firewall_rule, etc.).

Sandbox Infra Changes

  • If you change sandbox infrastructure, ensure the relevant sandbox e2e path succeeds.

Commits

  • Always use Conventional Commits format for commit messages
  • Format: <type>(<scope>): <description> (scope is optional)
  • Common types: feat, fix, docs, chore, refactor, test, ci, perf
  • Sign off on each commit for DCO compliance. Use the --signoff option to git commit to add the Signed-off-by footer to ensure the user's configured email address is used.
  • Never mention Claude or any AI agent in commits (no author attribution, no Co-Authored-By, no references in commit messages)

Pre-commit

  • Run mise run pre-commit before committing.
  • Install the git hook when working locally: mise generate git-pre-commit --write --task=pre-commit

Testing

  • mise run pre-commit — Lint, format, license headers. Run before every commit.
  • mise run test — Unit test suite. Run after code changes.
  • mise run e2e — End-to-end tests against a running gateway. Run for infrastructure, sandbox, or policy changes.
  • mise run ci — Full local CI (lint + compile/type checks + tests). Run before opening a PR.

Python

  • Always use uv for Python commands (e.g., uv pip install, uv run, uv venv)

Docker

  • Always prefer mise commands over direct docker builds (e.g., mise run docker:build instead of docker build)

Cluster Infrastructure Changes

  • If you change gateway deployment infrastructure (e.g., Helm values/templates, gateway image packaging, or deploy logic in openshell-cli), update the debug-openshell-cluster skill in .agents/skills/debug-openshell-cluster/SKILL.md to reflect those changes.

Documentation

  • When making changes, update the relevant documentation in the architecture/ directory.
  • When changes affect user-facing behavior, update the relevant published docs pages under docs/ and navigation in docs/index.yml.
  • When changing gateway TOML fields, driver-specific config options, config defaults, or Helm rendering of gateway.toml, update docs/reference/gateway-config.mdx in the same branch.
  • fern/ contains the Fern site config, components, preview workflow inputs, and publish settings.
  • Follow the docs style guide in docs/CONTRIBUTING.mdx: active voice, minimal formatting, no filler introductions, shell fences for copyable commands, and no duplicate body H1.
  • Fern PR previews run through .github/workflows/branch-docs.yml, and production publish runs through the publish-fern-docs job in .github/workflows/release-tag.yml.
  • Use the update-docs skill to scan recent commits and draft doc updates.

Architecture Docs

  • Architecture docs are short canonical subsystem overviews, not exhaustive implementation notes.
  • Update one of the existing top-level architecture docs before adding a new file.
  • Put useful crate-specific details in the relevant crate README.md.
  • Add a new top-level architecture doc only when explicitly requested or when an RFC-level design needs a stable home.
  • Keep architecture docs focused on stable boundaries, data/control flow, invariants, and operational constraints.
  • Remove stale detail instead of preserving it by default.
  • Do not include testing transcripts, historical debugging notes, long source-file inventories, or field-by-field schema references.
  • Put user-facing instructions in docs/, broad design proposals in rfc/, and temporary plans in ignored architecture/plans/.

Security

  • Never commit secrets, API keys, or credentials. If a file looks like it contains secrets (.env, credentials.json, etc.), do not stage it.
  • Do not run destructive operations (force push, hard reset, database drops) without explicit human confirmation.
  • Scope changes to the issue at hand. Do not make unrelated changes in the same branch.