Files
OpenShell/AGENTS.md
T
John T. Myers b7779bdefa feat(sandbox): integrate OCSF structured logging for sandbox events (#720)
* feat(sandbox): integrate OCSF structured logging for all sandbox events

WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).

- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
  SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
  AppLifecycle)
- Add openshell-ocsf to all Docker build contexts

* fix(scripts): attach provider to all smoke test phases to avoid rate limits

GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.

Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.

* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp

The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:

  15:49:11 sandbox INFO  15:49:11.649 I NET:OPEN ALLOWED ...

Now the shorthand is just the severity + structured content:

  15:49:11 sandbox INFO  I NET:OPEN ALLOWED ...

* refactor(ocsf): replace single-char severity with bracketed labels

Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].

Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.

Before: sandbox INFO  I NET:OPEN ALLOWED ...
After:  sandbox INFO  NET:OPEN ALLOWED ...

Before: sandbox INFO  M NET:OPEN DENIED ...
After:  sandbox INFO  [MED] NET:OPEN DENIED ...

* feat(sandbox): use OCSF level label for structured events in log push

Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:

  sandbox OCSF  NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
  sandbox OCSF  NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
  sandbox INFO  Fetching sandbox policy via gRPC

* fix(sandbox): convert new Landlock path-skip warning to OCSF

PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.

* fix(sandbox): use rolling appender for OCSF JSONL file

Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.

* fix(sandbox): address reviewer warnings for OCSF integration

W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
    class name (NET:OPEN, HTTP:GET) already identifies structured events
    and the LogPushLayer separately sets the level field.

W2: Log a debug message when OCSF_CTX.set() is called a second time
    instead of silently discarding via let _.

W3: Document the boundary between OCSF-migrated events and intentionally
    plain tracing calls (DEBUG/TRACE, transient, internal plumbing).

W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
    (IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
    consistency with the IPv4 bypass rule failure already migrated.

W5: Migrate malformed inference request warn to NetworkActivity with
    ActivityId::Refuse and SeverityId::Medium.

W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
    FORWARD proxy paths) to match the CONNECT deny severity pattern.
    Allows and audits remain Informational.

* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled

The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.

* fix(ocsf): add timestamps to shorthand file layer output

The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.

Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After:  2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...

* fix(docker): touch openshell-ocsf source to invalidate cargo cache

The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.

Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.

* fix(ocsf): add OCSF level prefix to file layer shorthand output

Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.

Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After:  2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...

* fix(ocsf): clean up shorthand formatting for listen and SSH events

- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
  action is empty (e.g., 'NET:LISTEN [INFO]  10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
  from OCSF events to debug!() traces. Only the final verdict
  (accepted/denied) is an OCSF event now, reducing noise from 3 events
  to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.

* docs(observability): update examples with OCSF prefix and formatting fixes

Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.

* docs(agents): add OCSF logging guidance to AGENTS.md

Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.

Also adds openshell-ocsf to the Architecture Overview table.

* fix: remove workflow files accidentally included during rebase

These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.

* docs(observability): use sandbox connect instead of raw SSH

Users access sandboxes via 'openshell sandbox connect', not direct SSH.

* fix(docs): correct settings CLI syntax in OCSF JSON export page

The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.

* fix(e2e): update log assertions for OCSF shorthand format

The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).

* feat(sandbox): convert WebSocket upgrade log calls to OCSF

PR #718 added two log calls for WebSocket upgrade handling:

- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
  This is a significant state change (L7 enforcement drops to raw relay).

- Unsolicited 101 without client Upgrade header → DetectionFinding with
  High severity. A non-compliant upstream sending 101 without a client
  Upgrade request could be attempting to bypass L7 inspection.
2026-04-07 13:01:13 -07:00

11 KiB

Agent Instructions

This file is the primary instruction surface for agents contributing to OpenShell. It is injected into your context on every interaction — keep that in mind when proposing changes to it.

See CONTRIBUTING.md for build instructions, task reference, project structure, and the full agent skills table.

Project Identity

OpenShell is built agent-first. We design systems and use agents to implement them — this is not vibe coding. The product provides safe, sandboxed runtimes for autonomous AI agents, and the project itself is built using the same agent-driven workflows it enables.

Skills

Agent skills live in .agents/skills/. Your harness can discover and load them natively — do not rely on this file for a full inventory. The detailed skills table is in CONTRIBUTING.md (for humans).

Workflow Chains

These pipelines connect skills into end-to-end workflows. Individual skill files don't describe these relationships.

  • Community inflow: triage-issue → create-spike → build-from-issue
    • Triage assesses and classifies community-filed issues. Spike investigates unknowns. Build implements.
  • Internal development: create-spike → build-from-issue
    • Spike explores feasibility, then build executes once state:agent-ready is applied by a human.
  • Security: review-security-issue → fix-security-issue
    • Review produces a severity assessment and remediation plan. Fix implements it. Both require the topic:security label; fix also requires state:agent-ready.
  • Policy iteration: openshell-cli → generate-sandbox-policy
    • CLI manages the sandbox lifecycle; policy generation authors the YAML constraints.

Architecture Overview

Path Components Purpose
crates/openshell-cli/ CLI binary User-facing command-line interface
crates/openshell-server/ Gateway server Control-plane API, sandbox lifecycle, auth boundary
crates/openshell-sandbox/ Sandbox runtime Container supervision, policy-enforced egress routing
crates/openshell-policy/ Policy engine Filesystem, network, process, and inference constraints
crates/openshell-router/ Privacy router Privacy-aware LLM routing
crates/openshell-bootstrap/ Cluster bootstrap K3s cluster setup, image loading, mTLS PKI
crates/openshell-ocsf/ OCSF logging OCSF v1.7.0 event types, builders, shorthand/JSONL formatters, tracing layers
crates/openshell-core/ Shared core Common types, configuration, error handling
crates/openshell-providers/ Provider management Credential provider backends
crates/openshell-tui/ Terminal UI Ratatui-based dashboard for monitoring
python/openshell/ Python SDK Python bindings and CLI packaging
proto/ Protobuf definitions gRPC service contracts
deploy/ Docker, Helm, K8s Dockerfiles, Helm chart, manifests
.agents/skills/ Agent skills Workflow automation for development
.agents/agents/ Agent personas Sub-agent definitions (e.g., reviewer, doc writer)
architecture/ Architecture docs Design decisions and component documentation

Vouch System

  • First-time external contributors must be vouched before their PRs are accepted. The vouch-check workflow auto-closes PRs from unvouched users.
  • Org members and collaborators bypass the vouch gate automatically.
  • Maintainers vouch users by commenting /vouch on a Vouch Request discussion. The vouch-command workflow appends the username to .github/VOUCHED.td.
  • Skills that create PRs (create-github-pr, build-from-issue) should note this requirement when operating on behalf of external contributors.

Issue and PR Conventions

  • Bug reports must include an agent diagnostic section — proof that the reporter's agent investigated the issue before filing. See the issue template.
  • Feature requests must include a design proposal, not just a "please build this" request. See the issue template.
  • PRs must follow the PR template structure: Summary, Related Issue, Changes, Testing, Checklist.
  • PRs from unvouched external contributors are automatically closed. See the Vouch System section above.
  • Security vulnerabilities must NOT be filed as GitHub issues. Follow SECURITY.md.
  • Skills that create issues or PRs (create-github-issue, create-github-pr, build-from-issue) should produce output conforming to these templates.

Plans

  • Store plan documents in architecture/plans. This is git ignored so its for easier access for humans. When asked to create Spikes or issues, you can skip to GitHub issues. Only use the plans dir when you aren't writing data somewhere else specific.
  • When asked to write a plan, write it there without asking for the location.

Sandbox Logging (OCSF)

When adding or modifying log emissions in openshell-sandbox, determine whether the event should use OCSF structured logging or plain tracing.

When to use OCSF

Use an OCSF builder + ocsf_emit!() for events that represent observable sandbox behavior visible to operators, security teams, or agents monitoring the sandbox:

  • Network decisions (allow, deny, bypass detection)
  • HTTP/L7 enforcement decisions
  • SSH authentication (accepted, denied, nonce replay)
  • Process lifecycle (start, exit, timeout, signal failure)
  • Security findings (unsafe policy, unavailable controls, replay attacks)
  • Configuration changes (policy load/reload, TLS setup, inference routes, settings)
  • Application lifecycle (supervisor start, SSH server ready)

When to use plain tracing

Use info!(), debug!(), warn!() for internal operational plumbing that doesn't represent a security decision or observable state change:

  • gRPC connection attempts and retries
  • "About to do X" events where the result is logged separately
  • Internal SSH channel state (unknown channel, PTY resize)
  • Zombie process reaping, denial flush telemetry
  • DEBUG/TRACE level diagnostics

Choosing the OCSF event class

Event type Builder When to use
TCP connections, proxy tunnels, bypass NetworkActivityBuilder L4 network decisions, proxy operational events
HTTP requests, L7 enforcement HttpActivityBuilder Per-request method/path decisions
SSH sessions SshActivityBuilder Authentication, channel operations
Process start/stop ProcessActivityBuilder Entrypoint lifecycle, signal failures
Security alerts DetectionFindingBuilder Nonce replay, bypass detection, unsafe policy. Dual-emit with the domain event.
Policy/config changes ConfigStateChangeBuilder Policy load, Landlock apply, TLS setup, inference routes, settings
Supervisor lifecycle AppLifecycleBuilder Sandbox start, SSH server ready/failed

Severity guidelines

Severity When
Informational Allowed connections, successful operations, config loaded
Low DNS failures, non-fatal operational warnings, LOG rule failures
Medium Denied connections, policy violations, deprecated config
High Security findings (nonce replay, Landlock unavailable)
Critical Process timeout kills

Example: adding a new network event

use openshell_ocsf::{
    ocsf_emit, NetworkActivityBuilder, ActivityId, ActionId,
    DispositionId, Endpoint, Process, SeverityId, StatusId,
};

let event = NetworkActivityBuilder::new(crate::ocsf_ctx())
    .activity(ActivityId::Open)
    .action(ActionId::Denied)
    .disposition(DispositionId::Blocked)
    .severity(SeverityId::Medium)
    .status(StatusId::Failure)
    .dst_endpoint(Endpoint::from_domain(&host, port))
    .actor_process(Process::new(&binary, pid))
    .firewall_rule(&policy_name, &engine_type)
    .message(format!("CONNECT denied {host}:{port}"))
    .build();
ocsf_emit!(event);

Key points

  • crate::ocsf_ctx() returns the process-wide SandboxContext. It is always available (falls back to defaults in tests).
  • ocsf_emit!() is non-blocking and cannot panic. It stores the event in a thread-local and emits via tracing::info!().
  • The shorthand layer and JSONL layer extract the event from the thread-local. The shorthand format is derived automatically from the builder fields.
  • For security findings, dual-emit: one domain event (e.g., SshActivityBuilder) AND one DetectionFindingBuilder for the same incident.
  • Never log secrets, credentials, or query parameters in OCSF messages. The OCSF JSONL file may be shipped to external systems.
  • The message field should be a concise, grep-friendly summary. Details go in builder fields (dst_endpoint, firewall_rule, etc.).

Sandbox Infra Changes

  • If you change sandbox infrastructure, ensure mise run sandbox succeeds.

Commits

  • Always use Conventional Commits format for commit messages
  • Format: <type>(<scope>): <description> (scope is optional)
  • Common types: feat, fix, docs, chore, refactor, test, ci, perf
  • Never mention Claude or any AI agent in commits (no author attribution, no Co-Authored-By, no references in commit messages)

Pre-commit

  • Run mise run pre-commit before committing.
  • Install the git hook when working locally: mise generate git-pre-commit --write --task=pre-commit

Testing

  • mise run pre-commit — Lint, format, license headers. Run before every commit.
  • mise run test — Unit test suite. Run after code changes.
  • mise run e2e — End-to-end tests against a running cluster. Run for infrastructure, sandbox, or policy changes.
  • mise run ci — Full local CI (lint + compile/type checks + tests). Run before opening a PR.

Python

  • Always use uv for Python commands (e.g., uv pip install, uv run, uv venv)

Docker

  • Always prefer mise commands over direct docker builds (e.g., mise run docker:build instead of docker build)

Cluster Infrastructure Changes

  • If you change cluster bootstrap infrastructure (e.g., openshell-bootstrap crate, deploy/docker/Dockerfile.images, cluster-entrypoint.sh, cluster-healthcheck.sh, deploy logic in openshell-cli), update the debug-openshell-cluster skill in .agents/skills/debug-openshell-cluster/SKILL.md to reflect those changes.

Documentation

  • When making changes, update the relevant documentation in the architecture/ directory.
  • When changes affect user-facing behavior, also update the relevant pages under docs/.
  • Follow the style guide in docs/CONTRIBUTING.md: active voice, no unnecessary bold, no em dash overuse, no filler introductions.
  • Use the update-docs skill to scan recent commits and draft doc updates.

Security

  • Never commit secrets, API keys, or credentials. If a file looks like it contains secrets (.env, credentials.json, etc.), do not stage it.
  • Do not run destructive operations (force push, hard reset, database drops) without explicit human confirmation.
  • Scope changes to the issue at hand. Do not make unrelated changes in the same branch.