* feat(sdk/go): add Go SDK foundation, types, and sandbox client (A)
Add the Go SDK module with the full API contract and a working sandbox
client as the first vertical slice. All other resource clients are present
as stubs returning Unimplemented errors, to be replaced with real
implementations in subsequent PRs.
Contents:
- Module setup (go.mod, Makefile, mise.toml)
- All domain types (types/ package)
- Full ClientInterface with all sub-client accessors
- Shared infrastructure (errors, auth, gRPC connection, logging)
- Sandbox client with converter and tests (fully functional)
- Stub clients for remaining resources (exec, file, health, provider,
profile, config, refresh, policy, service, ssh, tcp)
Part of the Go SDK decomposition plan (#2270).
Implements #2044.
* fix(sdk/go): address review feedback on PR #2271
- Make scheme parsing drive transport selection: http:// uses plaintext
gRPC, https:// or no scheme uses TLS. Add regression tests.
- Add Resources and DriverConfig fields to SandboxTemplate and update
both converter directions (SandboxFromProto/SandboxSpecToProto).
- Regenerate proto bindings from current canonical proto sources to
eliminate drift (SigV4/MCP fields, params matchers, reserved fields).
- Run gofmt/goimports on all handwritten Go files.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* fix(sdk/go): address principal engineer review findings
- Remove dead boolCount function that would fail golangci-lint (#1)
- Emit EventAdded for the first watch event instead of EventModified,
matching k8s watch semantics (#7)
- Add mutex locking to all mock server methods that access the shared
sandboxes map, fixing latent race conditions (#12)
- Skip HealthCheck integration test that calls an unimplemented stub (#13)
- Scope doc.go examples: mark sections for sub-clients not yet available
in this PR with "available in a future release" (#4)
- Document Config.Timeout/RetryPolicy/Logger and WatchOptions fields
as reserved for future use (#2, #6)
Signed-off-by: Roland Huß <rhuss@redhat.com>
* refactor(sdk/go): migrate mise config to centralized task include
Move Go SDK mise configuration from standalone sdk/go/mise.toml into
the project's centralized pattern:
- Add Go tools (go, golangci-lint, protoc-gen-go, protoc-gen-go-grpc)
to root mise.toml [tools] section
- Create tasks/go.toml with all SDK tasks using go: namespace prefix
and dir=sdk/go for working directory
- Update sdk/go/Makefile to reference namespaced task names
- Update proto:sync default path for monorepo layout
Addresses review feedback from drew on PR #2271 regarding mise
convention alignment.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* refactor(sdk/go): remove UPSTREAM_VERSION standalone repo artifact
Remove sdk/go/proto/UPSTREAM_VERSION file and its exclusion from
proto:check. This was a leftover from the standalone repo prototype.
In a monorepo, proto drift is detectable via git diff between
sdk/go/proto/ and proto/ directly.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* refactor(sdk/go): switch proto generation from protoc to buf
Replace raw protoc invocations with buf for Go SDK proto code generation,
aligning with the TS SDK approach (PR #2122).
- Add repo-level buf.yaml declaring proto/ as the buf module with lint
and breaking change detection config
- Add sdk/go/buf.gen.yaml configuring buf to generate Go code directly
from root proto/ (no more vendored .proto copies)
- Delete vendored .proto source files from sdk/go/proto/
- Rewrite go:proto:gen and go:proto:check mise tasks to use buf
- Remove go:proto:sync and go:proto:clean tasks (no longer needed)
- Add proto target to sdk/go/Makefile
- Add buf 1.72.0 to root mise.toml tool dependencies
- Include options.proto in generation (was stripped from vendored copies)
- Regenerate all .pb.go files via the new buf pipeline
Signed-off-by: Roland Huß <rhuss@redhat.com>
* test(sdk/go): add proto-converter field coverage detection
Use protobuf reflection to enumerate all fields on key proto messages
(SandboxSpec, SandboxTemplate, SandboxStatus, SandboxCondition,
SandboxPolicy) and compare against explicit handled/skipped sets in the
converter tests.
Unhandled fields produce warnings (t.Log), not failures, so proto
contributors are not forced to fix SDK converters in the same PR. Stale
entries in the handled set (removed proto fields) do fail, since they
indicate the converter references something that no longer exists.
A follow-up CI workflow will create GitHub issues when converter drift
lands on main.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* fix(sdk/go): bump Go to 1.26 and fix errcheck lint violations
The upstream go.mod now has `toolchain go1.26.4`, which requires Go 1.26
to build golangci-lint. Bump the mise.toml Go version from 1.25 to 1.26
and wrap deferred Close() calls in test helpers to satisfy errcheck.
Assisted-By: 🤖 Claude Code
* feat(sdk/go): add ObjectMeta fields (annotations, workspace, deletion_timestamp)
Add three new proto ObjectMeta fields to Sandbox and Provider domain
types: Annotations (map), Workspace (string), and DeletionTimestamp
(*time.Time). Update converters in both directions, deep-copy maps at
the proto/SDK boundary, and add TimeFromMillisPtr/MillisFromTimePtr
helper functions.
Assisted-By: 🤖 Claude Code
* chore(sdk/go): regenerate proto bindings after rebase
Pick up workspace fields from upstream PR #2445 (Wire authorization
into workspace model). All request messages now include workspace
parameter in the generated Go bindings.
Assisted-By: 🤖 Claude Code
* feat(sdk/go): add workspace scoping to all RPC interfaces
Add workspace parameter to every sandbox-scoped RPC method across all
interfaces (Sandbox, Exec, File, Service, SSH, TCP, Config, Policy,
Provider, Profile, Refresh). The workspace string is passed as the
second parameter after ctx, following the convention workspace then
resource-name.
Key changes:
- SandboxInterface: all 10 methods gain workspace parameter
- sandbox_client.go: passes Workspace field in every proto request
- ListOptions: add AllWorkspaces field for cross-workspace queries
- All stub interfaces updated to match new signatures
- All sandbox client tests updated with "default" workspace
Assisted-By: 🤖 Claude Code
* chore(sdk/go): remove coverage.out from tracking
Assisted-By: 🤖 Claude Code
* fix(sdk/go): address review feedback from mrunalp
- Add RefreshStrategyAWSStsAssumeRole to match proto enum value 6,
fulfilling the "all domain types upfront" contract
- Wrap context.DeadlineExceeded and context.Canceled in StatusError
so IsDeadlineExceeded() and IsCancelled() helpers work correctly
- Return error from mapToStruct/SandboxSpecToProto instead of silently
discarding structpb.NewStruct failures on invalid template maps
Signed-off-by: Roland Huss <rhuss@redhat.com>
* fix(sdk/go): address remaining review items
- Wire go:ci into root ci task so SDK is tested in repository CI
- Fix gofmt formatting on converter files
- Add goimports to mise.toml tools
- Add coverage.out to .gitignore
- Add Go SDK section to AGENTS.md and CONTRIBUTING.md
- Add regression tests for context-error wrapping (IsDeadlineExceeded,
IsCancelled) and invalid template map rejection
- Remove panic from SandboxToProto, return error instead
Signed-off-by: Roland Huss <rhuss@redhat.com>
* fix(sdk/go): pin goimports version and update lockfile
Pin goimports to 0.48.0 instead of "latest" and regenerate mise.lock
to include the new entry.
Signed-off-by: Roland Huss <rhuss@redhat.com>
* fix(sdk/go): TLS.Insecure means skip-verify, not plaintext
Align TLS.Insecure semantics with the Rust SDK: Insecure: true now
uses TLS with InsecureSkipVerify (skip cert verification) instead of
switching to plaintext. Only the http:// scheme triggers plaintext.
This fixes token auth against dev/k3d gateways: StaticToken and
RefreshableToken require transport security, which real TLS (even
with InsecureSkipVerify) satisfies, but plaintext does not.
For http:// + token auth (dev gateways without TLS), wrap the auth
provider to override RequireTransportSecurity, matching the Rust
SDK's behavior where http:// accepts any auth mode.
Transport decision table (matches Rust SDK crates/openshell-sdk):
http:// + any TLS config -> plaintext (TLS config ignored)
https:// + Insecure: true -> TLS, skip cert verify
https:// + Insecure: false -> TLS, full verification
no scheme -> same as https://
Signed-off-by: Roland Huss <rhuss@redhat.com>
* feat(sdk/go): add missing policy proto fields
Add 6 previously silently dropped fields to the network policy types
and converters, preventing security-relevant data loss on round-trip:
NetworkEndpoint fields 19-23:
- CredentialSigning: SigV4 re-signing mode
- SigningService: AWS service name for SigV4
- SigningRegion: AWS region override for SigV4
- JsonRpcMaxBodyBytes: JSON-RPC body inspection limit
- Mcp: MCP-specific policy options (new McpOptions type)
L7Allow and L7DenyRule field 9:
- Params: MCP params matcher map for tools/call filtering
New type McpOptions with StrictToolNames and AllowAllKnownMcpMethods
optional booleans matching the proto definitions.
Signed-off-by: Roland Huss <rhuss@redhat.com>
* fix(sdk/go): enforce coverage test and extend to policy messages
Change coverage_test.go from t.Logf (silent) to t.Errorf so that
unhandled proto fields fail the test immediately. Add coverage tests
for NetworkEndpoint (23 fields), L7Allow (8 fields), L7DenyRule
(8 fields), and McpOptions (2 fields).
Any new proto field that is not in the handled set or explicitly
skipped now breaks the build, closing the silent-drift gap.
Signed-off-by: Roland Huss <rhuss@redhat.com>
* ci(sdk/go): add Go SDK job to branch-checks workflow
Add a Go SDK job to branch-checks.yml that runs mise run go:ci
(lint, build, test, proto-check, docs-check) on every PR. This
ensures the SDK is tested in CI, not just locally.
Signed-off-by: Roland Huss <rhuss@redhat.com>
* fix(sdk/go): address should-fix review items
#6 Fix broken godoc examples: add workspace parameter to all method
calls in doc.go that were broken after workspace scoping.
#7 Add Err field to Event[T]: Watch error events now carry the
underlying error instead of discarding it.
#8 Separate Unauthenticated from PermissionDenied: add
ErrorUnauthenticated code and IsUnauthenticated() helper. gRPC
Unauthenticated (401) now maps to its own code instead of
collapsing into PermissionDenied (403).
#9 Add Unwrap to StatusError: replace dead Details field with Cause
error field. StatusError.Unwrap() returns Cause, enabling
errors.Is/As unwrapping. FromGRPCError and contextError both
populate Cause.
Signed-off-by: Roland Huss <rhuss@redhat.com>
* ci(sdk/go): add go:format:check to CI pipeline
Add gofmt format verification to go:ci. Catches unformatted Go files
before they reach the PR. Fix formatting on coverage_test.go.
Signed-off-by: Roland Huss <rhuss@redhat.com>
* chore(sdk/go): remove Makefile in favor of mise tasks
All build, lint, test, and proto-gen tasks are already defined in
tasks/go.toml and invoked via mise. The Makefile was a leftover
that duplicated this and raised questions in review.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* feat(sdk/go): sync proto bindings and add credential handle support
Regenerate Go proto bindings after rebase to pick up new
CredentialHandle message and Provider.credential_handles and
profile_workspace fields from upstream. Add domain types, converter
support, and proto field coverage tests for Provider and
CredentialHandle.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* fix(sdk/go): reject plaintext auth leak and fix watch error handling
Reject http:// addresses when the auth provider requires transport
security instead of silently stripping the requirement. Remove the
insecureAuthWrapper that overrode RequireTransportSecurity.
Fix watch stream error handling: use blocking send for terminal
errors so they are never silently dropped when the channel is full,
and wrap mid-stream errors with converter.FromGRPCError so SDK error
helpers like IsUnavailable work on watch Event.Err.
Signed-off-by: Roland Huß <rhuss@redhat.com>
* fix(sdk/go): address review findings from multi-agent code review
- WaitReady now detects SandboxDeleting phase and returns immediately
instead of polling indefinitely
- Watch goroutine defers streamCancel() to prevent context leaks
- Fix StopOnTerminal=false test to keep stream open (was wrong-reason
pass due to stream ending, not StopOnTerminal logic)
- Add EventDeleted test covering the Deleting phase branch
- Add provider converter unit tests for CredentialHandle round-trip,
nil handling, and empty maps
Signed-off-by: Roland Huß <rhuss@redhat.com>
---------
Signed-off-by: Roland Huß <rhuss@redhat.com>
Signed-off-by: Roland Huss <rhuss@redhat.com>
* feat(credentials): add provider credential storage drivers
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(credentials): harden credential update handling
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(credentials): harden credential driver security, correctness, and performance
Address review findings from the credential storage drivers PR:
- Route additional_credentials through the driver on refresh to prevent
silent data loss for multi-credential providers (e.g. AWS STS)
- Clean up stored credential handles on CAS failure during refresh to
prevent orphaned secrets in external backends
- Enforce namespace validation in the Kubernetes Secrets driver to
prevent cross-namespace credential access when allow_reference_namespace
is not enabled
- Cache Vault Kubernetes auth tokens with 80% TTL to avoid re-authenticating
on every credential operation
- Parallelize resolve_credentials in all three drivers using try_join_all
for faster sandbox startup
- Add existingSecret support for the KEK Secret to fix helm template/GitOps
workflows where lookup returns empty and regenerates the key
- Document RBAC blast radius for the Kubernetes Secrets credential driver
and recommend a dedicated namespace
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): add optimistic concurrency, fix thundering herd, parallelize operations
Use resourceVersion optimistic concurrency with retry loop for K8s
Secret ownership checks to prevent TOCTOU races. Switch Vault token
cache from RwLock to Mutex with double-check pattern to prevent
thundering herd on cache miss. Parallelize credential store and delete
operations across independent keys using try_join_all.
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): handle partial failures, add delete retry, consolidate cleanup
Replace try_join_all with join_all in credential store/delete operations
to handle partial failures — successfully-stored handles are cleaned up
when another key fails. Add retry loop with conflict detection to
db-credstore delete_credential, matching the K8s driver pattern.
Consolidate 4 manual cleanup_pre_stored_provider_credentials call sites
into a single error handler using an async block. Remove inconsistent
.trim() from db-credstore validate_handle_owner.
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): fix retry loop guard and remove unprotected validation
Remove attempt-count guard from 409/Aborted match arms in retry loops
so the post-loop Status::aborted error is reachable after exhausting
retries. Previously, last-attempt conflicts fell through to the
catch-all error arm, producing misleading Status::unavailable errors.
Remove duplicate validation calls that ran after
prepare_provider_credential_update but outside the cleanup-protected
async block, which would leak pre-stored handles on failure.
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): add workspace/provider UUID to credential backend paths
Include workspace and provider ID in credential backend object paths to ensure
cross-workspace uniqueness and prevent credential collision (GATOR-1806c9be-01).
- Updated credential driver proto to include workspace and provider_id fields
- Modified Vault driver to include workspace/provider_id in managed_secret_path
- Modified Kubernetes Secrets driver to include workspace/provider_id in
credential_owner_id and managed_secret_name
- Updated all credential runtime calls to pass workspace/provider_id
- Updated tests to use the new signatures
This prevents two workspaces sharing the same external credential store from
colliding on provider names, which was a critical security issue (CWE-639).
* fix(credentials): preserve provider-level expiration for handle-backed credentials
Compute effective expiration from both provider and driver values using the
earliest non-zero timestamp and skip expired values before insertion
(GATOR-1806c9be-02).
- Modified resolve_provider_handles to check provider credential_expires_at_ms
- Skip expired credentials during resolution instead of returning them
- Use effective expiration (min of provider and driver) in resolution results
- Fix inference.rs to preserve earliest expiration when merging
This ensures handle-backed credentials respect the same expiration semantics
as inline credentials.
* fix(credentials): stage refresh changes under new handles before validation
Stage credential replacements under new immutable handles instead of reusing
existing handles to prevent overwriting committed values before validation/CAS
(GATOR-1806c9be-03).
- Stage credentials with empty existing_handles map to force new handle creation
- Validate and CAS before the new values are committed to backend storage
- Delete old handles only after successful CAS
- On CAS failure, delete only the newly staged handles
- This prevents CWE-362/CWE-367 race conditions where failed refreshes could
still modify or delete the active credential
The fix ensures that a rejected refresh cannot modify the backend object still
referenced by the committed provider record.
* fix(credentials): add timeouts to credential driver RPCs
Apply configured timeouts to both startup capability negotiation and runtime
RPCs to prevent indefinite hangs (GATOR-1806c9be-05).
- Add DEFAULT_CREDENTIAL_DRIVER_RPC_TIMEOUT_SECS constant (30s)
- Apply timeout to GetCapabilities during startup connection
- Apply timeout to all runtime RPCs (store, delete, resolve)
- Use tokio::time::timeout to bound the entire GetCapabilities operation
during startup, not just the socket connection
- Return contextual deadline errors on timeout
This prevents a faulty or overloaded driver from hanging gateway operations
indefinitely.
* fix(credentials): fix test to use consistent workspace/provider identity
The Kubernetes auth Vault resolve test was constructing a managed path with
test-workspace/test-provider-id but sending default/prov-123 in the request,
causing validation to reject the request (GATOR-18e32351-01).
- Update test to use test-workspace and test-provider-id in the request to
match the logical_path construction
- This ensures the test exercises the intended code path and validates
Kubernetes auth resolution properly
The test now passes and correctly validates identity enforcement.
* fix(credentials): use unique staging ID for refresh to avoid overwrites
Stage refresh replacements under genuinely distinct immutable handles using
a unique staging ID to prevent overwriting committed values (GATOR-1806c9be-03).
- Generate a unique staging ID using UUID for each refresh operation
- Use this staging ID when storing credentials instead of the real provider ID
- Pass the same staging ID during cleanup on failure to delete only staged objects
- This ensures deterministic paths (Vault) and object names (K8s) don't collide
with the committed provider's credentials
The fix prevents failed refreshes from silently replacing active credentials
or breaking providers by deleting still-referenced backend objects.
* fix(credentials): wrap credential driver RPCs in local timeouts
Add local tokio::time::timeout wrappers around credential driver RPCs to
bound non-compliant or stalled UDS peers (GATOR-1806c9be-05).
- Wrap StoreCredential, DeleteCredential, and ResolveCredentials in local timeouts
- Return contextual deadline_exceeded errors when timeouts occur
- Keep existing gRPC timeout metadata for compliant implementations
- GetCapabilities during startup was already wrapped in previous commit
This ensures a faulty local driver cannot hang gateway operations indefinitely,
even if it accepts the connection but never responds to the RPC.
* fix(credentials): preserve ownership for staged refreshes
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* fix(credentials): bound startup capability probe
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* test(provider): authenticate credential handler requests
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* fix(ci): grant actions read to credential driver e2e
Signed-off-by: Seth Jennings <sjenning@redhat.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Taylor Mutch <taylormutch@gmail.com>
Co-authored-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* perf(net): set TCP_NODELAY on all tunnel and proxy TCP hops
The sandbox tunnel added ~44 ms of latency to every small
request/response because no socket in the path disabled Nagle's
algorithm, so sub-MSS writes waited on delayed ACKs at each hop.
Set TCP_NODELAY on every latency-sensitive TCP socket:
- gateway: accepted connections on the public listener (gRPC relay
frames and WS tunnel writes)
- CLI: edge tunnel local accept + underlying WebSocket TCP stream,
insecure TLS connector (tonic's default connector already does
this), and service-forward accepted sockets
- supervisor: direct-tcpip connect into the sandbox netns, TCP relay
target dials, egress proxy accepted connections, and all upstream
CONNECT/HTTP dials (via a new connect_upstream helper)
Setting TCP_NODELAY on connect is best-effort: a failure only costs
latency, so we log and continue rather than fail the connection.
The sandbox SSH transport rides a unix domain socket and gRPC client
channels use tonic defaults (nodelay on), so no change is needed there.
Fixes#2219
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* refactor(net): house TCP_NODELAY helper in a shared net module
Address review feedback on the TCP_NODELAY change:
- Move the shared set-nodelay helper out of supervisor_session into a
new crate-private `net` module in openshell-supervisor-process, so
ssh and supervisor_session no longer reach across modules through a
pub(crate) item.
- Make the best-effort comments at each call site terse and consistent.
No behavior change; the benchmark ladder reproduces the same numbers.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* refactor(net): consolidate TCP_NODELAY helpers into openshell_core::net
Move the best-effort TCP_NODELAY helpers into the shared openshell_core::net
module so every crate dials and configures sockets the same way:
- Add set_tcp_nodelay_best_effort (accepted/existing streams) and
connect_tcp_nodelay_best_effort (dial + set) with unit tests.
- Migrate all call sites in openshell-cli, openshell-server, and the
supervisor crates to the shared helpers.
- Remove the crate-private net module from openshell-supervisor-process.
- Document socket guidance in AGENTS.md (Network Sockets).
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* perf(net): set TCP_NODELAY on exec bridge and metadata server
The gateway-side single-use SSH-over-relay loopback bridge and the
sandbox IMDS metadata server were missed latency-sensitive TCP hops.
Set TCP_NODELAY on the accepted client connection and both russh client
dials of the exec bridge — interactive keystrokes and line-buffered PTY
output are the most tinygram-heavy traffic in the system — and on the
metadata server's accepted connections.
Also log unrecognized MaybeTlsStream variants in the edge tunnel so a
future TLS-backend change surfaces a silent TCP_NODELAY miss instead of
skipping it quietly.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* perf(net): set TCP_NODELAY on openshell-sdk socket paths
The openshell-sdk crate landed on main with its own copies of the CLI's
hand-rolled sockets, which the CLI and TUI are meant to consume. Give
them the same treatment as the CLI equivalents:
- edge_tunnel: the accepted local tunnel connection and the WebSocket's
underlying TCP socket (plain and rustls variants).
- transport: the dial in InsecureTlsConnector, tonic's custom-connector
path.
Only these hand-rolled sockets need it. Tonic's own connector defaults
tcp_nodelay to true and applies it itself, so plain Endpoint::connect
callers were already covered.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
---------
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
Extract common OpenTelemetry provider construction into openshell-otel so
OpenShell services share one OTLP/gRPC export implementation.
The shared crate owns:
- typed exporter setup errors and non-fatal provider enablement;
- endpoint trimming and URI validation before lazy exporter connection;
- fixed and environment-or-default service-name policies;
- service version and caller-supplied resource attributes;
- batch tracer-provider construction; and
- span-only tracing layers that exclude OpenTelemetry exporter callsites.
Migrate the gateway to the shared provider while retaining its configurable
service name, error marking, and tracing test collector. Add the shared crate
to the architecture inventory and document the tracing boundary.
Refs #2507
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Add a published Issue Triage and Lifecycle page and align AGENTS.md,
CONTRIBUTING.md, README.md, the PR template, and the issue-handling
skills on the state:*/agent:* label model.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(sdk): add openshell-sdk crate
Additive extraction of the shared async gRPC client core (transport, TLS,
OIDC single-flight refresh, edge tunnel, high-level sandbox surface, raw
escape hatch) as a new workspace crate. No existing consumers yet; CLI/TUI
migration follows in a separate PR.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* refactor(sdk,core): reuse shared JWT exp decoder for refresh deadlines
Extract the signature-unverified JWT exp decode out of
openshell-core/grpc_client.rs into openshell_core::jwt::parse_exp_secs,
and have the openshell-sdk refresh path reuse it to derive a proactive
refresh deadline from a bearer JWT when the caller does not advertise
expires_at. Addresses review feedback to reuse pre-existing logic rather
than reimplement JWT expiry handling per client.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* fix(sdk): harden refresh single-flight and redact tokens in Debug
Addresses review feedback on the openshell-sdk refresh path.
- Make the single-flight cleanup cancellation-safe. The in-flight slot is
now cleared by the shared refresh computation itself (epoch-guarded)
rather than the leader's post-await code. Previously, if the leader future
was dropped (e.g. an FFI caller cancelling its promise) after a follower
drove the refresh to completion, the completed future was stranded in the
slot and later refresh_now() calls re-joined it, pinning the client to a
stale or already-rejected token. Adds a regression test that cancels the
leader and asserts the next refresh starts a fresh attempt.
- Redact bearer secrets from Debug. RefreshedToken and the oidc
RefreshTokenInput/RefreshTokenOutput now use manual Debug impls that omit
the access/refresh token fields via finish_non_exhaustive, matching the
house style (e.g. SecretResolver, SandboxJwtIssuer). Prevents a stray
{:?} or a containing struct's derived Debug from writing tokens to logs.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* fix(sdk): fail refresh when the new token can't be encoded as metadata
store_bearer now returns an error instead of silently keeping the previous
bearer value. The TokenSource commits the refreshed token to its state
before the client writes it into the interceptor slot, so a silent drop
left the interceptor on the old (expiring) token with no path back to a
refresh. Surfacing the error fails the call loudly instead. Adds a unit
test covering a token that can't be encoded as gRPC metadata.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* fix(sdk): refresh OIDC tokens on raw routes and harden rotation
Raw gRPC access never triggered OIDC refresh: a client that only used
raw_grpc/raw_inference kept sending the initial bearer until it expired,
with no proactive or reactive refresh. Add raw_grpc_fresh and
raw_inference_fresh accessors that refresh before returning the client,
plus force_refresh for reactive recovery after an Unauthenticated raw
RPC.
Guard the single-flight refresh commit against a concurrent replace().
The in-flight attempt now records the generation it started from and
skips its write when an external replace() has advanced it, so timer or
callback driven rotation is no longer clobbered by a slower refresh.
Remove TokenSource::snapshot(): it returned an empty string under write
contention and had no consumer on the CLI/TUI path. Tests read committed
state directly instead.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* docs(sdk): rewrite crate README for consumers
Recast the openshell-sdk README as a usable crate README rather than an
RFC excerpt. Drop the Responsibilities/Non-responsibilities/Consumers
scope-boundary sections and the mTLS migration rationale, folding the
useful facts (explicit token, no disk/name resolution, Refresh trait,
SdkError mapping) into the intro, a new Auth and refresh section, and
Public surface. Remove the dead relative RFC link and status-label
prose so the doc renders cleanly wherever it is published.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* fix(sdk): address review feedback on OIDC refresh and transport
Apply Drew's review notes on PR #1862:
- Drop unused `rustls-pemfile` dependency and move `tokio-stream` to
dev-dependencies (only used by tests).
- Guard OIDC `expires_at` against u64 overflow with `saturating_add`.
- Fix stale `#[non_exhaustive]` rationale in `AuthConfig` (the struct
`Oidc` variant it described as future already ships).
- Stop double-wrapping refresh errors: store the bare refresh-error text
so the single `SdkError::auth` wrap happens once at await.
- Strip stale CLI porting breadcrumbs from `build_channel` docs, keeping
the branch table.
- Treat proactive token refresh as best-effort: a transient failure falls
through to the request instead of failing an RPC whose current token is
still valid, with a regression test.
- Collapse `exec`'s inline auth retry into the shared `unary` helper.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* fix(sdk): preserve transient/terminal distinction in refresh errors
The refresh single-flight collapsed both `RefreshError::Transient` and
`RefreshError::Terminal` into a stringified `SdkError::Auth`, so consumers
(CLI, TUI, future language bindings) had no machine-readable way to tell a
retryable IdP blip from a dead session that needs re-authentication.
Carry the `RefreshError` through the shared outcome (kept `Clone` for
`Shared`) instead of its rendered text, and map it at the await site to a
new `retryable` flag on `SdkError::Auth`. Add `SdkError::auth_retryable`
and a `SdkError::retryable()` accessor; transient refresh failures report
`true`, every other error `false`. Add classification tests.
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
---------
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* AGENTS.md: Add more detailed signoff guidance
My coding agent started hallucinating my email address to be
`rbryant@nvidia.com`. Add more guidance in `AGENTS.md` about how the
`Signed-off-by` header should be added so that it uses the email
address from git configuration instead of one it makes up.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
* Update AGENTS.md
Co-authored-by: Evan Lezar <elezar@nvidia.com>
---------
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
* feat(sandbox): integrate OCSF structured logging for all sandbox events
WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).
- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
AppLifecycle)
- Add openshell-ocsf to all Docker build contexts
* fix(scripts): attach provider to all smoke test phases to avoid rate limits
GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.
Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.
* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp
The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:
15:49:11 sandbox INFO 15:49:11.649 I NET:OPEN ALLOWED ...
Now the shorthand is just the severity + structured content:
15:49:11 sandbox INFO I NET:OPEN ALLOWED ...
* refactor(ocsf): replace single-char severity with bracketed labels
Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].
Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.
Before: sandbox INFO I NET:OPEN ALLOWED ...
After: sandbox INFO NET:OPEN ALLOWED ...
Before: sandbox INFO M NET:OPEN DENIED ...
After: sandbox INFO [MED] NET:OPEN DENIED ...
* feat(sandbox): use OCSF level label for structured events in log push
Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:
sandbox OCSF NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
sandbox OCSF NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
sandbox INFO Fetching sandbox policy via gRPC
* fix(sandbox): convert new Landlock path-skip warning to OCSF
PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.
* fix(sandbox): use rolling appender for OCSF JSONL file
Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.
* fix(sandbox): address reviewer warnings for OCSF integration
W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
class name (NET:OPEN, HTTP:GET) already identifies structured events
and the LogPushLayer separately sets the level field.
W2: Log a debug message when OCSF_CTX.set() is called a second time
instead of silently discarding via let _.
W3: Document the boundary between OCSF-migrated events and intentionally
plain tracing calls (DEBUG/TRACE, transient, internal plumbing).
W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
(IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
consistency with the IPv4 bypass rule failure already migrated.
W5: Migrate malformed inference request warn to NetworkActivity with
ActivityId::Refuse and SeverityId::Medium.
W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
FORWARD proxy paths) to match the CONNECT deny severity pattern.
Allows and audits remain Informational.
* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled
The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.
* fix(ocsf): add timestamps to shorthand file layer output
The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.
Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After: 2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...
* fix(docker): touch openshell-ocsf source to invalidate cargo cache
The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.
Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.
* fix(ocsf): add OCSF level prefix to file layer shorthand output
Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.
Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After: 2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...
* fix(ocsf): clean up shorthand formatting for listen and SSH events
- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
action is empty (e.g., 'NET:LISTEN [INFO] 10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
from OCSF events to debug!() traces. Only the final verdict
(accepted/denied) is an OCSF event now, reducing noise from 3 events
to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.
* docs(observability): update examples with OCSF prefix and formatting fixes
Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.
* docs(agents): add OCSF logging guidance to AGENTS.md
Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.
Also adds openshell-ocsf to the Architecture Overview table.
* fix: remove workflow files accidentally included during rebase
These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.
* docs(observability): use sandbox connect instead of raw SSH
Users access sandboxes via 'openshell sandbox connect', not direct SSH.
* fix(docs): correct settings CLI syntax in OCSF JSON export page
The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.
* fix(e2e): update log assertions for OCSF shorthand format
The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).
* feat(sandbox): convert WebSocket upgrade log calls to OCSF
PR #718 added two log calls for WebSocket upgrade handling:
- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
This is a significant state change (L7 enforcement drops to raw relay).
- Unsolicited 101 without client Upgrade header → DetectionFinding with
High severity. A non-compliant upstream sending 101 without a client
Upgrade request could be attempting to bypass L7 inspection.
Closes#26
## Summary
Consolidates sandbox policy data into `dev-sandbox-policy.yaml` and deletes `dev-sandbox-policy-data.rego`. The YAML file becomes the single source of truth for sandbox policy data across both paths:
- **`nav create sandbox`** (CLI reads YAML → proto → gRPC → server stores → sandbox fetches)
- **`mise run sandbox`** (sandbox reads YAML directly via regorus)
## Why YAML
The policy data file is user-facing — it's the interface users interact with to configure sandbox security policy (allowed endpoints, binary identities, filesystem access, etc.).
Keeping this as YAML rather than rego:
- **Decouples data from enforcement engine** — if we change from OPA/rego to a different policy engine, the user's data file stays the same
- **Still works with regorus** — regorus can natively load YAML data via its `yaml` feature, so there's no conversion step in the local dev path
- **Enables future indirection** — we can add a transformation layer between the YAML file and what the policy engine receives, without changing the user-facing format
## Changes
### Policy data (`dev-sandbox-policy.yaml`)
- Flat top-level structure: `filesystem_policy`, `landlock`, `process`, `inference`, `network_policies`
- Renamed `filesystem` → `filesystem_policy` to match rego/proto convention
- Deleted `dev-sandbox-policy-data.rego` (was the rego equivalent of this data)
### Rego rules (`dev-sandbox-policy.rego`)
- All data references updated: `data.sandbox.*` → `data.*` (dropped `sandbox.` prefix)
- `package navigator.sandbox` unchanged (that's the rego rule namespace, not the data path)
### Sandbox entrypoint (`navigator-sandbox`)
- `--rego-policy`/`--rego-data` → `--policy-rules`/`--policy-data`
- `NAVIGATOR_REGO_POLICY`/`NAVIGATOR_REGO_DATA` → `NAVIGATOR_POLICY_RULES`/`NAVIGATOR_POLICY_DATA`
- `from_files()`: loads YAML natively via `regorus::Value::from_yaml_file()`
- `from_strings()` and `reload()`: accept YAML data via `Value::from_yaml_str()`
- `proto_to_opa_data_json()`: emits data at root level (no `"sandbox"` wrapper)
- Enabled regorus `yaml` feature
### CLI (`navigator-cli`)
- `DevSandboxPolicyFile.filesystem` → `filesystem_policy`
### Tests
- All inline test data converted from rego/JSON to YAML
- `test_engine()` uses `from_strings(rules, yaml)` with the dev YAML file
### Documentation
- Updated `architecture/plans/sandbox-policy-refactor/plan.md`, `architecture/plans/opa/plan.md`, `architecture/sandbox.md`, `architecture/containers.md`
Roland Huss