mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-11-14 13:35:13 +08:00
def7da876cb5665d1d9b1acde5fc31eda79bf93f
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Adel Zaalouk
|
def7da876c |
fix(ocsf): emit schema-conformant OCSF 1.8 events (#4288)
Native events failed the official OCSF 1.8.0 validator: HTTP Activity emitted is_src_dst_assignment_known, sandbox Process Activity events had no actor, connection_info lacked direction_id, and SSH Activity events named no endpoint. Schema tests missed these because they checked only top-level, non-profile attributes. Add a connection direction enum, require an actor on Process Activity and an endpoint on SSH Activity at compile time, and name the supervisor's SSH server with svc_name and its socket path. Validate against OCSF 1.8.0 only. validate_required_fields and validate_known_fields share one recursive walker that rejects undefined attributes at every level and, for events, attributes from undeclared profiles; objects are validated as objects. Records labelled with another version fail. Vendor tactic and technique so ATT&CK-mapped findings validate, drop the unused v1.7.0 schemas, and make the embedded-schema drift test tolerate CRLF checkouts. Keep it fixed with a per-builder conformance test and a Podman e2e test that runs an L7 policy against a local server and validates the supervisor's real Network, HTTP and SSH Activity JSONL. Top-level container stays unchanged; #4915 tracks it. The schema downgrade is unchanged. Closes #4283 Signed-off-by: Adel Zaalouk <azaalouk@redhat.com> |
||
|
krishicks
|
a24340beba |
feat(auth): emit OCSF events for gateway authorization denials (#4728)
When the gateway refuses a caller that has already authenticated, it now emits an OCSF API Activity event. The caller gets exactly the same error as before. The event records: - Who: the principal (user, mTLS operator, sandbox, or gateway peer). - What: the gRPC method, plus the sandbox or workspace that was refused when there is one. - Why: a reason such as cross_sandbox, missing_role or not_workspace_member. - Correlation: the caller's address and request ID, the same fields the authentication events carry. That covers the denials made centrally by the router and the shared guards, plus the checks inside handlers that have no shared guard: a sandbox naming another sandbox (in either sandbox-name resolver), a non-user caller refused by mutation admission, a sandbox sending a config update it isn't allowed to, and a non-admin granting the workspace admin role. Denials raised in the mutation admission worker, which runs in a spawned task, keep the request's context. When a sandbox credential is refused for calling a method it may not call, touching another sandbox, or acting across workspaces, the gateway also emits a High-severity Detection Finding. It carries the same correlation_uid as the denial event. A real supervisor never makes these requests, so a finding means the credential is probably being misused. Two cases are recorded without a finding: a method the gateway doesn't recognize (unknown_method), so a newer supervisor doesn't look like a compromised one, and a supervisor whose own sandbox was deleted mid-request, which is refused but isn't misuse. The shorthand line names the refused sandbox or workspace, for example sandbox:<id>, so operators reading console logs can see the target without JSONL or OTLP export. To support this, the OCSF crate's Actor can now hold a user instead of a process, and the API Activity builder gained the fields gateway events need. Supervisor inference events come out unchanged. Gateway events declare the container profile only when they concern a sandbox, as MXC findings do. The OCSF 1.8.0 resource_details schema is vendored so the refused sandbox or workspace goes in the standard resources field. Refs #2912 Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
krishicks
|
fdfbf84f8b |
fix(ocsf): report the required device.os.type_id (#4677)
OCSF 1.8 requires type_id on the os object, but device.os carried only a name. Events now report Linux (200), Windows (100), macOS (300), or Other (99) with the platform name as the type label. The os object schema is vendored so tests validate it. Refs #2912 Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
krishicks
|
f41f24b549 |
feat(auth): record peer address and request ID on authentication events (#4673)
Authentication events now carry the connection's immediate peer as src_endpoint, or edge-tunnel for WebSocket-tunneled requests, and the request's x-request-id as metadata.correlation_uid when it is a UUID. Refs #2912 Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
krishicks
|
9c6867ccec |
feat(auth): emit OCSF events for credential-less gateway requests (#4632)
Requests the gateway rejects before consulting any authenticator now emit an OCSF Authentication [3002] event: no credential presented, mTLS required without a client certificate, or an operator certificate combined with an authorization header. Each event carries a reason category in status_detail and never the presented credential. The response returned to the caller is unchanged. openshell-ocsf gains the Authentication class with user and service objects, and a shared OcsfCapture test layer. Schema validation now rejects attributes the class does not define and profile attributes whose profile the event does not declare. Refs #2912 Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
krishicksandEvan Lezar
|
a875add234 |
feat(server): write gateway OCSF events to JSONL (#3264)
* feat(server): write gateway OCSF events to JSONL Previously, gateway security activity was available only in diagnostic output, and events not associated with a sandbox, such as TLS certificate reloads, had no independent structured record. Now, configuring `openshell.gateway.ocsf_log` writes every gateway-produced OCSF record to a bounded JSONL destination independently of `RUST_LOG`. The destination supports daily or disabled rotation, retention limits, queue bounds, and optional schema downgrade to OCSF 1.1 or 1.3. Additionally, existing gateway emitters (TLS reloads, service routing, and policy approval and auto-approval audits) emit structured events, so they reach the JSONL destination, console shorthand, and the affected sandbox's log stream. Records identify the gateway by its configured name in `device.uid` and `device.name`, shared across replicas, with `device.hostname` identifying the replica and `device.os` the gateway's operating system. Metrics and warnings expose known best-effort losses. Refs #2762 Signed-off-by: Kris Hicks <khicks@nvidia.com> * fix(mxc): attribute ETW events to gateway Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Kris Hicks <khicks@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com> |
||
|
|
ae57979b03 |
feat(mxc): add Windows ETW-to-OCSF audit trail (#3015)
* feat(mxc): ETW->OCSF audit consumer + Windows OCSF JSONL parity (cp6 P1) Add a Windows MXC ETW->OCSF audit trail in openshell-driver-mxc: a real-time Sandboxing-provider ETW consumer that decodes events (TDH), attributes each to an OpenShell sandbox_id, and maps them to OCSF (lifecycle 6002, config 5019, process 1007, finding 2004). cp6 Phase 1 - durable OCSF JSONL audit-file parity with Linux: - openshell-ocsf: add emit_ocsf_event_routed (populates the event-bridge thread-local AND stamps sandbox_id+message in one dispatch) plus public set/clear_current_event; OS-aware device (Device::windows/for_current_os) so device.os.name reflects the host instead of a hardcoded Linux stub. - etw_consumer: emit via the routed emit (previously fired a bare info! that never populated the bridge, so the structured event was dropped). - openshell-server: install OcsfJsonlLayer over a synchronous daily-rotated appender (durable under force-kill), gated by OPENSHELL_OCSF_JSON, path via %PROGRAMDATA%\OpenShell\logs (override OPENSHELL_OCSF_LOG_DIR). - device.hostname now resolves to the real gateway machine name. Box-proven on 7F203-MXC-001: JSONL lines == shorthand OCSF rows, all valid OCSF JSON, per-sandbox attribution intact, disabled state writes nothing. Signed-off-by: Akber Raza <akberr@nvidia.com> * feat(mxc): map remaining Sandboxing ETW events to OCSF Close the last three ETW->OCSF gaps so the audit trail covers the full set of events the Sandboxing provider emits (12/12): - ProcessLaunched -> Process Activity [1007] "Launch" (confirmed start; carries the real processId/threadId, the twin of CreateProcessInSandbox which only has the request + command line). - SandboxProxyConfigured -> Device Config State Change [5019] (the one network-plane setup event; surfaces proxyPort, "no proxy" when 0). - SandboxConsoleReferencePlumbed -> Device Config State Change [5019] (console-handle plumbing). map_config_state now handles the full config/hardening/setup family and carries proxyPort/hasConsoleReference/creationFlags as unmapped fields. Verified on 7F203-MXC-001: 11/12 event types emit OCSF without a proxy (SandboxProxyConfigured requires proxy config to fire). Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc): seed ETW attribution under registry lock + Device tests Address CodeRabbit review on !31: - Prevent stale ETW attribution on a delete/launch race: register the wxc-exec pid while holding the registry lock, and bail if the sandbox entry is already gone. Previously the attribution key could be seeded after `delete` had removed the sandbox, leaving a stale key that could misroute later Sandboxing ETW events to a dead sandbox_id. Lock order (registry -> attribution) matches the delete path, so no deadlock. - Add unit tests for the new Device::windows and Device::for_current_os constructors to harden Windows/Linux OCSF device parity. Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc-etw): buffer+replay racing events and harden attribution keys Addresses two ETW->OCSF attribution review items (Shailendra #1, #2). #2 early-event loss: ETW delivers the sandbox create/config burst the instant wxc-exec starts, which can beat the driver's register_launch (now under the registry lock post-Ready). process_event previously dropped anything unresolved, losing the racing burst. Add a bounded, time-bounded pending buffer (PENDING_MAX=4096, PENDING_TTL=5s): unresolved events are held and replayed once attribution lands, aged-out ones dropped. Consumer switched to a timed recv_timeout(200ms) so the buffer is re-driven after each event and on a tick. Emit path factored into shared emit_resolved(). #1 attribution collisions: a Windows PID is recycled after exit and a command line is commonly identical across sandboxes. register_launch now rebinds by_pid on reuse and clears the stale last_pid_sid hint (warns if the PID still pointed at a different, leaked sandbox); command line is held in by_cmd only while unique and demoted to a new ambiguous_cmds set on a second owner, so a duplicate command refuses to resolve rather than misroute. Unit tests: buffer replay (direct + cross-link), buffer bound, PID-reuse rebind, duplicate-cmd non-resolution. Box-verified on 7F203-MXC-001 (5 sandboxes, identical cmd -> 5 isolated sandbox_ids, 50/50 OCSF/JSONL, BuffersLost=0). Signed-off-by: Akber Raza <akberr@nvidia.com> * docs(mxc-etw): note cmd_line is captured raw with no privacy filtering Review item #3 (Shailendra): add a PRIVACY NOTE on map_process_launch stating cmd_line is copied verbatim into OCSF process.cmd_line with no redaction, so secrets/PII on a command line land unredacted in the durable audit trail (deliberate audit-fidelity trade-off; treat the log as sensitive). Redaction is owned by an upstream privacy layer, not this path; no general audit-output PII scrubber exists today (openshell_core::secrets [CREDENTIAL] redaction is scoped to the proxy HTTP-target logging, a separate egress path). Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc-etw): open ETW trace on caller thread so start_session reports real status Review item #4 (Shailendra): start_session previously returned Ok(EtwSession) as soon as the pump thread was spawned, but OpenTraceW ran later inside that thread; if it failed we still handed back a live-looking session and logged 'consumer started' (silent failure = false audit coverage). Split the two Win32 calls instead of adding a channel handshake (avoids any lost-wakeup/hang risk): the quick, synchronous OpenTraceW now runs on the caller thread (open_trace), and only the blocking ProcessTrace runs on the pump thread (run_trace). start_session returns Err if OpenTraceW fails (reclaiming the boxed Sender so the consumer disconnects, stopping the session, joining the consumer) and returns Ok/logs 'started' only once capture is genuinely open. Opened handle + LoggerName buffer + boxed Sender are carried to the pump via a Send OpenedTrace so they outlive ProcessTrace. Box-verified on 7F203-MXC-001: consumer started=True, failed-to-start=False, 50 OCSF rows / 50 JSONL, BuffersLost=0 (no regression to capture/emit). Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc-etw): guard pending-event replay against PID recycling CodeRabbit flagged that drain_resolved() re-resolved buffered events against the live by_pid map, so if Windows recycled a wxc-exec PID within PENDING_TTL a stale event from the dead sandbox could be emitted under the new owner. Stamp each by_pid registration with its Instant and add resolve_replay(), used only on the buffered/replay path. It (a) never falls back to the recycle-/ambiguity-prone by_cmd or last_pid_sid keys, and (b) trusts a PID match only when the registration is not newer than the buffered event by more than REPLAY_PID_GRACE (2s) - a recycled PID's registration lands well outside that window, so the stale event ages out instead of misattributing. The legitimate #2 seed race (registration lands ~immediately) still replays. Adds unit tests for the recycle-refusal, in-grace acceptance, and weak-fallback exclusion. Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc-etw): surface unexpected ProcessTrace termination (review #4) start_session already returns Err on OpenTraceW failure (runs on the caller thread since e41a7701), closing the first half of Shailendra's #4. This closes the second half: ProcessTrace's result was discarded, so if capture died mid-run the backend had no way to know. Add a shared CaptureHealth (stopped/stopping/exit_code) between the pump thread and EtwSession. run_trace now records ProcessTrace's WIN32_ERROR and, when the pump returns without a deliberate stop, logs at ERROR that MXC OCSF capture is no longer running. EtwSession::stop() sets `stopping` before teardown so a normal shutdown isn't misreported, and EtwSession::is_capture_alive() exposes the state for status/diagnostics. Box-verified on 7F203-MXC-001: 5 sandboxes, 50 attributed OCSF rows, JSONL parity 50/50, BuffersLost=0, clean start/stop (no false failure). Signed-off-by: Akber Raza <akberr@nvidia.com> * feat(mxc-ocsf): add ETW->OCSF audit-trail example kit; fix proxy-configured message Add a runnable OCSF audit-trail example under examples/ (run-ocsf-audit.ps1, mxc-ocsf-audit.toml, ocsf-audit.yaml, README) that spins up sandboxes with the in-process ETW consumer and egress proxy on, emitting a full OCSF JSONL audit trail across all four classes (6002/5019/1007/2004). Fix SandboxProxyConfigured mapping to log "MXC sandbox proxy configured" instead of a misleading "(no proxy)" when the provider reports proxyPort=0; the event's presence already indicates proxy configuration. Verified on-box: 26 events, all mapped ETW event types present. Signed-off-by: Akber Raza <akberr@nvidia.com> * feat(mxc-ocsf): clearer audit report + client-safe run-ocsf-audit.ps1 Improve the ETW to OCSF audit-trail example output and make it safe to ship. Report: - Add an event-type coverage count ("N of M expected event types fired"); the denominator auto-adjusts (8 with proxy on, 7 with -NoProxy). - Split the checklist into expected event types vs anomaly findings (ActivityError/FallbackError), which are reported separately and not counted toward coverage (a clean run may emit none). - Verdict is now coverage-based (all expected types must fire) instead of the looser "at least 3 OCSF classes". - Call out the absolute path to the durable OCSF JSONL log prominently. Client-safety: - Default -ShareOut to empty (no auto-copy); pass -ShareOut a UNC path to opt in. Removes a hardcoded internal share path from a published example. - Drop internal-team wording ("Hand that zip back for evaluation", "BUNDLE:") in favor of neutral "Results bundle:". - Update README-ocsf-audit.txt to match the opt-in -ShareOut behavior. Verified on both MXC boxes: 7F203-MXC-001 (base-container) -> PASS, 8 of 8 event types, 26 OCSF events across 4 classes; 7F203-MXC-003 (AppContainer fallback) -> reduced set as expected, clean output. Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc): configure OCSF audit workloads per sandbox - remove unsupported gateway-scoped workload fields from the shipped MXC audit example. - build the command, working directory, and filesystem grant from each run's ShareDir - pass the workload through --driver-config-json. - preserve the host CONNECT proxy configuration and conditional audit coverage for the future host_connect_proxy merge - require the workload output when determining the audit verdict. Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(mxc): omit command arguments from OCSF audit logs - record only the executable basename for MXC CreateProcessInSandbox audit events - leave process.cmd_line unset so workload arguments cannot reach shorthand or JSONL logs - cover tokens, passwords, signed URLs, and PII with a secret-leak regression test - update the audit example, architecture guidance, and published logging documentation - preserve ETW attribution and future host_connect_proxy enforcement behavior Signed-off-by: Akber Raza <akberr@nvidia.com> * feat(etw): enhance ETW session management with distinct naming for concurrent gateways * fix(etw): bound the audit queue during overload - replace the unbounded ETW callback channel with count- and byte-bounded buffering - keep the ETW callback non-blocking and count records rejected during overload - emit immediate, rate-limited warnings that identify resulting audit coverage gaps - make the audit example fail when queue overload causes dropped ETW records - cover stalled consumers, oversized events, and warning throttling with unit tests Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(etw): harden sandbox audit attribution - remove command-line and persistent per-PID fallback keys from live and replay resolution - retire the driver-owned wxc-exec PID before publishing child completion - retain established identity, activity, and correlation-vector links only for the five-second late-event window - prevent buffered records from crossing rapid PID retirement and reuse boundaries - add resolver and lifecycle coverage and document the attribution trust boundary Signed-off-by: Akber Raza <akberr@nvidia.com> * chore(mxc): address rebase follow-ups Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(mxc): align OCSF audit example with driver config - remove unsupported egress proxy settings - stop requiring the unavailable proxy audit event - update example documentation for supported event coverage Signed-off-by: Akber Raza <akberr@nvidia.com> * fix(etw): redact command-line secrets in DecodedEtwEvent summary * fix(etw): enhance PID resolution and event attribution logic for ETW records * fix(ocsf): restrict gateway-local JSONL sink to Windows/MXC path with opt-in configuration * address rebase issues * fix(mxc): address ETW audit review feedback Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(mxc): fail closed across ambiguous PID reuse Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(mxc): bind ETW attribution to process generation Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Akber Raza <akberr@nvidia.com> Signed-off-by: Drew Newberry <anewberry@nvidia.com> Co-authored-by: Jamie King <jamiek@nvidia.com> Co-authored-by: Drew Newberry <anewberry@nvidia.com> |
||
|
krishicks
|
67374efdf8 |
fix(ocsf): emit schema-valid event identities (#3247)
Previously, every event from a sandbox reused the sandbox ID as its event ID. Consumers deduplicating security records could mistake separate events for the same record, and missing device types or empty image objects could prevent schema validation. Give each event its own ID, retain the sandbox association separately, and classify the environment as Other/Sandbox while keeping the OS separate. Omit unknown container details instead of emitting empty objects. Security tooling can now distinguish events from the same sandbox and read their identity consistently after serialization. Refs #1055 Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
Adel ZaaloukandClaude Opus 4.6
|
600bbae845 |
feat(ocsf): emit AI inference events via ai_operation profile on ApiActivity (#2664)
Apply the official OCSF ai_operation profile (introduced in v1.8.0) to ApiActivity [6003] events when the inference proxy routes a model call through inference.local. Attaches an ai_model object (name, ai_provider) and puts token counts and latency in unmapped fields. ApiActivity [6003] is the schema-correct class for the ai_operation profile in v1.8.0 (HttpActivity only gets it in v1.9.0). In Splunk CIM, ApiActivity maps to the "Change" data model, naturally separating inference events from regular HTTP proxy traffic. Changes: - Add AiModel object and ai_model field on BaseEventData - Add ApiActivityEvent struct and ApiActivityBuilder - Add emit_ai_inference in proxy.rs using ApiActivity with ai_operation - Vendor OCSF v1.8.0 schemas including api_activity class, ai_model object, and ai_operation profile definitions - Bump OCSF_VERSION to 1.8.0 - Update schema validation to skip profile-gated required fields Shorthand: API:INFERENCE [INFO] claude-3-haiku via anthropic 701ms [POST /v1/messages] Splunk/SIEM backward compatibility (v1.1/v1.3 CIM mapping) is tracked separately in #2662 as a configurable serialization concern. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Russell Bryant
|
c5d1d76d94 |
refactor(sandbox): replace iptables with nftables for network policy enforcement (#1401)
Migrate all sandbox and VM driver network policy enforcement from iptables to nftables. nftables provides atomic ruleset loading, a cleaner rule syntax, and is the standard netfilter interface in modern kernels. Sandbox bypass enforcement (openshell-sandbox): - Replace iptables chain of individual rule insertions with a single atomic nftables ruleset load via nft -f - New nft_ruleset module with pure functions for ruleset generation and unit tests - Combine log and reject rules in one inet family table (handles both IPv4 and IPv6 in a single ruleset) - Fall back to reject-only ruleset when kernel lacks nft_log support - Enable net.netfilter.nf_log_all_netns so log rules work from non-init network namespaces - Use temp file for nft ruleset loading instead of stdin for compatibility with minimal VM guest environments VM TAP networking (openshell-driver-vm): - Replace iptables NAT/forwarding rules with nftables equivalents - New nft_ruleset module for TAP network rule generation with unit tests - Atomic table-per-TAP-device lifecycle (create/destroy) - Host-side rules provide NAT infrastructure and defense-in-depth isolation (input chain restricts VM to gateway port only, forward chain blocks unsolicited inbound); primary security enforcement happens inside the VM guest via the sandbox supervisor's own rules VM init script: - Load nft kernel modules at sandbox init - Enable nf_log_all_netns sysctl for bypass detection logging OCSF / docs: - Update firewall rule engine references from iptables to nftables - Document host firewall interaction model and two-layer enforcement architecture in VM driver README and compute drivers reference Closes #1335 Signed-off-by: Russell Bryant <rbryant@redhat.com> |
||
|
John T. Myers
|
09af1b6bad |
fix(sandbox): add JSON bodies to proxy 403/502 responses and include port in HTTP log URLs (#809)
L4 CONNECT and forward-proxy denial responses (403) and upstream failure responses (502) were returned with empty bodies, making them indistinguishable from each other when using curl -s. Add a build_json_error_response() helper that produces consistent JSON bodies with error and detail fields, matching the format already used by the L7 deny path. Also fix Url::to_display_string() in openshell-ocsf to include non-default ports in the shorthand log output. Requests to e.g. http://host:9876/path were logged as http://host/path, dropping the port number. Closes #807, closes #808 |
||
|
John T. Myers
|
51aeffc9a7 |
feat(ocsf): create openshell-ocsf crate — standalone OCSF event types, formatters, and tracing layers (#489)
feat(ocsf): create openshell-ocsf crate with OCSF v1.7.0 event types, formatters, and tracing layers |