Commit Graph
11 Commits
Author SHA1 Message Date
Mesut Oezdil ae5127f14d fix: correct example paths in local-inference README (#1676)
* fix: correct example paths in local-inference README

* fix: correct example paths in local-inference routes.yaml
2026-06-02 21:53:14 +00:00
Drew Newberry 70a0f6c547 refactor(cli): remove gateway lifecycle management (#1221) 2026-05-07 09:54:13 -07:00
Piotr Mlocek df38d1f66f feat(ci): add Markdown and Mermaid linting (#933) 2026-04-24 11:27:02 -07:00
Drew Newberry fbd93a4632 refactor: rename navigator- crate prefix to openshell- (#277) 2026-03-13 02:02:18 -07:00
Drew Newberry 7b0a243304 ci: remove sandbox docker build from publish and e2e workflows (#275) 2026-03-13 00:56:07 -07:00
Piotr Mlocek 14e296d318 fix(cli): add --no-keep for ephemeral sandbox create cleanup (#258) 2026-03-12 17:18:39 -07:00
John T. Myers 2e4c2fcc84 fix(proxy): stream inference responses instead of buffering entire body (#261)
* fix(proxy): stream inference responses instead of buffering entire body

The inference.local proxy path called response.bytes().await which
buffered the entire upstream response before sending anything to the
client. For streaming SSE responses this inflated TTFB from sub-second
to the full generation time, causing clients with TTFB timeouts to abort.

Add a streaming proxy variant that returns response headers immediately
and forwards body chunks incrementally using HTTP chunked transfer
encoding. Non-streaming responses and mock routes continue to work
through the existing buffered path.

Closes #260

* docs: update architecture docs and example for inference streaming

* test(example): add direct NVIDIA endpoint tests via L7 TLS intercept

Expand inference example to 4 test cases: inference.local and direct
endpoint, each streaming and non-streaming. The direct path exercises
the L7 REST relay (relay_chunked) to verify it already streams
correctly. NVIDIA_API_KEY is picked up from the sandbox env when
started with --provider nvidia.
2026-03-12 10:36:41 -07:00
Drew Newberry 984d1a6e5c chore: rename project from NemoClaw to OpenShell (#198) 2026-03-10 11:49:09 -07:00
Piotr MlocekandMiyoung Choi 107c85d1d7 docs(inference): clarify local inference routing (#190)
* docs(inference): clarify local inference routing

* docs(inference): update provider and model examples

* fix doc build

---------

Co-authored-by: Miyoung Choi <miyoungc@nvidia.com>
2026-03-09 21:34:14 -07:00
Drew Newberry 07b9d5d00e feat(cli): restructure CLI commands for simpler UX (#156) 2026-03-06 16:26:39 -08:00
Piotr Mlocek 31d7ca53ff refactor(inference): simplify routing — introduce inference.local, remove implicit catch-all (#146)
Remove multi-route CRUD system and replace with single managed cluster
route (inference.local). Key changes:

- Remove inference route CRUD RPCs and CLI commands
- Remove InspectForInference OPA action; policy is binary allow/deny
- Introduce AuthHeader enum and InferenceProviderProfile in navigator-core
- Router is now provider-agnostic: auth style carried on ResolvedRoute
- Replace InferenceRouteSpec with ClusterInferenceConfig (2 fields vs 8)
- Rename proto: routing_hint->name, SandboxResolvedRoute->ResolvedRoute,
  GetSandboxInferenceBundle->GetInferenceBundle, drop sandbox_id param
- Rename RouteConfig.route -> RouteConfig.name; use inference.local
- Add 'nemoclaw cluster inference update' for partial config changes
- Delete stale navigator.inference.v1.rs checked-in proto file
- Update architecture docs, agent skills, and CLI reference

Closes #133
2026-03-06 13:54:23 -08:00