mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-20 06:00:06 +08:00

Files

T

mmilutinovic371 36bb9e3ec2 feat(providers): add DeepInfra as a built-in inference provider (#1902 )

* feat(providers): add DeepInfra as a built-in inference provider (v2 only)

- Adds `deepinfra` as a built-in Providers v2 profile (`providers/deepinfra.yaml`)
  with inference category, Bearer auth, and `DEEPINFRA_API_KEY` discovery
- Adds `DEEPINFRA_PROFILE` to inference routing so `inference.local` works
  with the `deepinfra` provider type
- Fixes `build_backend_url` to strip `/v1` from request paths when the base
  URL contains `/v1/` as an internal segment (e.g. `api.deepinfra.com/v1/openai`),
  preventing double-versioned paths like `.../v1/openai/v1/chat/completions`
- Updates `docs/sandboxes/providers-v2.mdx` and `docs/sandboxes/manage-providers.mdx`
  with DeepInfra entries; removes the old v1 workaround row that used `openai`
  type with `OPENAI_API_KEY`

Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com>

* fix(providers): address gator review findings for DeepInfra provider

- Narrow build_backend_url /v1 dedupe to URLs whose path component is
  exactly /v1 or starts with /v1/ — prevents regression on proxy
  endpoints where /v1 is buried deeper (e.g. /api/v1/openai); add
  regression test for the nested proxy path case
- Add deepinfra provider plugin with DEEPINFRA_API_KEY discovery,
  registered in ProviderRegistry so known_types() and TUI include it
- Add deepinfra to unsupported-inference-provider error message in
  openshell-server for accurate user-facing debugging guidance
- Add deepinfra to openai_compatible_profiles_include_embeddings test
  to lock in the OpenAI-compatible protocol contract

Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com>

* fix(router): handle /v1 as final path segment in build_backend_url dedup

Extends the /v1 deduplication logic to also strip /v1 from request paths
when the base URL's path ends with /v1 (e.g. https://api.groq.com/openai/v1).
The previous fix only matched paths starting with /v1/, which regressed
providers like Groq whose base path has /v1 as the last segment rather than
the first. The nested-proxy exclusion (e.g. /api/v1/openai) is preserved
since /v1 appears in the middle — neither first nor last segment. Adds a
regression test for the Groq-style base URL.

Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com>

* fix(providers): add deepinfra telemetry bucket and update profile list test

- Add DeepInfra variant to ProviderProfile telemetry enum and from_raw()
  mapping so deepinfra providers are tracked in their own bucket rather
  than falling through to Custom
- Map deepinfra in telemetry_provider_profile() in openshell-server
- Add deepinfra to list_provider_profiles_returns_built_in_profile_categories
  test (sorted between cursor and github)
- Update architecture/gateway.md inference provider list to include deepinfra

Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com>

* style(router): apply cargo fmt to backend.rs

Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com>

---------

Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com>

2026-06-16 16:39:03 -07:00

src

feat(providers): add DeepInfra as a built-in inference provider (#1902 )

2026-06-16 16:39:03 -07:00

tests

feat(providers): add Google Vertex AI inference provider (#1568 )

2026-06-02 10:45:50 -05:00

Cargo.toml

feat(telemetry): add build-time option to compile out telemetry (#1845 )

2026-06-09 14:19:26 -07:00

README.md

docs(architecture): reset subsystem docs (#1184 )

2026-05-07 08:59:02 -07:00

README.md

openshell-router

openshell-router is the inference routing and upstream execution engine used by the sandbox proxy and gateway inference validation paths.

Responsibilities

Select an upstream route from a candidate set (based on protocol compatibility).
Forward raw HTTP requests to the selected upstream backend.
Normalize upstream failures into router-level errors (unauthorized, unavailable, protocol/internal errors).
Keep routing decision logic in one place so strategies can evolve (fallbacks, scoring, health-based routing).

Non-responsibilities

Authentication and sandbox identity.
Authorization and policy enforcement.
Persistence of routes/entities.
Loading sandbox or policy objects.

These are owned by openshell-server and openshell-sandbox.

Integration Contract

Current split:

openshell-server:
- authenticates user-facing inference configuration changes
- resolves managed route candidates from provider records
- validates backend endpoints
openshell-sandbox:
- intercepts https://inference.local
- detects the source inference protocol
- passes sanitized requests and resolved route candidates to the router
openshell-router:
- picks a route from candidates (proxy_with_candidates)
- forwards the HTTP request upstream and returns the raw response

Public APIs

Router::proxy_with_candidates(source_protocol, method, path, headers, body, &[ResolvedRoute])
- Filters candidates by protocol compatibility, then forwards the request to the first match.
- Preferred path for entity-driven server routing.

Notes

Route selection matches candidates by protocol field (e.g. openai_chat_completions).
Route selection is intentionally simple and will evolve.