mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-17 05:31:16 +08:00
Maxime GrenuandJohn Myers
0e5ebb6f76
* fix(router): use max_completion_tokens for OpenAI GPT-5+ validation probe OpenAI GPT-5 models reject the legacy max_tokens parameter and require max_completion_tokens. The inference validation probe now sends max_completion_tokens as the primary parameter, with an automatic fallback to max_tokens when the backend returns HTTP 400 (for legacy/self-hosted backends that only support the older parameter). Closes #517 Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com> * style(router): fix cargo fmt import order and line length --------- Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
openshell-router
openshell-router is the inference routing and upstream execution engine used by openshell-server.
Responsibilities
- Select an upstream route from a candidate set (based on protocol compatibility).
- Forward raw HTTP requests to the selected upstream backend.
- Normalize upstream failures into router-level errors (
unauthorized,unavailable, protocol/internal errors). - Keep routing decision logic in one place so strategies can evolve (fallbacks, scoring, health-based routing).
Non-responsibilities
- Authentication and sandbox identity.
- Authorization and policy enforcement.
- Persistence of routes/entities.
- Loading sandbox or policy objects.
These are owned by openshell-server.
Integration contract with openshell-server
Current split:
openshell-server:- authenticates request origin
- resolves cluster-managed inference route candidates from providers
- loads enabled route candidates from the entity store
openshell-router:- picks a route from candidates (
proxy_with_candidates) - forwards the HTTP request upstream and returns the raw response
- picks a route from candidates (
Public APIs
Router::proxy_with_candidates(source_protocol, method, path, headers, body, &[ResolvedRoute])- Filters candidates by protocol compatibility, then forwards the request to the first match.
- Preferred path for entity-driven server routing.
Notes
- Route selection matches candidates by
protocolfield (e.g.openai_chat_completions). - Route selection is intentionally simple and will evolve.