mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-11-05 08:36:09 +08:00
Polite_realism
5d9b0f0476
* fix(inference): prepend publisher prefix for Vertex non-Anthropic models Vertex AI's OpenAI-compatible endpoint requires the request body's model field to carry a publisher prefix (e.g. google/gemini-2.5-flash), but validate_vertex_model_id rejects slash as a path-traversal guard. This created a deadlock: bare model IDs pass validation but are rejected by Vertex with HTTP 400 "Malformed publisher model"; prefixed IDs are rejected at configuration time. Fix: in resolve_vertex_ai_route, compute body_model_id for non-Anthropic routes by prepending the publisher from infer_vertex_publisher() or the explicit VERTEX_AI_PUBLISHER config value. The bare model_id still goes through the path-traversal validator unchanged. Anthropic rawPredict routes encode the model in the URL path, not the body, and are unaffected. Both the project/region path and the base-URL-override path apply the prefix. For unrecognised models with no explicit publisher the bare ID is forwarded unchanged; Vertex's 400 is the correct observable signal in that case. Add an integration test in openshell-router that spins up a mock Vertex endpoint accepting only the publisher-prefixed form and rejecting the bare model name, verifying the body rewrite produces the required format. Closes #2351 Signed-off-by: politerealism <burdcat17@gmail.com> * fix(inference): propagate publisher-prefixed model_id through inference bundle resolve_route_by_name_with_credentials built the ResolvedRoute with config.model_id (the bare stored value) rather than resolved.route.model (the publisher-prefixed value computed by resolve_vertex_ai_route). As a result, the bundle delivered to sandboxes carried e.g. "gemini-2.5-flash" instead of "google/gemini-2.5-flash", so live sandbox requests still hit Vertex AI with the bare model name and received HTTP 400 "Malformed publisher model". Fix: use resolved.route.model in the bundle construction so the publisher prefix survives the bundle boundary and the router sends the correct body to Vertex AI. Update the existing gemini bundle test to assert the prefixed model_id and add a dedicated regression test that verifies the bundle carries the publisher prefix for non-Anthropic Vertex routes. Signed-off-by: politerealism <burdcat17@gmail.com> * style(inference): apply rustfmt to new regression test Signed-off-by: politerealism <burdcat17@gmail.com> * fix(inference): address clippy lints in build_vertex_route - Invert if !is_anthropic to satisfy clippy::if_not_else - Replace match on Option with map_or_else to satisfy clippy::option_if_let_else Signed-off-by: politerealism <burdcat17@gmail.com> --------- Signed-off-by: politerealism <burdcat17@gmail.com>