mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-20 06:00:06 +08:00
50230616d5
* feat(sandbox): default to official Alpine sandbox image default_sandbox_image() now returns docker.io/library/alpine:3.22, a generic version-qualified official image, so a fresh install no longer depends on the community sandbox image catalog. All compute drivers (docker, podman, kubernetes, vm) inherit this fallback. Part of #3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com> * feat(deploy): default deployment configs to the official Alpine sandbox image Update the shared gateway default_image, Helm chart values, the standalone Kubernetes manifest, and the dev gateway task scripts to use docker.io/library/alpine:3.22 instead of the community base image, consistent with default_sandbox_image(). GPU e2e image-build base is left unchanged (CUDA needs a glibc base). Part of #3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com> * feat(driver): default to numeric non-root identity for USER-less images With the default sandbox image now Alpine, images that declare no OCI USER must start instead of being rejected. When the image declares no USER and the policy requests none, the Podman and Docker drivers now supply a numeric non-root identity (DEFAULT_SANDBOX_UID/GID = 1000) instead of rejecting, matching the numeric-identity behavior of the Kubernetes and VM drivers. The supervisor's resolved-identity path runs the sandbox as a synthesized non-root account without the account existing in the image. Images that declare a USER keep the OCI resolution path unchanged. Part of #3116. Signed-off-by: Akram <akram.benaissi@gmail.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(conformance): use Alpine workload image Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(policy): drop community image /app path from default policy The restrictive default policy granted read-only access to /app, a directory that only existed in the community base image. A generic Alpine default has no /app, so remove it. Landlock best-effort already ignores absent paths; this just stops advertising a community-specific layout in the default. Part of #3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com> * docs(config): document Alpine default images Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(podman): report early sandbox termination Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(podman): initialize rootless workspace ownership Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(sandbox): qualify NVIDIA Ubuntu default Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(podman): initialize rootful default workspace Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(sftp): add native sandbox adapter Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(sftp): gate runtime helper support to Linux Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(sftp): support standard OpenSSH file operations Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(sftp): harden rename and special file handling Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(runtime): remove community image dependencies Signed-off-by: Drew Newberry <anewberry@nvidia.com> * test(e2e): build provider readiness tool fixture Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(e2e): use a dedicated Noble fixture for Docker tests Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Signed-off-by: Drew Newberry <anewberry@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Drew Newberry <anewberry@nvidia.com>
100 lines
3.1 KiB
Plaintext
100 lines
3.1 KiB
Plaintext
---
|
|
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
title: "Run Local Inference with Ollama"
|
|
sidebar-title: "Inference with Ollama"
|
|
slug: "get-started/tutorials/inference-ollama"
|
|
description: "Authorize an OpenShell sandbox to use an Ollama service running on the gateway host."
|
|
keywords: "Generative AI, Cybersecurity, Tutorial, Ollama, Local Inference, Sandbox, Provider Profile"
|
|
---
|
|
|
|
This tutorial connects an explicitly authorized sandbox to an Ollama service running on the same machine as the OpenShell gateway.
|
|
|
|
## Prerequisites
|
|
|
|
Complete the [Quickstart](/get-started/quickstart) before proceeding. Your workload image must contain the client or agent you plan to use.
|
|
|
|
## Start Ollama on a Reachable Address
|
|
|
|
Install Ollama on the gateway host and listen on an address reachable from sandboxes:
|
|
|
|
```shell
|
|
curl -fsSL https://ollama.com/install.sh | sh
|
|
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
|
```
|
|
|
|
In another terminal, pull a model:
|
|
|
|
```shell
|
|
ollama pull qwen3.5:0.8b
|
|
```
|
|
|
|
## Import an Endpoint-bearing Profile
|
|
|
|
Save this as `ollama-openai.yaml`:
|
|
|
|
```yaml
|
|
id: ollama-openai
|
|
display_name: Ollama
|
|
description: Host-level Ollama OpenAI-compatible API
|
|
category: inference
|
|
inference_capable: true
|
|
credentials: []
|
|
endpoints:
|
|
- host: host.openshell.internal
|
|
port: 11434
|
|
protocol: rest
|
|
access: read-write
|
|
enforcement: enforce
|
|
binaries:
|
|
- /usr/bin/curl
|
|
- /usr/local/bin/curl
|
|
- /usr/bin/python3
|
|
- /usr/local/bin/python
|
|
- /sandbox/.uv/python/**
|
|
- /sandbox/.venv/**
|
|
```
|
|
|
|
```shell
|
|
openshell provider profile lint -f ollama-openai.yaml
|
|
openshell provider profile import -f ollama-openai.yaml
|
|
openshell provider create --name ollama --type ollama-openai
|
|
```
|
|
|
|
The provider has no secret because this Ollama server does not authenticate. Its attachment still carries the endpoint and binary policy.
|
|
|
|
## Attach and Verify
|
|
|
|
Use an explicit workload image containing `curl`:
|
|
|
|
```shell
|
|
openshell sandbox create \
|
|
--name ollama-client \
|
|
--from registry.example.com/tools/curl:latest \
|
|
--provider ollama \
|
|
-- \
|
|
curl http://host.openshell.internal:11434/v1/chat/completions \
|
|
--json '{
|
|
"model":"qwen3.5:0.8b",
|
|
"messages":[{"role":"user","content":"hello"}],
|
|
"max_tokens":10
|
|
}'
|
|
```
|
|
|
|
For an OpenAI SDK client, set its base URL to `http://host.openshell.internal:11434/v1`, use any non-empty API key value the SDK accepts, and select the real Ollama model in the request.
|
|
|
|
## Troubleshooting
|
|
|
|
- Bind host-level Ollama to `0.0.0.0`, not `127.0.0.1`.
|
|
- Use `host.openshell.internal`, not `localhost`, from a sandbox.
|
|
- Confirm attachment with `openshell sandbox provider list ollama-client`.
|
|
- Inspect the effective policy with `openshell policy get ollama-client --full`.
|
|
- Run `ollama ps` and `ollama pull <model>` when the model is unavailable.
|
|
- If the gateway is remote, the hostname refers to the remote gateway host, not your laptop. Use a shared service address or tunnel instead.
|
|
|
|
## Next Steps
|
|
|
|
- [Provider-backed Inference](/sandboxes/inference-routing)
|
|
- [Profiles](/providers/profiles)
|
|
- [Customize Sandbox Policies](/sandboxes/policies)
|