---
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
title: "Run Local Inference with Ollama"
sidebar-title: "Inference with Ollama"
slug: "get-started/tutorials/inference-ollama"
description: "Authorize an OpenShell sandbox to use an Ollama service running on the gateway host."
keywords: "Generative AI, Cybersecurity, Tutorial, Ollama, Local Inference, Sandbox, Provider Profile"
---

This tutorial connects an explicitly authorized sandbox to an Ollama service running on the same machine as the OpenShell gateway.

## Prerequisites

Complete the [Quickstart](/get-started/quickstart) before proceeding. Your workload image must contain the client or agent you plan to use.

## Start Ollama on a Reachable Address

Install Ollama on the gateway host and listen on an address reachable from sandboxes:

```shell
curl -fsSL https://ollama.com/install.sh | sh
OLLAMA_HOST=0.0.0.0:11434 ollama serve
```

In another terminal, pull a model:

```shell
ollama pull qwen3.5:0.8b
```

## Import an Endpoint-bearing Profile

Save this as `ollama-openai.yaml`:

```yaml
id: ollama-openai
display_name: Ollama
description: Host-level Ollama OpenAI-compatible API
category: inference
inference_capable: true
credentials: []
endpoints:
  - host: host.openshell.internal
    port: 11434
    protocol: rest
    access: read-write
    enforcement: enforce
binaries:
  - /usr/bin/curl
  - /usr/local/bin/curl
  - /usr/bin/python3
  - /usr/local/bin/python
  - /sandbox/.uv/python/**
  - /sandbox/.venv/**
```

```shell
openshell provider profile lint -f ollama-openai.yaml
openshell provider profile import -f ollama-openai.yaml
openshell provider create --name ollama --type ollama-openai
```

The provider has no secret because this Ollama server does not authenticate. Its attachment still carries the endpoint and binary policy.

## Attach and Verify

Use an explicit workload image containing `curl`:

```shell
openshell sandbox create \
  --name ollama-client \
  --from registry.example.com/tools/curl:latest \
  --provider ollama \
  -- \
  curl http://host.openshell.internal:11434/v1/chat/completions \
    --json '{
      "model":"qwen3.5:0.8b",
      "messages":[{"role":"user","content":"hello"}],
      "max_tokens":10
    }'
```

For an OpenAI SDK client, set its base URL to `http://host.openshell.internal:11434/v1`, use any non-empty API key value the SDK accepts, and select the real Ollama model in the request.

## Troubleshooting

- Bind host-level Ollama to `0.0.0.0`, not `127.0.0.1`.
- Use `host.openshell.internal`, not `localhost`, from a sandbox.
- Confirm attachment with `openshell sandbox provider list ollama-client`.
- Inspect the effective policy with `openshell policy get ollama-client --full`.
- Run `ollama ps` and `ollama pull <model>` when the model is unavailable.
- If the gateway is remote, the hostname refers to the remote gateway host, not your laptop. Use a shared service address or tunnel instead.

## Next Steps

- [Provider-backed Inference](/sandboxes/inference-routing)
- [Profiles](/providers/profiles)
- [Customize Sandbox Policies](/sandboxes/policies)