Calling Ollama from code

The HTTP API, streaming responses, embeddings, and a small wrapper you can reuse across a project.

The HTTP API

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Explain HTTP caching in two sentences.",
  "stream": false
}'

import requests

def ask(prompt, model="llama3.2", **opts):
    r = requests.post("http://localhost:11434/api/generate",
                      json={"model": model, "prompt": prompt,
                            "stream": False, "options": opts},
                      timeout=120)
    r.raise_for_status()
    return r.json()["response"]

print(ask("Summarise this ticket in one line.", temperature=0.1))

• /api/generate for a single prompt, /api/chat for message lists with roles.
• "stream": true returns newline-delimited JSON chunks — use it for anything interactive.
• options carries sampling settings: temperature, num_predict, top_p.
• The server has no authentication by default: bind it to localhost, never expose it to the internet as-is.

Streaming

import json, requests

with requests.post("http://localhost:11434/api/chat",
                   json={"model": "llama3.2",
                         "messages": [{"role": "user", "content": "Write a haiku about latency."}],
                         "stream": True},
                   stream=True, timeout=300) as r:
    for line in r.iter_lines():
        if not line:
            continue
        chunk = json.loads(line)
        print(chunk["message"]["content"], end="", flush=True)
        if chunk.get("done"):
            break

Embeddings for search

r = requests.post("http://localhost:11434/api/embed",
                  json={"model": "nomic-embed-text", "input": ["first doc", "second doc"]})
vectors = r.json()["embeddings"]

import numpy as np
def cosine(a, b):
    a, b = np.array(a), np.array(b)
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

Embed locally, then rank with cosine similarity. This gives you private semantic search with no per-query cost — the foundation of a local RAG pipeline.

FAQ

Related

Running models locally Memory and retrieval

Last refreshed 2026-09-18.