Working with large language models for NLP tasks

Zero-shot and few-shot prompting, instruction tuning, structured extraction into JSON, and a decision rule for prompting versus fine-tuning.

Zero-shot, few-shot and instruction design

import json
from openai import OpenAI

client = OpenAI()

SYSTEM = """You are a support ticket classifier.
Return JSON with exactly two keys: category and urgency.
category is one of: billing, technical, account, other.
urgency is one of: low, normal, high.
Return only the JSON object, with no prose and no markdown fences."""

def classify(text, examples=None, model="gpt-4o-mini"):
    messages = [{"role": "system", "content": SYSTEM}]
    for example in examples or []:
        messages.append({"role": "user", "content": example["text"]})
        messages.append({"role": "assistant",
                         "content": json.dumps(example["label"], ensure_ascii=False)})
    messages.append({"role": "user", "content": text})

    response = client.chat.completions.create(
        model=model, messages=messages, temperature=0,
        response_format={"type": "json_object"},
    )
    return json.loads(response.choices[0].message.content)

few_shot = [
    {"text": "I was charged twice this month.", "label": {"category": "billing", "urgency": "high"}},
    {"text": "How do I change my email address?", "label": {"category": "account", "urgency": "low"}},
]

print(classify("The dashboard has been blank since the update.", few_shot))

• A structured output constraint beats a politely worded instruction. Schema-constrained decoding guarantees valid JSON; asking for it only makes it likely.
• Few-shot examples teach format and boundaries more than they teach the task. Two or three well-chosen examples near the decision boundary beat ten random ones.
• Use temperature=0 for extraction and classification. Non-zero temperature makes the same input produce different labels across runs, which makes evaluation impossible.
• Keep the system prompt stable and version it. Changing a label definition mid-experiment invalidates every metric you collected before the change.

Structured extraction and validation

from pydantic import BaseModel, Field, ValidationError
from typing import Literal, Optional

class LineItem(BaseModel):
    description: str
    quantity: int = Field(ge=1, le=1000)
    unit_price: float = Field(ge=0)

class Invoice(BaseModel):
    vendor: str
    currency: Literal["GBP", "USD", "EUR"]
    total: float = Field(ge=0)
    due_date: Optional[str] = None
    line_items: list[LineItem] = []

def extract_invoice(text, model="gpt-4o-mini"):
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "Extract the invoice. Omit fields you cannot find."},
            {"role": "user", "content": text},
        ],
        temperature=0,
        response_format={"type": "json_object"},
    )
    raw = json.loads(response.choices[0].message.content)

    try:
        return Invoice.model_validate(raw), None
    except ValidationError as exc:
        return None, exc.errors()          # route to review, do not guess

parsed, errors = extract_invoice(sample)
print(parsed or errors)

• Validate the model's output against a schema and route failures to a human or a retry. Trusting well-formed JSON without validation is how bad numbers reach a database.
• Make every field optional and let the model omit rather than invent. A required field with no evidence in the text is an invitation to hallucinate.
• Ask for the source span for each field when accuracy matters. It costs a few tokens and enables an automatic grounding check.
• For long documents, extract per section and merge afterwards. A single prompt over 100 pages loses detail in the middle of the context window.

Prompting or fine-tuning

# build the training set for a fine-tune from the examples you already labelled
import json

def to_chat_records(pairs):
    records = []
    for text, label in pairs:
        records.append({
            "messages": [
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": text},
                {"role": "assistant","content": json.dumps(label, ensure_ascii=False)},
            ]
        })
    return records

with open("train.jsonl", "w", encoding="utf-8") as fh:
    for record in to_chat_records(annotated):
        fh.write(json.dumps(record, ensure_ascii=False) + "\n")

# hold out a split that is never used for prompt development, then compare
# cost, latency, and accuracy of the prompted and fine-tuned systems on it

FAQ

Related

Classical text classification Evaluating NLP systems

Last refreshed 2026-09-18.