Generative AI¶

Generative AI covers any model that creates new content: text (LLMs), images (diffusion), audio (TTS), and video. This sheet maps the landscape — what the model families are, how they differ, and the core concepts (embeddings, context, temperature, structured output) that appear in every API.

Installation / Setup¶

pip install openai anthropic google-genai   # or: uv add openai anthropic google-genai

No single install covers everything — each API has its own package.

Core Syntax¶

Model families at a glance¶

Family	Type	Examples	Best for
GPT-4o / o3	Autoregressive LLM	OpenAI	Chat, reasoning, code, vision
Claude 3.5/4	Autoregressive LLM	Anthropic	Long context, safety, analysis
Gemini 1.5/2.0	Autoregressive LLM	Google	Multimodal, long context, search
Llama 3 / Mistral	Open-weight LLM	Meta / Mistral	Self-hosted, fine-tuning
Stable Diffusion / DALL·E / Imagen	Diffusion	Stability / OpenAI / Google	Image generation
Whisper	Encoder-decoder	OpenAI	Speech-to-text

Core concepts that appear everywhere¶

Concept	What it is	Typical values
Temperature	Randomness in output	0 = deterministic, 1 = creative, 0.7 = balanced
Top-p	Nucleus sampling threshold	0.9 = allow most tokens, 1.0 = all
Max tokens	Maximum output length	256–8192 (model-dependent)
Context window	Total input + output tokens	4K–200K (model-dependent)
Tokens	Subword units (~0.75 words per token)	1 token ≈ 4 chars
Embeddings	Dense vector representations	1536–3072 dimensions
System message	Instructions that frame the assistant's behavior	—
Structured output	Constrained output format (JSON, schema)	—

Prompting anatomy (messages format)¶

messages = [
    {"role": "system", "content": "You are a helpful data science tutor."},
    {"role": "user", "content": "Explain bias-variance trade-off in 3 sentences."},
]

Role	Purpose
`system`	Sets behavior, tone, constraints
`user`	The human's input (your query)
`assistant`	Previous model output (for multi-turn)

Common Operations¶

Temperature and sampling¶

# deterministic (factual, code, extraction)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    temperature=0.0,
)

# creative (brainstorming, writing)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    temperature=0.9,
    top_p=0.95,
)

Tokens and context windows¶

context window = input tokens + output tokens

Example: GPT-4o has 128K context
  → your prompt (2000 tokens) + system (200 tokens) + output (800 tokens) = 3000 tokens used
  → 125K tokens remaining

Token counting:

# OpenAI
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
print(len(enc.encode("Hello world")))    # ~2 tokens

Structured output (JSON mode)¶

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Return a JSON object with keys: summary, sentiment, topics."},
        {"role": "user", "content": text},
    ],
    response_format={"type": "json_object"},
)
import json
data = json.loads(response.choices[0].message.content)

Embeddings¶

response = client.embeddings.create(
    model="text-embedding-3-small",
    input=["Python is great for data science", "JavaScript runs in browsers"],
)
vectors = [item.embedding for item in response.data]
# vectors[0] is a 1536-dim list of floats

Streaming¶

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Function / tool calling¶

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"},
            },
            "required": ["city"],
        },
    },
}]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)
# response.choices[0].message.tool_calls contains the function call

Practical Examples¶

Example 1 — Build a text summarizer¶

from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY")

def summarize(text, style="concise"):
    system = "Summarize the text. Be concise." if style == "concise" else "Summarize in one detailed paragraph."
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": text},
        ],
        temperature=0.3,
    )
    return response.choices[0].message.content

Example 2 — Structured extraction¶

import json
from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY")

def extract_info(text):
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": """Extract structured info. Return JSON:
{"people": [{"name": str, "role": str}], "companies": [str], "dates": [str]}"""},
            {"role": "user", "content": text},
        ],
        response_format={"type": "json_object"},
        temperature=0.0,
    )
    return json.loads(response.choices[0].message.content)

Example 3 — Semantic search with embeddings¶

from openai import OpenAI
import numpy as np

client = OpenAI(api_key="YOUR_API_KEY")

docs = ["Python is great for ML", "JavaScript runs in browsers", "R is used for stats"]
doc_embs = client.embeddings.create(model="text-embedding-3-small", input=docs)
doc_vectors = np.array([e.embedding for e in doc_embs.data])

query = "data analysis tools"
q_emb = client.embeddings.create(model="text-embedding-3-small", input=[query])
q_vec = np.array(q_emb.data[0].embedding)

scores = np.dot(doc_vectors, q_vec) / (np.linalg.norm(doc_vectors, axis=1) * np.linalg.norm(q_vec))
print(docs[np.argmax(scores)])

Common Patterns¶

Pattern: multi-turn conversation¶

messages = [{"role": "system", "content": "You are a helpful assistant."}]

def chat(user_input):
    messages.append({"role": "user", "content": user_input})
    response = client.chat.completions.create(
        model="gpt-4o", messages=messages, temperature=0.7)
    assistant_msg = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_msg})
    return assistant_msg

Pattern: guardrails via system message¶

system = """You are a financial analyst assistant.
- Never give investment advice
- Always cite sources
- If unsure, say 'I don't know'
- Respond in bullet points"""

Pattern: retries with exponential backoff¶

import time

def call_with_retry(func, max_retries=3):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(2 ** attempt)

Best Practices¶

System message is powerful — use it to set behavior, output format, and constraints.
Temperature 0 for factual/structured tasks; 0.7–0.9 for creative tasks.
Always count tokens before sending long documents — context windows are finite.
Structured output (response_format) is more reliable than asking "return JSON" in the prompt.
Streaming is essential for UX — don't wait for the full response.
Function calling is the standard way to connect LLMs to external tools.
Use embeddings for semantic search, not keyword search.
Cache API calls when possible — same input + temperature 0 = same output.

Common Mistakes¶

Mistake	Fix
Hard-coding API keys in source code	Use environment variables or secrets manager
Temperature 0 for creative writing	Use 0.7–0.9 for creative tasks
Not checking token count before API call	Use `tiktoken` or model-specific tokenizer
Asking "return JSON" without `response_format`	Use `response_format={"type": "json_object"}`
Forgetting `max_tokens` limit	Set `max_tokens` to avoid truncation
Ignoring rate limits	Implement backoff; check headers for limits
Sending huge documents without chunking	Chunk text to fit context window
Using expensive model for simple tasks	GPT-4o-mini or Haiku for classification/extraction

Troubleshooting¶

Symptom	Cause / Fix
`RateLimitError`	Hit API rate limit — back off and retry
`ContextLengthExceeded`	Input too long — truncate or chunk
Model returns incomplete JSON	Increase `max_tokens` or simplify the schema
`AuthenticationError`	Wrong API key — check `YOUR_API_KEY`
Output is hallucinated facts	Lower temperature, add context, use retrieval (RAG)
Inconsistent output format	Use structured output mode, not prompt-based
Streaming stops mid-sentence	Hit `max_tokens` — increase it

openai-api — OpenAI API in detail
anthropic-api — Anthropic/Claude API
google-gemini — Google Gemini API
llm-concepts — LLM fundamentals
ollama — running LLMs locally
huggingface — Hugging Face for LLMs
litellm — unified multi-provider API
sentence-transformers — embeddings for search
RAG — retrieval-augmented generation
Data Visualization — plotting model outputs

Official Documentation¶

OpenAI API — https://platform.openai.com/docs
Anthropic API — https://docs.anthropic.com/
Google AI — https://ai.google.dev/docs
Hugging Face — https://huggingface.co/docs

Last Updated¶

2026-09-06 — Initial version.

Generative AI¶

Table of Contents¶

Installation / Setup¶

Core Syntax¶

Model families at a glance¶

Core concepts that appear everywhere¶

Prompting anatomy (messages format)¶

Common Operations¶

Temperature and sampling¶

Tokens and context windows¶

Structured output (JSON mode)¶

Embeddings¶

Streaming¶

Function / tool calling¶

Practical Examples¶

Example 1 — Build a text summarizer¶

Example 2 — Structured extraction¶

Example 3 — Semantic search with embeddings¶

Common Patterns¶

Pattern: multi-turn conversation¶

Pattern: guardrails via system message¶

Pattern: retries with exponential backoff¶

Best Practices¶

Common Mistakes¶

Troubleshooting¶

Related Cheat Sheets¶

Official Documentation¶

Last Updated¶