Google Gemini API¶

Google's multimodal LLM family. Gemini natively handles text, images, audio, and video in a single API call. It supports grounding (connecting to Google Search), function calling, and large context windows (1M+ tokens for Gemini 1.5 Pro).

Table of Contents¶

Installation / Setup
Core Syntax
Common Operations
Practical Examples
Common Patterns
Best Practices
Common Mistakes
Troubleshooting
Related Cheat Sheets
Official Documentation

Installation / Setup¶

pip install google-genai                  # or: uv add google-genai
export GEMINI_API_KEY="YOUR_API_KEY"     # never commit real keys

from google import genai
client = genai.Client(api_key="YOUR_API_KEY")

Core Syntax¶

Generate content (the core endpoint)¶

response = client.models.generate_content(
    model="gemini-2.0-flash",              # or gemini-2.5-pro, gemini-2.5-flash
    contents="Explain machine learning in one paragraph.",
)
print(response.text)

Models available¶

Model	Context	Best for	Cost
`gemini-2.5-pro`	1M	Complex reasoning, analysis	$$$
`gemini-2.5-flash`	1M	Fast, good quality, multimodal	$
`gemini-2.0-flash`	1M	Fastest, most cost-effective	$

Messages / chat format¶

from google.genai import types

# single turn
response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents=[
        types.Content(role="user", parts=[types.Part.from_text("Hello!")]),
    ],
    config=types.GenerateContentConfig(
        system_instruction="You are a helpful assistant.",
        temperature=0.7,
        max_output_tokens=1024,
    ),
)

# multi-turn
chat = client.chats.create(model="gemini-2.0-flash")
response = chat.send_message("What is Python?")
print(response.text)
response = chat.send_message("What are its main features?")
print(response.text)

Vision (image understanding)¶

from google.genai import types

# from URL
response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents=[
        types.Part.from_uri("https://example.com/photo.jpg", mime_type="image/jpeg"),
        "What is in this image?",
    ],
)

# from local file
import pathlib
image = pathlib.Path("photo.jpg").read_bytes()
response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents=[
        types.Part.from_bytes(data=image, mime_type="image/jpeg"),
        "Describe this image in detail.",
    ],
)

Video understanding¶

# Gemini can process video directly
video = pathlib.Path("video.mp4").read_bytes()
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=[
        types.Part.from_bytes(data=video, mime_type="video/mp4"),
        "Summarize this video.",
    ],
)

Function calling (tool use)¶

from google.genai import types

def get_weather(city: str) -> dict:
    """Get weather for a city."""
    return {"temperature": 72, "condition": "sunny", "city": city}

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="What's the weather in San Francisco?",
    config=types.GenerateContentConfig(
        tools=[types.Tool(function_declarations=[
            types.FunctionDeclaration(
                name="get_weather",
                description="Get weather for a city",
                parameters=types.Schema(
                    type=types.Type.OBJECT,
                    properties={
                        "city": types.Schema(type=types.Type.STRING, description="City name"),
                    },
                    required=["city"],
                ),
            ),
        ])],
    ),
)

# check for function calls
for part in response.candidates[0].content.parts:
    if part.function_call:
        print(f"Function: {part.function_call.name}")
        print(f"Args: {dict(part.function_call.args)}")

Streaming¶

for chunk in client.models.generate_content_stream(
    model="gemini-2.0-flash",
    contents="Write a haiku about coding.",
):
    print(chunk.text, end="", flush=True)

Embeddings¶

result = client.models.embed_content(
    model="gemini-embedding-exp-03-07",
    contents=["Hello world", "Testing embeddings"],
)
print(result.embeddings[0].values)   # list of floats

Grounding (connect to Google Search)¶

from google.genai import types

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="What happened in AI research this week?",
    config=types.GenerateContentConfig(
        tools=[types.Tool(google_search=types.GoogleSearch())],
    ),
)
# response.candidates[0].grounding_metadata contains search results

JSON mode¶

from google.genai import types

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="List 3 programming languages with their year of creation.",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
    ),
)
# response.text is valid JSON

Practical Examples¶

Example 1 — Multimodal document analysis¶

import pathlib
from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

def analyze_document(pdf_path, question):
    pdf_bytes = pathlib.Path(pdf_path).read_bytes()
    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=[
            types.Part.from_bytes(data=pdf_bytes, mime_type="application/pdf"),
            question,
        ],
        config=types.GenerateContentConfig(max_output_tokens=4096),
    )
    return response.text

Example 2 — Video summarization¶

import pathlib
from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

def summarize_video(video_path):
    video_bytes = pathlib.Path(video_path).read_bytes()
    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=[
            types.Part.from_bytes(data=video_bytes, mime_type="video/mp4"),
            "Provide a detailed summary with key timestamps.",
        ],
    )
    return response.text

Example 3 — Structured extraction with JSON¶

import json
from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

def extract_structured(text):
    response = client.models.generate_content(
        model="gemini-2.0-flash",
        contents=f"Extract from this text:\n\n{text}",
        config=types.GenerateContentConfig(
            system_instruction="Return JSON with keys: entities (list), summary (string), sentiment (string).",
            response_mime_type="application/json",
        ),
    )
    return json.loads(response.text)

Common Patterns¶

Pattern: chat session with system instruction¶

chat = client.chats.create(
    model="gemini-2.0-flash",
    config=types.GenerateContentConfig(
        system_instruction="You are a data science tutor. Explain concepts simply.",
        temperature=0.7,
    ),
)

r1 = chat.send_message("What is overfitting?")
r2 = chat.send_message("How do I prevent it?")

Pattern: grounding for factual queries¶

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="Latest developments in quantum computing",
    config=types.GenerateContentConfig(
        tools=[types.Tool(google_search=types.GoogleSearch())],
    ),
)
# response.candidates[0].grounding_metadata has source citations

Pattern: batch classification¶

from concurrent.futures import ThreadPoolExecutor

def classify(text):
    response = client.models.generate_content(
        model="gemini-2.0-flash",
        contents=f"Classify as positive/negative/neutral:\n{text}",
        config=types.GenerateContentConfig(temperature=0),
    )
    return response.text.strip()

with ThreadPoolExecutor(max_workers=5) as pool:
    labels = list(pool.map(classify, texts))

Best Practices¶

Use gemini-2.0-flash for most tasks — fast and cheap.
Use gemini-2.5-pro for complex reasoning, analysis, and long documents.
Gemini excels at multimodal — send images, video, and PDFs directly.
Use grounding (GoogleSearch) for factual/current-events queries.
response_mime_type="application/json" for structured output — more reliable than prompt-based.
system_instruction is the equivalent of system messages in other APIs.
Streaming via generate_content_stream for user-facing apps.
Large context (1M tokens) — ideal for long documents and video.

Common Mistakes¶

Mistake	Fix
Using old `google-generativeai` package	Use `google-genai` (newer, recommended)
Not setting `response_mime_type` for JSON	Set it in `GenerateContentConfig` for reliable JSON
Sending uncompressed images	Resize before sending — Gemini handles URLs and bytes
Ignoring grounding for factual queries	Use `GoogleSearch` tool for up-to-date information
Not checking `response.candidates`	Some responses may have empty candidates — check before accessing `.text`
Using Pro for simple tasks	Use Flash for classification, extraction — much cheaper

Troubleshooting¶

Symptom	Cause / Fix
`Invalid API key`	Check `GEMINI_API_KEY` environment variable
`Resource exhausted`	Rate limit — implement backoff
`SAFETY` blocked response	Rephrase the prompt — safety filters may be too strict
Response is empty	Check `response.candidates[0].content` — may be blocked
Slow responses	Use `gemini-2.0-flash` instead of `gemini-2.5-pro`
JSON parse error	Add `response_mime_type="application/json"` to config
Token limit exceeded	Shorten input or use a model with larger context

openai-api — OpenAI alternative
anthropic-api — Claude alternative
generative-ai.md — AI landscape overview
litellm — unified multi-provider API
llm-concepts — LLM fundamentals
FastAPI — serving LLM apps
JSON — structured output format

Official Documentation¶

Google AI docs — https://ai.google.dev/docs
Gemini API reference — https://ai.google.dev/api
Function calling — https://ai.google.dev/gemini-api/docs/function-calling

Last Updated¶

2026-09-06 — Initial version.

Google Gemini API¶

Table of Contents¶

Installation / Setup¶

Core Syntax¶

Generate content (the core endpoint)¶

Models available¶

Messages / chat format¶

Vision (image understanding)¶

Video understanding¶

Function calling (tool use)¶

Streaming¶

Embeddings¶

Grounding (connect to Google Search)¶

JSON mode¶

Practical Examples¶

Example 1 — Multimodal document analysis¶

Example 2 — Video summarization¶

Example 3 — Structured extraction with JSON¶

Common Patterns¶

Pattern: chat session with system instruction¶

Pattern: grounding for factual queries¶

Pattern: batch classification¶

Best Practices¶

Common Mistakes¶

Troubleshooting¶

Related Cheat Sheets¶

Official Documentation¶

Last Updated¶