Skip to content

FastAPI

The modern Python framework for serving models. FastAPI gives you typed request/response validation with pydantic, automatic OpenAPI docs at /docs, and async support — the default choice for ML/AI backends.

Table of Contents

Installation / Setup

pip install fastapi "uvicorn[standard]"     # or: uv add fastapi uvicorn

# main.py
from fastapi import FastAPI
app = FastAPI(title="Model API", version="1.0.0")

@app.get("/health")
def health():
    return {"status": "ok"}

uvicorn main:app --reload --port 8000
# docs:  http://localhost:8000/docs   (Swagger UI)
# alt:   http://localhost:8000/redoc

Core Syntax

Path & query parameters

@app.get("/v1/models/{model_id}")                 # path param
def get_model(model_id: str,
              include_meta: bool = False):        # query param
    return {"model": model_id, "meta": include_meta}

# path params are typed; FastAPI validates & converts (422 on mismatch)
@app.get("/v1/runs/{run_id}")
def get_run(run_id: int): ...                     # "abc" -> 422

Request & response models (pydantic)

from pydantic import BaseModel, Field

class PredictRequest(BaseModel):
    text: str = Field(min_length=1, max_length=2000)
    model: str = "v1"
    top_k: int = Field(default=5, ge=1, le=50)

class PredictResponse(BaseModel):
    label: str
    score: float = Field(ge=0, le=1)

@app.post("/v1/predict", response_model=PredictResponse)
def predict(req: PredictRequest):      # body validated against the schema
    ...
    return PredictResponse(label=label, score=score)

Invalid bodies return 422 with a precise error listing each bad field — clients can fix themselves.

Status codes & errors

from fastapi import HTTPException, status

@app.get("/v1/models/{model_id}")
def get_model(model_id: str):
    model = registry.get(model_id)
    if model is None:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail={"code": "model_not_found", "message": f"no model {model_id}"},
        )
    return model

@app.post("/v1/jobs", status_code=201)     # set success code
def create_job(job: JobIn):
    return job_service.start(job)

Common Operations

Dependency injection (auth, shared clients)

from fastapi import Depends, Header

def require_api_key(x_api_key: str = Header(...)) -> None:
    if x_api_key != API_KEY:
        raise HTTPException(401, "invalid API key")

@app.get("/v1/predictions", dependencies=[Depends(require_api_key)])
def list_predictions(): ...

# reusable model handle — load once, inject everywhere
def get_model():
    if not hasattr(get_model, "model"):
        get_model.model = load_model()          # module-level singleton
    return get_model.model

@app.post("/v1/predict")
def predict(req: PredictRequest, model=Depends(get_model)):
    return run(model, req.text)

CORS (allow browser front ends)

from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:5173"],   # or ["*"] for demos
    allow_methods=["*"],
    allow_headers=["*"],
)

Async & background tasks

import asyncio

@app.post("/v1/predict")
async def predict(req: PredictRequest):         # async def = no thread block
    text = await preprocess_async(req.text)     # async I/O inside
    return run_inference(text)

# fire-and-forget work (e.g. logging)
from fastapi import BackgroundTasks

@app.post("/v1/jobs")
def create_job(job: JobIn, bg: BackgroundTasks):
    job_id = jobs.create(job)
    bg.add_task(jobs.execute, job_id)           # runs after response
    return {"job_id": job_id}

Static files & file responses

from fastapi.staticfiles import StaticFiles
app.mount("/static", StaticFiles(directory="static"), name="static")

from fastapi.responses import FileResponse
@app.get("/v1/download/{name}")
def download(name: str):
    return FileResponse(f"artifacts/{name}", filename=name)

Practical Examples

1. Single-model text classifier

import pickle, time
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title="Sentiment API")

with open("model.pkl", "rb") as f:      # load at startup
    model = pickle.load(f)

class Req(BaseModel):
    text: str

class Res(BaseModel):
    label: str
    confidence: float

@app.post("/v1/predict", response_model=Res)
def predict(req: Req):
    label, conf = model.predict_proba([req.text])[0]
    return Res(label=label, confidence=conf)

2. Load model lazily (fast startup, dev-friendly)

model_handle = None

def get_model():
    global model_handle
    if model_handle is None:
        model_handle = load_model()     # heavy init once
    return model_handle

@app.get("/ready")
def ready():
    return {"ready": get_model() is not None}

3. Versioned router

from fastapi import APIRouter
v1 = APIRouter(prefix="/v1")

@v1.get("/predict")
def predict_v1(...): ...

app.include_router(v1)
# new breaking change -> v2 = APIRouter(prefix="/v2")

Common Patterns

# Lifespan context: load & release resources once
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app):
    app.state.model = load_model()         # startup
    yield
    # shutdown cleanup

app = FastAPI(lifespan=lifespan)

# Union/optional fields, defaults
class Req(BaseModel):
    text: str
    lang: str = "en"                       # optional with default
    extra: dict | None = None              # optional dict

Best Practices

  • Type everything: request/response pydantic models double as docs and validation.
  • Load heavy models once (module scope / lifespan / Depends singleton) — not per request.
  • Use response_model to guarantee the output shape and strip secrets.
  • Return proper status codes + JSON errors; never leak stack traces.
  • Prefix routes with /v1/ and group with APIRouter.
  • Add /health and /ready for orchestrators (see rest-api.md).
  • Background long inference as jobs (202 + status endpoint) instead of blocking.
  • Let uvicorn workers handle CPU-bound inference: run with --workers 4 (each worker loads the model).
  • Pin dependencies and test clients with fastapi.testclient.

Common Mistakes

Mistake Consequence Fix
Loading the model inside every request Milliseconds of latency per call Load once
def (sync) endpoints doing blocking CPU work Thread pool bottleneck Understand sync vs async; scale workers
No response_model Unstable output / leaking internals Declare it
Returning raw dicts for errors Inconsistent errors HTTPException with detail
CORS not configured Browser front ends fail CORSMiddleware
Mutable default in pydantic fields Shared state bugs Field(default_factory=list)
Serving on 127.0.0.1 in Docker Unreachable container --host 0.0.0.0
No versioning Breaking client contracts /v1/ prefix from the start

Troubleshooting

Symptom Cause Fix
422 on every request Body doesn't match schema Read the detail array; check names/types
ImportError on uvicorn Not installed in active env pip install uvicorn in the venv
Slow first request Lazy model load Preload at startup; --workers
Reload not working --reload missing Restart uvicorn with --reload
Docs empty Wrong URL /docs (Swagger) or /redoc
500 with no detail in prod Exception unhandled Add exception handlers / log tracebacks

Official Documentation

Last Updated

2026-09-06 — Initial version.