Skip to content

DevOps Fundamentals

The practices that ship and operate software — and ML systems — reliably. DevOps is a culture of automation and feedback: small changes, fast validation, reproducible environments, and measurable systems.

Table of Contents

Installation / Setup

DevOps is a practice, not a package. The core local toolchain:

git --version            # version control (20-Git-GitHub)
docker --version         # packaging (17-Docker)
gh --version             # GitHub CLI — PRs, Actions, releases
python3 --version        # scripts & automation glue

Everything below assumes you have git, Docker, and CI (GitHub Actions) in place.

Core Syntax

The core loop

code → commit → CI (lint/test/build) → image → deploy → monitor → feedback
Practice Answers
Version control What changed, when, by whom?
CI (continuous integration) Does every change still pass tests/build?
CD (continuous delivery/deployment) Can every change ship safely?
IaC (infrastructure as code) What exactly is my environment?
Observability Is it working in production?

The DevOps kata: every change through the pipeline

  1. Feature branch + PR (reviewed).
  2. CI runs on the PR: lint → tests → build image → (optionally) deploy to staging.
  3. Green CI + review → merge to main.
  4. CD deploys main (or a tagged release) automatically.
  5. Dashboards/logs confirm health; alerts page humans on anomalies.

Environment model

Environment Purpose Deployed by
Local developer iteration developer
CI automated checks per commit runner
Staging pre-prod validation pipeline
Production serves real users pipeline (guarded)

Each environment gets parity (same container, same config shape) so what works locally works in prod.

Common Operations

Reproducible environments

# Pin everything, everywhere
git tag v1.2.0 && docker build -t app:v1.2.0 .   # code + image pinned
uv lock                                          # python deps pinned (see uv.md)

The feedback loop

git push origin feature/x        # triggers CI on the PR
gh pr checks                    # watch checks
gh run watch                    # live CI logs
gh pr merge --squash            # green + reviewed -> merge

Deploy strategies

Strategy Idea When
Recreate stop old, start new simple services, tiny downtime ok
Rolling new replaces old gradually default for stateless services
Blue/green two full environments, switch traffic zero-downtime, instant rollback
Canary ship to 5% then ramp risky model/LLM changes
Feature flags ship code off by default decouple deploy from release

ML-specific: models should deploy behind versioned APIs so inference is swappable without redeploying code.

Practical Examples

1. Turn a model repo into a deployable service

# repo layout
#   src/train.py  src/serve.py  Dockerfile  pyproject.toml
#   .github/workflows/ci.yml  .github/workflows/deploy.yml

docker build -t inference:v1 .
docker tag inference:v1 ghcr.io/you/inference:v1
docker push ghcr.io/you/inference:v1
# deploy step (any platform): pull + run the image, healthcheck, route traffic

2. Versioned release with artifacts

gh release create v1.2.0 --generate-notes      # tag + notes
# CI attached artifacts (model file, metrics report) automatically

3. Staging parity test

# compose override for staging (same image, different config)
docker compose -f compose.yaml -f compose.staging.yaml up -d
curl -f http://localhost:8000/health && echo "staging healthy"

Common Patterns

# .github/workflows/ci.yml — the minimum viable pipeline
on: [push, pull_request]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -e ".[dev]"
      - run: ruff check .
      - run: pytest

Best Practices

  • Automate the boring parts: formatting, linting, tests run in CI, not by memory.
  • Small, frequent merges beat big-bang releases — every commit should be mergeable.
  • Keep main always green — broken main blocks everyone; fix or revert fast.
  • Make environments reproducible: same container image from dev to prod.
  • Treat infrastructure as code (Terraform, compose files, manifests) — never click-configure prod.
  • Design for rollback (previous image, feature flag) — you will ship a bad release.
  • Measure before you scale: dashboards for latency, errors, throughput, saturation.
  • Blameless postmortems — the process failed, not the person.
  • Start simple: git + tests + one CI workflow + containers beats a tool zoo.

Common Mistakes

Mistake Consequence Fix
No CI on PRs Broken code reaches main Workflow triggers on pull_request
Deploying from a laptop "It worked on my machine" Deploy from CI/registry images only
Config drift between envs Staging green, prod broken Same image + env-file shape everywhere
Testing only in production Users find the bugs Staging parity + canary
Manual release checklist Missed steps under pressure Automate tag → build → deploy
Ignoring logs/metrics until outage Long firefights Dashboards + alerts from day one
chmod 777 / running prod as root Security holes Least privilege, non-root containers

Troubleshooting

Symptom Cause Fix
"Works locally, fails in CI" Env drift / unpinned deps Lock deps; run CI-like environment locally
Merge to main breaks prod No staging/canary Guard deploys; rollback-ready tags
Deploy succeeds but app broken Healthcheck missing or wrong Add real /health checks + post-deploy verification
CI flaky Timing/parallelism Retries, deterministic tests, isolate shared state
Slow deploys Big images / cold starts Slim images, layer caching, warm pools

Official Documentation

Last Updated

2026-09-06 — Initial version.