Skip to content

MLOps

MLOps (Machine Learning Operations) is the practice of reliably and efficiently deploying and maintaining ML systems in production. This sheet covers the end-to-end ML lifecycle.

Table of Contents

Installation / Setup

pip install mlflow dvc great-expectations

Core Syntax

The ML Lifecycle

Data → Training → Evaluation → Deployment → Monitoring → Retraining

┌─────────────────────────────────────────────────────────────┐
│                      ML LIFECYCLE                            │
├──────────┬──────────┬──────────┬──────────┬─────────────────┤
│   Data   │ Training │ Evaluate │ Deploy   │ Monitor         │
│          │          │          │          │                 │
│ - Ingest │ - Train  │ - Metrics│ - Serve  │ - Drift         │
│ - Clean  │ - Tune   │ - Compare│ - Batch  │ - Performance   │
│ - Split  │ - Track  │ - Validate│ - API   │ - Alerts        │
│ - Version│ - Log    │ - Sign   │ - Shadow │ - Retraining    │
└──────────┴──────────┴──────────┴──────────┴─────────────────┘

Key Concepts

Concept Description Tools
Experiment Tracking Log params, metrics, artifacts MLflow, W&B, DVC
Model Registry Version models, stage transitions MLflow Registry
Data Versioning Track dataset versions DVC, LakeFS
Model Serving Deploy models for inference FastAPI, BentoML, SageMaker
Monitoring Track model performance in production Evidently, Whylabs
CI/CD Automated testing and deployment GitHub Actions, Airflow
Feature Store Centralized feature management Feast, Tecton
Reproducibility Repeatable experiments DVC, Docker, Poetry

MLOps Maturity Levels

Level Description Characteristics
0 Manual Manual training, manual deployment
1 Pipeline Automated training pipeline
2 CI/CD Automated testing and deployment
3 Full MLOps Continuous training, monitoring, retraining

Common Operations

Experiment Tracking

import mlflow

# Start experiment
mlflow.set_experiment("my_experiment")

# Log parameters
mlflow.log_param("learning_rate", 0.001)
mlflow.log_param("epochs", 10)

# Log metrics
mlflow.log_metric("accuracy", 0.95)
mlflow.log_metric("loss", 0.05)

# Log model
mlflow.sklearn.log_model(model, "model")

# Log artifacts
mlflow.log_artifact("confusion_matrix.png")

Model Versioning

# Register model
model_uri = "runs:/run_id/model"
mlflow.register_model(model_uri, "my_model")

# Transition model stage
client = mlflow.tracking.MlflowClient()
client.transition_model_version_stage(
    name="my_model",
    version=1,
    stage="production"
)

Data Versioning

# Initialize DVC
dvc init

# Track data
dvc add data/training.csv
git add data/training.csv.dvc

# Pull data
dvc pull

# Push data
dvc push

Model Deployment

# FastAPI deployment
from fastapi import FastAPI
import mlflow

app = FastAPI()
model = mlflow.pyfunc.load_model("models:/my_model/production")

@app.post("/predict")
def predict(data: dict):
    return model.predict(data)

Monitoring

# Evidently monitoring
from evidently.report import Report
from evidently.metric_preset import DataDriftPreset

report = Report(metrics=[DataDriftPreset()])
report.run(reference_data=train_data, current_data=production_data)
report.show()

Practical Examples

Example 1 — Full MLOps workflow

import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# 1. Load data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# 2. Start experiment
mlflow.set_experiment("churn_prediction")

with mlflow.start_run():
    # 3. Train model
    model = RandomForestClassifier(n_estimators=100, max_depth=10)
    model.fit(X_train, y_train)

    # 4. Evaluate
    y_pred = model.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)

    # 5. Log everything
    mlflow.log_param("n_estimators", 100)
    mlflow.log_param("max_depth", 10)
    mlflow.log_metric("accuracy", accuracy)
    mlflow.sklearn.log_model(model, "model")

    print(f"Accuracy: {accuracy}")

Example 2 — Model registry workflow

import mlflow

# Train and log model
with mlflow.start_run():
    model.fit(X_train, y_train)
    mlflow.sklearn.log_model(model, "model")
    run_id = mlflow.active_run().info.run_id

# Register model
model_uri = f"runs:/{run_id}/model"
mlflow.register_model(model_uri, "churn_model")

# Transition to staging
client = mlflow.tracking.MlflowClient()
client.transition_model_version_stage("churn_model", 1, "staging")

# Transition to production
client.transition_model_version_stage("churn_model", 1, "production")

Example 3 — CI/CD pipeline

# .github/workflows/ml-pipeline.yml
name: ML Pipeline
on:
  push:
    branches: [main]

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Train model
        run: python train.py
      - name: Run tests
        run: pytest tests/
      - name: Deploy
        if: success()
        run: python deploy.py

Common Patterns

Pattern: shadow deployment

# Deploy new model alongside old one
# Compare predictions before switching

# Shadow endpoint
@app.post("/predict/shadow")
def predict_shadow(data: dict):
    old_pred = old_model.predict(data)
    new_pred = new_model.predict(data)

    # Log comparison
    log_comparison(old_pred, new_pred)

    return {"prediction": old_pred}  # return old model's prediction

Pattern: canary deployment

# Gradually route traffic to new model
import random

@app.post("/predict")
def predict(data: dict):
    if random.random() < canary_percentage:
        return new_model.predict(data)
    return old_model.predict(data)

Pattern: rollback

# Keep previous model version for rollback
def rollback_model(model_name):
    client = mlflow.tracking.MlflowClient()
    versions = client.get_latest_versions(model_name, stages=["production"])
    if len(versions) > 1:
        # Transition previous version back to production
        client.transition_model_version_stage(
            model_name, versions[1].version, "production"
        )

Best Practices

  • Track everything — params, metrics, artifacts, data versions.
  • Version everything — code, data, models, configurations.
  • Automate training — pipelines, not manual runs.
  • Test before deploying — unit tests, integration tests, data validation.
  • Monitor in production — drift, performance, latency.
  • Use model registry — stage transitions, approval workflows.
  • Implement rollback — keep previous versions ready.
  • Document experiments — names, descriptions, results.

Common Mistakes

Mistake Fix
Manual training Automate with pipelines
Not tracking experiments Use MLflow or W&B
No data versioning Use DVC or LakeFS
Deploying without tests Add validation gates
No monitoring Implement drift detection
Hardcoding hyperparameters Log and tune systematically

Troubleshooting

Symptom Cause / Fix
Experiments not tracked Check MLflow server; check logging code
Model not found in registry Check model name; check stage
Deployment fails Check dependencies; check model format
Drift not detected Check monitoring config; check data quality
Reproducibility issues Pin versions; track data versions

Official Documentation

Last Updated

2026-09-06 — Initial version.