💡 Shipping a model isn’t one decision — it’s three decisions that all climb the same ladder (🧪 Dev → 🎭 Staging → 🏭 Prod): how you serve predictions, what you promote, and how you shift traffic. Mix them wrong and you get downtime, silent regressions, or a GPU bill that eats the roadmap. Mix them right and you get boring, reversible releases.

🗺️ The three-axis map

Most “deployment strategy” posts collapse these axes. Keep them separate:

1. 🍽️ Serving mode — batch, online, streaming, edge. Answers: how fresh must the prediction be?
2. 📦 Promotion pattern — deploy code vs deploy the model artifact. Answers: what moves toward production?
3. 🚦 Traffic / release strategy — recreate, rolling, blue/green, canary, linear, A/B, shadow, champion–challenger. Answers: how do users meet the new version?

flowchart TB
  subgraph SERVE["🍽️ Serving mode"]
    B["📦 Batch"]
    O["⚡ Online"]
    S["🌊 Streaming"]
    E["📱 Edge"]
  end
  subgraph PROMOTE["📦 What you promote"]
    C["🧩 Deploy code"]
    M["🎁 Deploy model"]
  end
  subgraph TRAFFIC["🚦 How traffic moves"]
    R["💥 Recreate"]
    RO["🔄 Rolling"]
    BG["🔵🟢 Blue/Green"]
    CA["🐦 Canary"]
    LI["📶 Linear"]
    AB["🅰️🅱️ A/B"]
    SH["👻 Shadow"]
    CC["🏆🆚 Challenger"]
  end
  SERVE --> PROMOTE --> TRAFFIC
            

🍽️ Axis 1 — Serving modes

Pick the serving mode from SLA, not from fashion. An LLM chatbot wants online; a nightly churn score wants batch; a fraud check on a clickstream wants streaming; a camera on a forklift wants edge.

📦 Batch predictions

WHAT. Score a table (or lake) on a schedule or trigger. Write predictions to storage; apps read later. Orchestrators: Airflow, Dagster, Prefect, SageMaker Processing / Batch Transform, Databricks Jobs.

WHERE. Recs that can be stale by minutes–hours, risk scores for morning dashboards, offline eval / backtests.

flowchart LR
  DATA["🗄️ Features / table"] --> JOB["🤖 Batch job
⏰ cron"]
  JOB --> MODEL["🧠 Model vN"]
  MODEL --> OUT["📄 Predictions
Parquet / warehouse"]
  OUT --> APP["📱 App / BI
reads later"]
            

• ✅ Cheap, simple rollback (re-run old job), easy offline eval.
• ❌ Stale by definition; not for “user clicked → score now.”

⚡ Online / real-time inference

WHAT. Request → features → model → response in the same call path. REST/gRPC behind FastAPI, TorchServe, Triton, SageMaker real-time endpoints, KServe, BentoML, etc.

WHERE. Rankers in the request path, fraud at checkout, personalized UI that must feel live.

flowchart LR
  USER["👤 User"] --> GW["🚪 API gateway"]
  GW --> LB["⚖️ Load balancer"]
  LB --> SVC["🛎️ Serving pods
🧠 model in memory"]
  SVC --> FS["🧩 Feature store
optional"]
  SVC --> USER
            

• ✅ Low latency; natural home for canary / A/B / shadow.
• ❌ Always-on cost; cold starts and model size matter a lot.

🌊 Streaming / near-real-time

WHAT. Consume events (Kafka, Kinesis, Pub/Sub), score asynchronously, write results to a store; a thin API serves the precomputed score. Hybrid of batch freshness + online read path.

WHERE. High-QPS entity scores that update every few seconds–minutes without blocking the user request on the model.

flowchart LR
  EVT["📨 Event stream"] --> CONS["👂 Consumer
🧠 model"]
  CONS --> DB["🗄️ Score store"]
  USER["👤 User"] --> API["⚡ Thin API"]
  API --> DB
  API --> USER
            

• ✅ Decouples inference cost from request latency.
• ❌ Eventual consistency; harder debugging + backpressure.

📱 Edge / on-device

WHAT. Ship a quantized / compiled artifact to phones, browsers, gateways, robots. Inference happens offline or with tiny round-trips.

WHERE. Privacy, offline UX, camera/IoT with bandwidth or latency constraints.

flowchart LR
  REG["🏛️ Model registry"] --> OTA["📡 OTA / app update"]
  OTA --> DEV["📱 Device
🧠 local model"]
  SENSOR["📷 Sensor"] --> DEV
  DEV --> UX["✨ Local UX"]
            

• ✅ Privacy + offline; no per-request GPU bill.
• ❌ Fragmented fleet versions; hard shadow traffic; slow rollbacks.

📦 Axis 2 — Deploy code vs deploy model

Databricks frames this cleanly [5]: models and training code move asynchronously. Weekly fraud retrain may change weights without touching serving code; a DNN may retrain rarely while monitoring code ships weekly.

🧩 Deploy code (usually recommended)

Training / pipeline code is reviewed, tested, and promoted. The model is retrained in each environment (dev → staging subset → prod full data). Same code, environment-specific data.

flowchart LR
  DEV["🧪 Dev
train + experiment"] --> STG["🎭 Staging
train on sample
integration tests"]
  STG --> PROD["🏭 Prod
train on full data
register champion"]
  PROD --> SERVE["🛎️ Serve"]
            

• ✅ Safer automated retrain; works when prod data cannot leave prod.
• ❌ Steeper handoff for DS; need templates + review culture.

🎁 Deploy model artifact

Train once (often in dev), promote the binary / MLflow run / model package through staging → prod. Prefer when training is ruinously expensive or everything lives in one workspace with no real CI.

flowchart LR
  DEV["🧪 Dev
train once 🎁"] --> STG["🎭 Staging
validate artifact"]
  STG --> PROD["🏭 Prod
load same artifact"]
  PROD --> SERVE["🛎️ Serve"]
            

• ✅ Simpler DS handoff; train once.
• ❌ Weak if prod data is sealed away; automated retrain gets awkward; supporting code still needs its own path.

🧪🎭🏭 The environment ladder — Dev → Staging → Prod

Serving mode, promotion pattern, and traffic strategy all climb the same ladder. An environment is not a folder name — it is isolation + data access + blast radius + who can approve a promote. Mature setups often map each rung to a separate cloud account / Unity Catalog / workspace so prod credentials never leak into notebooks [4] [5].

flowchart TB
  subgraph DEV["🧪 Dev"]
    D1["📓 Experiment / notebooks"]
    D2["🏋️ Train candidates"]
    D3["💥 Recreate freely"]
  end
  subgraph STG["🎭 Staging / Test"]
    S1["🔗 Integration + contract tests"]
    S2["⏱️ Latency / load smoke"]
    S3["🔄 Rolling or B/G dry-run"]
  end
  subgraph PROD["🏭 Prod"]
    P1["👻 Shadow · 🐦 Canary · 🅰️🅱️ A/B"]
    P2["🔵🟢 Guardrailed cutover"]
    P3["📡 Monitor → CT / rollback"]
  end
  DEV -->|"✅ PR + CI"| STG
  STG -->|"✍️ Manual or gated auto-approve
in model registry"| PROD
            

🧪 Dev

Job. Learn fast. Break things on purpose.

• 📓 Experimentation, feature ideas, hyperparameter thrash — tracked in an experiment store, not “mystery S3.”
• 💥 Recreate / short-lived endpoints are fine; SLAs are optional.
• 🔐 Use dev-shaped data (samples, synthetic, or tightly scoped prod extracts) — not a second copy of the crown jewels unless policy allows.
• 🎁 Viable candidates get registered (or the training code does) — that is the handoff, not a Slack “works on my laptop.”

🎭 Staging (aka ML test)

Job. Prove the system works before real users pay the price. This is where ops tests live — not where you discover product winners.

• 🧪 Unit + integration + schema/contract tests against the serving API.
• ⏱️ Latency, cold-start, and payload-size smoke under realistic traffic shapes.
• 🔄 Prefer rolling or a cheap blue/green dry-run of the same deploy path you will use in prod — staging should exercise the pipeline, not a one-off manual copy.
• 🚫 Do not treat staging A/B as proof of business lift. Staging traffic is fake; closed-loop metrics need prod [4].

🏭 Prod

Job. Serve users. Change slowly. Roll back faster than you rolled forward.

• 👻 Shadow, 🐦 canary, 📶 linear, 🔵🟢 blue/green, 🅰️🅱️ A/B, and 🏆 champion–challenger belong here — against real distribution and real side effects.
• 🏛️ Registry approval (human or policy) triggers the deploy; CloudWatch / Prometheus alarms own auto-rollback during the bake window [4].
• 📡 Continuous monitoring feeds the next retrain — CT is not the same as auto-promote.
• 🧱 Keep the previous champion warm until the bake ends.

📋 What belongs on each rung

🚦 Axis 3 — Traffic & release strategies

These are mostly borrowed from DevOps / UX, then applied to model variants behind a load balancer or serving mesh [1] [4]. Factors that force a strategy: model size/packaging, retrain cadence, routing needs, and drift [1].

💥 Recreate / Big Bang

WHAT. Tear down the old deployment; stand up the new one. Downtime is a feature, not a bug.

WHERE. Dev sandboxes, throwaway demos, batch jobs you can pause.

sequenceDiagram
  participant 👤 as Users
  participant 🔵 as v1 (live)
  participant 💥 as Tear-down
  participant 🟢 as v2 (new)
  👤->>🔵: traffic
  Note over 🔵,💥: ⛔ downtime window
  💥-->>🔵: RIP
  🟢-->>🟢: boot + load weights
  👤->>🟢: traffic resumes
            

• ✅ Dead simple; great for iterating infra.
• ❌ Unacceptable for customer-facing SLAs.

🔄 Rolling update

WHAT. Replace instances one-by-one (or N-at-a-time). Zero planned downtime; mixed versions briefly coexist.

WHERE. Staging fleets; prod only when you truly want 100% of the new model and trust it.

flowchart TB
  subgraph T0["⏱️ t0"]
    A1["🔵"] --- A2["🔵"] --- A3["🔵"] --- A4["🔵"]
  end
  subgraph T1["⏱️ t1"]
    B1["🟢"] --- B2["🔵"] --- B3["🔵"] --- B4["🔵"]
  end
  subgraph T2["⏱️ t2"]
    C1["🟢"] --- C2["🟢"] --- C3["🟢"] --- C4["🔵"]
  end
  subgraph T3["⏱️ t3"]
    D1["🟢"] --- D2["🟢"] --- D3["🟢"] --- D4["🟢"]
  end
  T0 --> T1 --> T2 --> T3
            

• ✅ No big-bang cutover; easy rollback mid-roll.
• ❌ Transient heterogeneity; bad if v1/v2 outputs must stay comparable in one session.

🔵🟢 Blue / Green

WHAT. Two parallel environments. Live traffic hits blue; green gets the new model. Flip the balancer when green is healthy. Instant rollback = flip back.

WHERE. Prod cutovers where you can afford double capacity for a baking window — SageMaker’s default model-update style is blue/green with guardrails [4].

flowchart LR
  U["👤 Users"] --> LB["⚖️ Balancer"]
  LB -->|"100% live"| BLUE["🔵 Blue
🧠 v1"]
  LB -.->|"0% until flip"| GREEN["🟢 Green
🧠 v2"]
  BLUE <-.->|"🔁 swap"| GREEN
            

• ✅ Near-zero downtime; clean rollback story.
• ❌ ~2× infra during overlap — painful for big GPU models.

📶 Blue/Green traffic-shift flavors

Same blue/green pair; different how fast traffic moves [4]:

• ⚡ All-at-once — 100% flip after green is ready; short baking period; fastest; blast radius = everyone.
• 🐦 Canary size — tiny % to green first, then full flip if alarms stay quiet.
• 📶 Linear / stepped — e.g. 10% → 20% → … with a bake at each step; slowest, safest, priciest.

flowchart TB
  ALL["⚡ All-at-once
0% → 100%"]
  CAN["🐦 Canary
5% bake → 100%"]
  LIN["📶 Linear
10% → 30% → 60% → 100%"]
  ALL --- CAN --- LIN
            

🐦 Canary

WHAT. Expose a small, sticky slice of users/traffic to v2 (often 5–30%, never “half the world”). Goal: prove the new model is operationally healthy before a full rollout [1].

WHERE. Staging + prod when you need live signal without betting the company.

Two implementations:

• 🩹 Canary rolling — patch a few pods inside the same cluster.
• 👯 Canary parallel — spin a small sibling fleet; route a tagged percentage to it.

flowchart LR
  U["👥 Users"] --> LB["⚖️ LB
session affinity"]
  LB -->|"🐥 ~10% sticky"| V2["🟢 Canary
🧠 v2"]
  LB -->|"🦉 ~90%"| V1["🔵 Stable
🧠 v1"]
            

• ✅ Small blast radius; great with latency / 5xx alarms.
• ❌ Not a substitute for statistical product A/B; sticky cohorts can bias results.

🅰️🅱️ A/B testing

WHAT. Randomly assign users/requests to variants to learn which model wins on business / UX metrics (CTR, conversion, retention), not just model loss [1] [4].

WHERE. Recommenders, ranking, personalization — anywhere closed-loop feedback maps predictions → money.

flowchart LR
  U["🎲 Users
random split"] --> A["🅰️ Variant A
🧠 v1"]
  U --> B["🅱️ Variant B
🧠 v2"]
  A --> M["📊 Metrics
CTR / $ / NPS"]
  B --> M
  M --> WIN["🏆 Promote winner"]
            

👻 Shadow / dark launch

WHAT. Mirror production requests to v2; only v1 answers the user. Log v2 outputs for offline compare (quality, latency, errors) [1] [4] [3].

WHERE. High-traffic prod when you lack a clean business feedback loop, or when a wrong answer is catastrophic (payments, medical triage, safety).

flowchart LR
  U["👤 User"] --> LB["⚖️"]
  LB --> V1["🔵 Prod v1
💬 reply"]
  LB -.->|"👻 copy"| V2["🟢 Shadow v2
📝 log only"]
  V1 --> U
  V2 --> LOG["📒 Diff / metrics"]
            

• ✅ Zero user risk; true prod distribution.
• ❌ ~2× inference cost; need careful PII / side-effect handling (shadow must not write money, send email, etc.).

🏆🆚 Champion–Challenger

WHAT. Continuous loop: production champion stays live; new candidates are challengers via shadow and/or A/B; promote only when challenger beats champion on agreed gates [3].

WHERE. Mature platforms with a model registry, automated eval, and ownership of promotion criteria.

flowchart TB
  CH["🏆 Champion
in prod"] --> SH["👻 Shadow / 🅰️🅱️ A/B"]
  NEW["🧪 New candidate"] --> SH
  SH --> GATE{"📊 Beats gates?"}
  GATE -->|"✅ yes"| PROMOTE["🚀 Challenger → Champion"]
  GATE -->|"❌ no"| KEEP["🗑️ Keep champion
archive challenger"]
  PROMOTE --> CH
            

🏭 Generic production shape

Regardless of strategy, real-time stacks usually look like: gateway → load balancer → service mesh / variants → model → feature + metadata stores, with monitoring feeding the next retrain [1] [6].

flowchart TB
  APP["📱 App"] --> GW["🚪 Gateway"]
  GW --> LB["⚖️ LB / mesh
routing rules live here"]
  LB --> V1["🔵 Variant A"]
  LB --> V2["🟢 Variant B"]
  V1 --> FS["🧩 Features"]
  V2 --> FS
  V1 --> REG["🏛️ Registry / artifacts"]
  V2 --> REG
  V1 --> MON["📡 Monitor
drift · latency · $"]
  V2 --> MON
  MON -->|"🔁 CT trigger"| TRAIN["🏋️ Retrain pipeline"]
  TRAIN --> REG
            

🧭 Rules of thumb — when to use what

Start from constraints, not from cool words on a slide.

1️⃣ Pick serving mode first

flowchart TD
  Q1{"⏱️ Need score
in < ~100–300ms
on the request?"}
  Q1 -->|"No"| Q2{"📨 Events
arrive continuously?"}
  Q1 -->|"Yes"| ONLINE["⚡ Online"]
  Q2 -->|"Yes"| STREAM["🌊 Streaming"]
  Q2 -->|"No"| Q3{"📱 Offline / privacy
or no reliable net?"}
  Q3 -->|"Yes"| EDGE["📱 Edge"]
  Q3 -->|"No"| BATCH["📦 Batch"]
            

2️⃣ Pick promotion pattern

• 🔐 Prod data cannot leave prod → deploy code.
• 💰 Training costs a fortune / is barely reproducible → deploy model (or hybrid).
• 🔁 Automated weekly retrain you trust → deploy code + continuous training [6].

3️⃣ Respect the environment ladder

• 🧪 Dev proves ideas; 🎭 staging proves pipelines; 🏭 prod proves users + money.
• 🚫 Never skip staging “because the notebook looked good.”
• 👻 Shadow / 🅰️🅱️ A/B need prod traffic — staging cannot fake that distribution.

4️⃣ Pick traffic strategy

• 🧪 Dev / infra thrash → 💥 Recreate.
• 🎭 Staging “just make it the new default” → 🔄 Rolling.
• 🏭 First prod cutover with spare capacity → 🔵🟢 Blue/Green (all-at-once if low risk).
• ⚠️ New model, scary blast radius, need live ops signal → 🐦 Canary (or B/G + canary size).
• 🐢 Highest caution, willing to pay double longer → 📶 Linear.
• 📈 Optimize business metric with closed loop → 🅰️🅱️ A/B.
• 🛡️ No closed loop / wrong answer is catastrophic → 👻 Shadow first, then canary/B/G.
• 🏛️ Platform with registry + gates → 🏆 Champion–Challenger as the forever loop.

5️⃣ Cheat-sheet matrix

*Rolling has no planned downtime but briefly mixes versions.

6️⃣ Compositions that actually ship

• 👻 Shadow → 🐦 Canary → 🔵🟢 Blue/Green flip — classic “prove parity, prove health, then cut over.”
• 🐦 Canary ops bake → 🅰️🅱️ A/B for product win — don’t A/B a model that 500s.
• 📦 Batch: “canary” = score 5% of keys with v2, compare offline; “blue/green” = dual output tables + swap the read view.
• 📱 Edge: staged store rings / percentage rollout in the app store; shadow is rare (mirror to cloud if you must).

📡 Don’t skip the boring gates

Strategies fail without the MLOps hygiene around them [6] [3]:

• 🏷️ Version code + data + model; registry is the source of truth for “what is live.”
• 🧪 CI validates more than unit tests — data schema, training smoke, inference contract, latency budget.
• 🔁 Continuous Training (CT) is not Continuous Delivery — retrain ≠ auto-promote.
• 📡 Monitor quality, drift, latency, and business KPIs; wire auto-rollback alarms into the traffic strategy [4].
• ↩️ Keep the previous champion warm until the bake window ends.

🪙 My 2 cents

Stop asking “what’s the best deployment strategy?” Ask the questions that force one:

• 🍽️ How fresh must the score be?
• 📦 Can we train where the data lives?
• 🧪🎭🏭 Which rung are we on — and what is allowed to fail there?
• 🛡️ What’s the blast radius if v2 is wrong?
• 📊 Do we have a closed-loop business metric?

Answer those and the strategy almost picks itself:

• 👻 Shadow when wrong answers are fatal
• 🐦 Canary when you need live ops confidence
• 🅰️🅱️ A/B when you need product proof
• 🔵🟢 Blue/green when you need a clean flip
• 📦 Batch when latency is a lie you don’t have to tell

🌱 Treat deployment as something you design across serve → promote → shift — not a last-mile “ship the pickle.”

📚 References

1. Yashaswi Nayak, “ML Model Deployment Strategies,” Towards Data Science, towardsdatascience.com/ml-model-deployment-strategies-72044b3c1410
2. Frank Adams, “The Four Machine Learning Model Deployment Types You Should Know,” Medium, medium.com/@FrankAdams7/…
3. Manuel Martin, “Model Deployment: Types, Strategies and Best Practices,” DagsHub, dagshub.com/blog/model-deployment-types-strategies-and-best-practices
4. Maira Ladeira Tanke et al., “MLOps deployment best practices for real-time inference model serving endpoints with Amazon SageMaker,” AWS Machine Learning Blog, aws.amazon.com/blogs/machine-learning/…
5. Databricks, “Model deployment patterns,” docs.databricks.com/…/deployment-patterns
6. MLOps.org, “MLOps Principles,” ml-ops.org/content/mlops-principles
7. GreenNode, “How to Deploy Machine Learning Models,” greennode.ai/blog/how-to-deploy-machine-learning-models