Kubernetes (often shortened to K8s) is an open-source system for automating the deployment, scaling, and operation of containerized applications across a fleet of machines. It grew out of Google's internal cluster manager, Borg, was open-sourced in 2014, and is now governed by the Cloud Native Computing Foundation — the de facto standard for running containers at scale.
The plain-English version: Kubernetes decides which of your servers runs which containers, restarts them when they die, scales them under load, wires them together over the network, and rolls out new versions without downtime — all from a description of the desired state you hand it. It turns a pile of machines into a single pool of compute you deploy against.
This guide is for engineering leaders, not a certification exam. We run Kubernetes in production for clients, so we will be blunt about where it is the wrong tool and how the bill and operational load surprise people. Want that read on your own stack? Our cloud consulting team does exactly this work.
Key takeaways
- Kubernetes automates running containers across many machines — scheduling, self-healing, scaling, networking, and zero-downtime rollouts from one desired-state description.
- It came out of Google (based on Borg), was open-sourced in 2014, and is now the industry-standard container orchestrator under the CNCF.
- The core idea is declarative reconciliation: you declare what should be true, and a control loop continuously makes reality match.
- It is genuinely the wrong choice for many small teams — a PaaS or serverless ships faster and costs far less to operate.
- The real cost is operational burden plus an easily-inflated cloud bill — not a licence fee, because there isn't one.
What problem Kubernetes actually solves
Containers solved packaging: a container bundles an app with its dependencies so it runs the same on a laptop, in CI, and in production. But a container runs on one machine. Real systems are dozens or hundreds of containers that must run across many machines, survive hardware failure, scale with demand, find each other over the network, and update without dropping traffic. Doing that by hand — SSHing into boxes, restarting crashed processes, editing load-balancer configs at 3 AM — does not survive scale.
Kubernetes answers one question: how do I run lots of containers across lots of machines, reliably, without a human in the loop for every event? Under the hood it is a set of control loops. You submit the desired state and Kubernetes reconciles reality toward it — rescheduling when a node dies, adding replicas when load rises, rolling out new versions gradually.
The core concepts, in plain English
You do not need the whole API, but a leader should understand these five ideas — they explain most of Kubernetes' behaviour.
Pods. The smallest unit Kubernetes runs: one or more containers that share a network address and lifecycle. You rarely create pods directly; you describe a higher-level object and let Kubernetes manage the pods for you.
Deployments. A Deployment describes the desired state for a stateless service — “run five replicas of this image.” Its controller creates and replaces pods to match, and handles rolling updates and instant rollbacks when a release goes wrong.
Services. Pods are ephemeral and their IP addresses change on every restart. A Service is a stable address and load balancer in front of a changing set of pods, so clients get one durable endpoint.
Control plane vs nodes. The control plane (API server, scheduler, controllers, etcd) is the brain: it stores the desired state and makes decisions. Nodes are the worker machines that run your pods. You talk to the API server — usually via kubectl — and it drives the nodes; you never hand-place workloads yourself.
Declarative desired state. The defining idea. You do not issue imperative commands like “start this, stop that” — you submit a description of what should be true, and a reconciliation loop makes it so and keeps it so. That is why Kubernetes self-heals: a crashed pod means reality no longer matches the desired state, so the controller quietly fixes it.
Why teams adopt Kubernetes
- Portability. A workload defined in Kubernetes manifests runs the same on any conformant cluster — any cloud, or on-prem. That reduces lock-in and makes hybrid and multi-cloud realistic.
- Self-healing and scaling. Dead pods get replaced automatically; autoscaling adds replicas under load and nodes as needed. Far fewer pages that just say “the process died.”
- A common operating substrate. Once everything speaks Kubernetes, every team shares one deployment model for config, secrets, rollouts, and networking — the foundation most platform engineering builds on, with CNCF building blocks for ingress, observability, policy, and GitOps rather than bespoke scripts.
When you should NOT use Kubernetes
The most expensive Kubernetes decision is adopting it before you need it. We say this as people who run it for a living: for many teams under fifteen engineers, the honest answer is “not yet, maybe not ever.” Reach for something simpler if:
- You have a handful of services and one team. A PaaS (Cloud Run, App Runner, Render, Fly) or serverless ships faster and costs less, with far fewer concepts to learn.
- Your traffic is steady and modest. The elastic scaling and bin-packing that justify Kubernetes buy you nothing — a complexity tax for capabilities you never exercise.
- You have no one to own the platform. Without an engineer who understands it — or a partner who runs it — you have an operational liability, not a solution.
- The real driver is a résumé or a conference talk. “We use Kubernetes” is not a business outcome. If the honest reason is “it's what everyone uses,” stop.
None of this is anti-Kubernetes — it is anti-premature Kubernetes. Adopt it at the right moment and teams love it; too early and you spend years fighting your platform instead of building product.
Managed vs self-hosted: EKS, AKS, and GKE
There are two ways to run Kubernetes, and the choice matters more than almost any other decision you will make about it.
Managed. AWS (EKS), Azure (AKS), and Google Cloud (GKE) run the control plane for you — the brain that is hardest and riskiest to operate. You manage the worker nodes and your workloads; they keep the API server, scheduler, and etcd healthy and upgraded. GKE is the most automated (Autopilot removes node management entirely), EKS integrates deepest with AWS, and AKS fits Azure shops. For almost everyone who needs Kubernetes, managed is the right default.
Self-hosted. Tools like kubeadm and kOps, or a bare-metal build, let you own the control plane too — justified for strict data-residency or air-gapped environments, huge scale, or specialised hardware, but the operational burden never ends. For most teams the decision collapses to one line: pick managed Kubernetes on the cloud you already run. Sizing clusters, controlling node cost, and safe upgrades are what our cloud consulting practice does.
The real cost of Kubernetes
Kubernetes itself is free. The two costs people underestimate are the operational load and the cloud bill.
Operational burden. Someone has to own upgrades — Kubernetes ships a minor version roughly every four months and old versions age out of support fast — plus security patching, RBAC, networking, and the long tail of “why is this pod stuck in Pending.” That is real, ongoing cost, and pretending otherwise is how clusters rot.
The cloud bill. Clusters are notoriously easy to overspend on: over-provisioned node pools, requests set far above usage, always-on non-production environments, a load balancer per service, and unbounded observability data all compound. GPU workloads are the extreme case — an idle H100 is one of the fastest ways to burn money in the cloud, which is why we wrote a full playbook on cutting LLM inference GPU cost.
The fix is treating cost as an engineering property, not a monthly surprise: right-size requests, autoscale to zero where you can, use spot capacity for fault-tolerant jobs, and put the bill in front of the team that generates it. That discipline is what FinOps is about — the difference between a platform that pays for itself and one that quietly doubles every year.
How to decide if you actually need Kubernetes
Run this checklist honestly. If you cannot answer “yes, clearly” to the first three, it is probably “not yet.”
- Count your services and teams. Kubernetes pays off across many services or teams deploying independently, not a monolith and one team.
- Be honest about scaling. Variable or high-volume load justifies elastic scaling and bin-packing; steady traffic does not.
- Check platform capacity. Do you have engineers who can own Kubernetes, or a partner who will? If not, fix that first.
- Evaluate managed first. Price out EKS, AKS, or GKE before considering self-hosting.
- Model the total cost. Compute, idle capacity, load balancers, observability, and engineering time — against your current platform, not a best case.
- Pilot one non-critical service. Prove the operating model on a low-stakes workload before betting production on it.
The migration pitfalls we see most
When a Kubernetes migration goes wrong, it is rarely the technology. It is the approach. The patterns repeat:
- Lift-and-shift with no re-architecting. Cramming a monolith into a pod without rethinking state, config, and health checks moves your problems into a more complex environment, not out of it.
- Skipping resource requests and limits. Without them the scheduler is guessing, and you get noisy-neighbour outages and runaway cost. Features like in-place pod resize in Kubernetes 1.33 help, but you still have to do it.
- No cost guardrails from day one. Teams that add budgets, requests, and autoscaling policy after the first shock bill spend months undoing avoidable waste.
- Treating it as a deployment target, not an operating model. Kubernetes changes how the whole team ships, debugs, and owns production — under-investing in that shift is the biggest predictor of a stalled migration.
We have written up the three failure modes that show up in almost every engagement: 3 Kubernetes migration mistakes →
Not sure whether Kubernetes is your next platform or an expensive detour? InfraZen runs a free 30-minute review that ends in honest advice — whether that is “adopt it, here's how” or “a PaaS will save you a year.” Book the review.
Related: Cloud & Kubernetes consulting · What is DevOps? · What is GitOps? · What is FinOps? · 3 Kubernetes migration mistakes · Cutting LLM inference GPU cost · DevOps vs SRE vs Platform Engineering