The limit is not a ceiling on your pod. It is a wall around everyone else's.
Most people learn resources.limits as a number you set so your own pod does not use too much memory. That is true, but it undersells the point. A pod with no memory limit is allowed to grow until it exhausts the node it is scheduled on — and once the node itself is short on memory, the kubelet starts evicting pods to recover, and it does not have to be polite about which ones. A limit turns "this leak took down the node" into "this leak killed one container, restarted it, and nothing else noticed." You are about to watch both versions of the same bug.
You will deploy a small container that deliberately over-allocates memory — the same stress tool Kubernetes' own documentation uses to demonstrate this — first with no limit, then with one. Then you will add a HorizontalPodAutoscaler to a second, well-behaved app, drive real load at it, and watch it add replicas on its own and remove them again once the load stops.
Without a limit, your bug is the node's problem.
You need a free Azure account and the Azure CLI (az), signed in with az login. First time? The 15-minute Set up your machine page covers the account, the installs, and sign-in. Prefer zero installs? Run everything in Azure Cloud Shell (Bash) — az and kubectl are preinstalled and already signed in.
This lab assumes the cluster from Your first AKS cluster. If you already tore that one down, Step 1 recreates it with the same command — one Standard_B2s node on the free control-plane tier, cents per hour. Delete the resource group at the end even if you plan to reuse the cluster for another lab today; you can always recreate it in a couple of minutes.
The cluster
If you still have the cluster from Your first AKS cluster, skip to get-credentials. Otherwise recreate it with the identical command — same node size, same free tier.
# Windows/Git Bash: stop it mangling /subscriptions/... arguments (harmless on macOS/Linux) export MSYS_NO_PATHCONV=1 RG="campux-lab-aks-rg" az group create -n "$RG" -l eastus # same command as the first-cluster lab: free control plane, one small node az aks create -n campux-aks -g "$RG" \ --tier free \ --node-count 1 \ --node-vm-size Standard_B2s \ --enable-managed-identity \ --generate-ssh-keys az aks get-credentials -n campux-aks -g "$RG" kubectl get nodes
kubectl get nodes lists one node, status Ready. AKS ships a metrics-server by default, so kubectl top node and kubectl top pod already work — no extra install.No limit: watch the leak take the node with it
Deploy a pod running stress, told to allocate 250Mi of memory — with no resources block at all, so nothing stops it. Watch memory climb, then watch what happens once it outgrows what the node can spare.
kubectl create namespace leak-demo
cat > leak-unbounded.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
name: leak-unbounded
namespace: leak-demo
spec:
containers:
- name: leak
image: polinux/stress
command: ["stress"]
args: ["--vm", "1", "--vm-bytes", "250M", "--vm-hang", "1"]
EOF
kubectl apply -f leak-unbounded.yaml
kubectl get pod leak-unbounded -n leak-demo -w # Ctrl-C once it is Running
While it runs, watch the node's memory climb and the pod's own report:
# run repeatedly, or in a second Cloud Shell tab
kubectl top node
kubectl top pod -n leak-demo
Standard_B2s node (4 GiB) can comfortably hand out to one pod on top of the system pods already running there. On a small node this single 250Mi allocation, with no ceiling, is enough to push the node into memory pressure — kubectl describe node shows a MemoryPressure condition, and the kubelet may evict other pods on the node to recover, not just this one. That is the entire danger of an unlimited container: its mistake is not contained to itself.kubectl describe node $(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') | grep -A3 Conditions
kubectl describe pod leak-unbounded -n leak-demo | tail -20
Conditions block, or its recent Events, show memory pressure and possibly an eviction — proof this pod's appetite is a node-level event, not a pod-level one. Clean it up before Step 3.kubectl delete pod leak-unbounded -n leak-demo
Requests and limits: the same leak, contained
Now run the identical leak, but give it a memory request (what the scheduler reserves) and a limit (the hard ceiling the kubelet enforces). The container will try to allocate the same 250Mi it did before; this time it cannot get past 100Mi.
cat > leak-limited.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
name: leak-limited
namespace: leak-demo
spec:
containers:
- name: leak
image: polinux/stress
command: ["stress"]
args: ["--vm", "1", "--vm-bytes", "250M", "--vm-hang", "1"]
resources:
requests:
memory: "50Mi"
cpu: "50m"
limits:
memory: "100Mi"
cpu: "250m"
EOF
kubectl apply -f leak-limited.yaml
kubectl get pod leak-limited -n leak-demo -w # Ctrl-C once you see it restart
OOMKilled, then it restarts under the Pod's default restart policy and climbs toward the same limit again — a loop, but a contained one. Confirm the cause: kubectl describe pod leak-limited -n leak-demo shows Last State: Terminated, Reason: OOMKilled and an increasing Restart Count. Meanwhile kubectl top node barely moves, and any other pod on the node is untouched — the exact same bug, now paying only for its own mistake.kubectl describe pod leak-limited -n leak-demo | grep -A6 "Last State" kubectl top node
Same image, same command, same 250Mi ask. Without a limit the kernel had no reason to stop it, so the request kept succeeding until the node itself ran short. With a 100Mi limit, the kubelet enforces a cgroup memory ceiling on that one container: the moment it crosses 100Mi, the kernel OOM-killer inside that cgroup kills the process, Kubernetes reports OOMKilled, and the container restarts — never touching memory that belongs to anything else. This is why every production container needs a memory limit, even a generous one: it converts a shared failure into a private one.
kubectl delete namespace leak-demo
Add a HorizontalPodAutoscaler
Deploy an app with a CPU request set (the HPA needs a request to compute percentage utilization against), then let kubectl autoscale create the HorizontalPodAutoscaler for you — it targets the current stable API, autoscaling/v2.
kubectl apply -f https://k8s.io/examples/application/php-apache.yaml
kubectl get deployment php-apache
kubectl get pod -l run=php-apache
# target 50% average CPU utilization across 1-5 replicas
kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=5
kubectl get hpa php-apache -o yaml | grep apiVersion
kubectl get hpa php-apache shows TARGETS as a current/target percentage (something like 0%/50% at rest) and REPLICAS at 1. The apiVersion line confirms autoscaling/v2, the API this lab was verified against on kubernetes.io.Drive load, watch it scale up — then back down
In one terminal, generate sustained load against the service. In another, watch the HPA react.
# terminal A — hammer the service with requests until you stop it
kubectl run -i --tty load-generator --rm --image=busybox:1.28 --restart=Never -- \
/bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
# terminal B — watch replicas climb as CPU utilization crosses 50%
kubectl get hpa php-apache --watch
TARGETS climbs well past 50% and REPLICAS rises from 1 toward the --max=5 ceiling — the HPA polls metrics every 15 seconds and adds replicas to bring average utilization back down. Stop the load generator (Ctrl-C in terminal A, then confirm the pod is gone: kubectl get pod load-generator). Keep terminal B running: utilization drops toward 0% almost immediately, but REPLICAS holds for a few minutes before it drops — the HPA's default scale-down stabilization window is 300 seconds, so it waits to be sure the drop is real before removing capacity. Give it the full five minutes and it settles back to 1.The autoscaler is a control loop, same as everything else in Kubernetes: it compares current CPU utilization against your 50% target and adjusts desired replica count to close the gap, scaling up almost immediately but scaling down cautiously — a five-minute stabilization window by default — so a brief lull in traffic does not trigger a scale-down right before the next spike. Reading TARGETS and watching REPLICAS lag it is the whole skill; the number you should be able to explain in an interview is that 300-second default, and why it exists.
kubectl delete hpa php-apache kubectl delete -f https://k8s.io/examples/application/php-apache.yaml
Tear it down
One resource group holds the cluster and its node. Delete it.
az group delete -n campux-lab-aks-rg --yes --no-wait
az group exists -n campux-lab-aks-rg # -> false once complete
MC_campux-lab-aks-rg_campux-aks_eastus) are gone.What you can now honestly claim
You ran an unbounded memory leak and watched it threaten a whole node, then ran the identical leak with requests and limits set and watched it get killed on its own, restarted, and contained. You added a HorizontalPodAutoscaler, drove real CPU load, watched replicas climb toward a ceiling, and watched them hold for five minutes before scaling back down — because you now know that number, not because you guessed it. That is the difference between having typed resources.limits once and understanding why every production Deployment carries one.