Skip to the document
Madhuopen lab
The Kubernetes Ninja PathTrack 1 — Kubernetes from the ground up

Chapter 14

Troubleshooting Like a Ninja, Tooling, Microservices & Your CKA Exam Playbook

8 min read read1,937 wordsAdvanced6 recall cards

Before you read, guess

What is the most important debugging habit when a Pod misbehaves?

Take ten seconds and guess — even a wrong guess makes the answer stick. Tap to see where the chapter lands, or just read on.

When a Pod misbehaves: check get pods STATUS, then always describe pod and read Events — the single most important debugging habit in this whole series.

Thirteen posts ago you barely remembered what a container was. Since then you've built a mental model of the control plane, deployed Pods, wired up Services and Ingress, locked things down with RBAC, packaged releases with Helm, and run StatefulSets and Jobs. Today isn't a new object to learn — it's turning everything you've read into reflexes, and pointing those reflexes straight at the CKA exam.

Troubleshooting Toolkit

Every post in this series quietly seeded a debugging habit. Let's pull all of it into one reference for when something's on fire.

Start with the STATUS column

Almost every investigation starts with kubectl get pods. A Pod is Kubernetes' smallest deployable unit — one or more containers running together on one machine. The STATUS column tells you which part of a Pod's life it's stuck in.

$ kubectl get pods
NAME                       READY   STATUS             RESTARTS   AGE
payments-7f8d9c-abc12      0/1     Pending            0          2m
inventory-6b5f4d-xyz98     0/1     CrashLoopBackOff   4          6m
notifications-9c8b7-qwe11  0/1     ImagePullBackOff   0          1m
worker-job-lm4kp           0/1     Error              1          3m
frontend-5d4c3b-jkl77      1/1     Running            0          10m
db-migrate-8f7e6-zzz01     0/1     Completed          0          15m

Pending — not scheduled yet; usually no node has room, or a rule is keeping it out (a Post 9-style scheduling problem). CrashLoopBackOff — the container starts and dies, repeatedly, waiting longer between each try; the app itself is failing. ImagePullBackOff — can't download the image: a typo, a missing pull secret, or a bad tag. Error — the container ran and exited non-zero. Running — healthy, as long as READY also shows the full count (like 1/1). Completed — exited 0 on purpose, normal for a Job (Post 13's "run once, not forever" object).

Describe the Pod — always

If there's one habit to walk away with, it's this: when a Pod misbehaves, run kubectl describe pod and read the Events section at the bottom before anything else. Not logs, not exec, not guessing — describe, first, every time.

$ kubectl describe pod inventory-6b5f4d-xyz98
...
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  6m                 default-scheduler  Successfully assigned default/inventory to worker-2
  Normal   Pulled     6m                 kubelet            Successfully pulled image "inventory:v3"
  Warning  BackOff    30s (x12 over 5m)  kubelet            Back-off restarting failed container

Events is a timeline of everything that happened to this Pod: scheduling, image pulls, failed health checks, out-of-memory kills, mount failures. It's the richest source of truth in the cluster — and the thing rushed engineers skip, right when they need it most.

Analogy: Picture a hospital room. kubectl get pods is glancing at the vitals monitor — one word, one beep. kubectl describe pod is picking up the chart at the foot of the bed. Mapped directly:
  • STATUS column = the one-word vitals-monitor reading.
  • Events section = the full written chart, in order.
  • A doctor never diagnoses from the beeping alone — don't diagnose a Pod from STATUS alone either.

Logs — including the ones you'd normally miss

kubectl logs shows a container's stdout and stderr (its normal output and error messages). But if the container already crashed and restarted, this shows the logs of the current attempt by default — which may be empty, telling you nothing about why the last one died.

$ kubectl logs inventory-6b5f4d-xyz98
(empty — the new attempt just started)

$ kubectl logs inventory-6b5f4d-xyz98 --previous
panic: failed to connect to database: connection refused

--previous pulls logs from the last terminated instance, where the real crash reason usually lives. If a Pod holds multiple containers — the sidecar pattern from Post 3, a helper container riding alongside your app — name the one you want:

$ kubectl logs app-pod -c log-shipper

Exec in, and widen the view

To look inside a running container's filesystem or environment, exec straight in — the same shell instinct you already have from years of Linux work:

$ kubectl exec -it inventory-6b5f4d-xyz98 -- sh
/ # env | grep DB_HOST

When the symptom is vague and you don't even know which Pod is the problem, pull the cluster-wide event stream sorted by time:

$ kubectl get events --sort-by=.metadata.creationTimestamp
LAST SEEN   TYPE      REASON              OBJECT                MESSAGE
90s         Warning   Unhealthy           pod/checkout-9f8d7    Liveness probe failed: HTTP 500

Check resource pressure with kubectl top pods / kubectl top nodes — ties back to Post 9's requests and limits (what a Pod is guaranteed, and what it's capped at). To isolate "is it the app or the network," port-forward straight to a Pod, bypassing its Service:

$ kubectl port-forward pod/checkout-9f8d7 8080:80

If the app responds fine on localhost:8080 but not through the Service, the app is innocent. A Service (Post 5) is just a stable name that routes traffic to the right Pods — so the problem is in that routing layer, not your code.

Try it yourself: Deploy a Pod, then deliberately break it — a bad database host, a bad image tag. Run the full sequence cold: get pods, describe pod, logs --previous if it crashed, port-forward to confirm reachability. Do it until it's boring — boring means muscle memory.

Tools for Daily Work

Raw kubectl is what you'll use on the exam and should understand deeply. Once you're back in a job, the right tooling on top makes day-to-day work faster.

Lens is a desktop app that reads your kubeconfig (kubectl's connection file) and gives you a clickable, visual view of the cluster — filterable Pod lists, live log streams, an exec terminal, resource graphs. Great for building intuition and for fast daily ops once you're back in a role.

k9s is the middle ground: a terminal UI, not a desktop app, for navigating Pods and logs with keyboard shortcuts at speed. Popular with engineers who live in the terminal but want something faster than raw kubectl.

Both are tools you reach for after certification, not exam tools.

Exam trap: Neither Lens nor k9s — nor any GUI or extra tooling — is available during the CKA exam. It's a bare terminal with kubectl and standard Linux tools. Get fluent with raw kubectl first; treat these as post-certification upgrades.

Microservices Architecture Patterns

Step back and look at what you've actually built. Microservices means many small, independently deployable services talking over the network instead of one big program — and nearly everything in this series exists to make that manageable. Deployments (Post 4, a set of identical Pod replicas kept running) scale each service independently. Services and cluster DNS (Post 5, a stable name every Pod can look up) give every service an address to call, no hardcoded IPs. ConfigMaps and Secrets (Post 6, config and credentials from outside the image) let each service carry its own settings. Namespaces (Post 8, walls dividing one cluster into areas) organize dozens of services by team or domain. Kubernetes and microservices grew up together — the toolkit fits almost too well.

Sidecar, ambassador, adapter

Post 3's sidecar pattern — a helper container riding alongside your main app, sharing its network and storage — branches into three named variations. A plain sidecar is general-purpose help, like a container that tails a log file and ships it elsewhere. An ambassador is a sidecar that proxies your main container's outbound calls — handling retries to an external service, so your app just calls localhost. An adapter is a sidecar that standardizes your main container's output — transforming app-specific logs or metrics into a standard format, without touching the app's code. All three share one shape: main container plus small helper, deployed and scaled together.

Service mesh — awareness, not mastery

Once you have dozens of microservices, hand-building a sidecar on every one for retries, encryption, and observability stops scaling. A service mesh — Istio and Linkerd are the well-known ones — automates that: it injects a sidecar proxy next to every service automatically, so encryption, retries, and observability just happen everywhere, uniformly. This is senior-level territory you'll grow into — just recognize the name and the shape of the problem it solves for now.

Your CKA Exam Playbook

The CKA is a hands-on, timed, terminal-only performance exam — not multiple choice. You get real broken or incomplete clusters and solve real tasks live, the same way you've practiced all series.

Built-in help

You will forget exact YAML fields under pressure. kubectl explain is documentation baked into your exam terminal:

$ kubectl explain deployment.spec.strategy.rollingUpdate

Pair it with the --dry-run=client -o yaml trick from Post 3 — generate a skeleton fast, edit only what's different:

$ kubectl run web --image=nginx --dry-run=client -o yaml > pod.yaml

Shave every second off your typing

Alias kubectl and enable completion the moment your terminal opens — ordinary shell config, explicitly allowed, saving real minutes across dozens of questions:

$ alias k=kubectl
$ source <(kubectl completion bash)

Check context and namespace — every time

A context is a saved "which cluster, which user, which namespace" combo that kubectl uses by default. Exam tasks target specific contexts and namespaces, and grading checks the exact target:

$ kubectl config use-context cluster1-admin
$ kubectl config set-context --current --namespace=finance
Exam trap: Flawless YAML in the wrong context or namespace scores zero — the grader checks a specific target and won't find your correct object sitting elsewhere. Run kubectl config current-context before every task, even when you're sure. Five seconds prevents the most avoidable way to lose points.

Manage the clock

Skim every question first, note point values, do the fast/easy ones first to bank points, and don't let one hard question burn disproportionate time — flag it and move on. Practice on killer.sh, the official simulator included with CKA registration, before exam day. And yes, the official Kubernetes docs at kubernetes.io are allowed as reference — knowing exactly where to find something fast matters as much as memorizing it.

Try it yourself: Set a 20-minute timer. Pick three topics — RBAC, a multi-container sidecar Pod, a Deployment with resource limits — and build all three using only kubectl explain, --dry-run=client -o yaml, and your alias/completion setup. No copy-pasting. This is the closest rehearsal for exam-day pressure.

Key Takeaways

  • Across 14 posts you've covered the full arc: containers and the reconciliation loop, control plane and node architecture, Pods, Deployments/ReplicaSets, Services/DNS, ConfigMaps/Secrets, Volumes, Namespaces/Labels, Scheduling/Probes/Resources, Ingress, RBAC, Helm, StatefulSets/Jobs/CronJobs, and now troubleshooting, tooling, microservices patterns, and exam strategy.
  • When a Pod misbehaves: check get pods STATUS, then always describe pod and read Events — the single most important debugging habit in this whole series.
  • kubectl logs --previous recovers logs from a crashed container; -c <container> targets one container in a multi-container Pod; get events --sort-by and port-forward round out the toolkit.
  • Lens (desktop GUI) and k9s (terminal UI) speed up daily operations after certification — neither is available during the CKA exam.
  • Kubernetes's core building blocks map directly onto microservices needs; sidecar, ambassador, and adapter are three shapes of the same helper-container idea, and a service mesh automates that pattern across an entire fleet.
  • On exam day: use kubectl explain and --dry-run=client -o yaml, alias kubectl and enable completion, always verify context/namespace, triage questions by time, and practice on killer.sh beforehand.

You've been rebuilding this skill from scratch while life pulled you in other directions, and that's not small — it's the hardest way to relearn something, and you did it anyway, one post at a time. What you're carrying now isn't a stack of half-remembered blog posts; it's the same reflexes — describe first, check the context, read the Events — that the engineer next to you in an interview or a 2am incident is relying on too. You made it. Go pass that exam.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?