Amazon EKS Part 1: Architecture, Networking and IAM

The job description asks for two things that rarely live in one person: someone who can run Red Hat OpenShift on-prem, and someone who can "design, deploy, and manage Amazon EKS clusters" with the "networking, IAM integration, security controls" that a bank needs in AWS. The interviewer will not ask you to recite the EKS console. They will probe one question in ten different shapes: do you know exactly what AWS manages, what you manage, and where the seam is? You used EKS years ago. Since then, access entries replaced aws-auth, Pod Identity arrived next to IRSA, Karpenter replaced Cluster Autoscaler on most teams, and EKS Auto Mode appeared. This post brings you current and leaves you able to draw a bank-grade EKS platform on a whiteboard: VPC, subnets, CNI, data plane, authentication, pod IAM, add-ons, ingress, DNS, storage, and the OpenShift-to-EKS translation the interviewer is really testing.

What AWS manages, what you manage: the seam

Everything in EKS follows from one design decision: AWS runs the Kubernetes control plane for you, and you run everything else. Amazon EKS = a managed Kubernetes service where the API servers, etcd, scheduler and controller managers live in an AWS-owned account and VPC, exposed to you only as an HTTPS endpoint. In Post 19 you saw OpenShift ship the whole stack, control plane included, as something you install and own. EKS splits that stack in half.

The managed control plane

When you create a cluster, AWS provisions a dedicated, single-tenant control plane for it: at least two API server instances and a three-node etcd cluster, spread across multiple Availability Zones in the region, auto-scaled and auto-healed by AWS. The control plane is patched by AWS, etcd is backed up by AWS, and the API server certificates are rotated by AWS. You pay a flat hourly fee per cluster for this: as of 2025 it is 0.10 USD per hour (roughly 73 USD a month) while the cluster runs a Kubernetes version in standard support, and 0.60 USD per hour once the version enters extended support. Nodes, load balancers, NAT gateways and volumes are billed separately, as ordinary EC2, ELB and EBS usage.

What you cannot touch matters as much as what you get: no SSH to control plane nodes, no API server flags, no etcdctl, no etcd snapshots, and no control plane pods visible in kube-system (only the components on your nodes appear there: CoreDNS, kube-proxy, the VPC CNI). What you can configure is exposed through the EKS API: the Kubernetes version, the endpoint access mode, the logging types (api, audit, authenticator, controllerManager, scheduler, shipped to CloudWatch Logs), KMS envelope encryption of secrets, the authentication mode, and a small set of add-ons.

The data plane you own

The data plane = the nodes that run your pods, plus the networking and storage they touch. It lives in your VPC, in your account, on your bill. Node operating systems, kubelet versions, instance sizes, autoscaling, the CNI configuration, ingress controllers, CSI drivers, monitoring agents: all of that is your responsibility even when AWS gives you a managed way to install it. This is the seam an interviewer wants you to describe unprompted: "AWS owns availability and patching of the API and etcd; I own everything from the kubelet outward, including keeping node AMIs current and keeping the CNI from running out of IPs."

How the control plane reaches your VPC

The control plane runs in an AWS-owned VPC, so how do API servers talk to your kubelets (for kubectl logs, exec, port-forward) and to admission webhooks running in your cluster? During cluster creation EKS places cross-account elastic network interfaces (ENIs) into at least two of the subnets you specify, in different AZs. In the EC2 console they show up with the description Amazon EKS <cluster-name>, and they belong to AWS even though they use your IPs. Traffic from the API server to nodes flows through these ENIs, protected by the cluster security group that EKS creates and attaches to both the ENIs and, by default, managed node groups. Two practical consequences: those subnets need a few free IPs forever, and if you delete or shrink them the control plane loses its path into the cluster.

The cluster endpoint: public, private, or both

The cluster endpoint is the HTTPS URL your kubeconfig points at, something like https://A1B2C3D4E5F6.gr7.ca-central-1.eks.amazonaws.com. It has three access modes:

• Public (the historical default): the endpoint resolves to public IPs on an AWS-managed load balancer. Anyone on the internet can reach the TLS listener; IAM authentication still stands in the way, but a leaked credential is one step from your API. You can restrict it with a CIDR allow-list (publicAccessCidrs), which is the minimum a serious team does.
• Private: the endpoint resolves only inside your VPC, to the IPs of those EKS-managed ENIs. EKS creates a Route 53 private hosted zone for the endpoint name and associates it with your VPC, which is why the VPC must have enableDnsHostnames and enableDnsSupport turned on. To run kubectl you must be inside the VPC, in a peered or Transit Gateway-attached VPC, or on the corporate network over a VPN or Direct Connect, and your DNS must be able to resolve the private name (a Route 53 Resolver inbound endpoint handles that from on-prem).
• Public and private: nodes and in-VPC clients use the private path; humans can come in over the public endpoint, ideally with a CIDR allow-list.

A bank picks private only: a production control plane has no listener on the internet, access rides the same private connectivity as everything else (Direct Connect, VPN as backup, Transit Gateway inside AWS), and pipelines run from runners inside the network. You can change the mode on a running cluster with aws eks update-cluster-config --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true.

Kubernetes version lifecycle

Every Kubernetes minor version gets about 14 months of standard support on EKS, then extended support for a further 12 months at the higher hourly fee. At the end of extended support AWS force-upgrades the control plane whether you are ready or not, so "we never upgrade" is not an option. A cluster's upgrade policy can be set to STANDARD instead of EXTENDED, meaning it is auto-upgraded when standard support ends and never accrues the extended fee, at the cost of the safety margin. Post 28 covers the upgrade procedure; for now, know two commands:

$ aws eks describe-cluster-versions --region ca-central-1 \
    --query 'clusterVersions[].{version:clusterVersion,status:status,stdEnd:endOfStandardSupportDate}' \
    --output table
----------------------------------------------------------
|                 DescribeClusterVersions                 |
+----------+--------------------+-----------------------+
|  version |  status            |  stdEnd               |
+----------+--------------------+-----------------------+
|  1.34    |  STANDARD_SUPPORT  |  2026-11-30T00:00:00  |
|  1.33    |  STANDARD_SUPPORT  |  2026-07-29T00:00:00  |
|  1.32    |  STANDARD_SUPPORT  |  2026-03-23T00:00:00  |
|  1.31    |  EXTENDED_SUPPORT  |  2025-11-26T00:00:00  |
|  1.30    |  EXTENDED_SUPPORT  |  2025-07-23T00:00:00  |
+----------+--------------------+-----------------------+

$ aws eks describe-cluster --name prod-ca --query 'cluster.{v:version,pv:platformVersion,mode:accessConfig.authenticationMode,upgrade:upgradePolicy.supportType}'
{
    "v": "1.33",
    "pv": "eks.12",
    "mode": "API",
    "upgrade": "STANDARD"
}

The exact dates will differ when you run it; what matters is that you know the command exists and that platformVersion (eks.12 above) is AWS's own revision counter for control plane patches within a Kubernetes minor, bumped by AWS without your involvement.

Three ways to create a cluster

Everything is the eks API underneath, whether you call it directly, click the console, use eksctl (the AWS-maintained CLI that takes a ClusterConfig YAML and drives CloudFormation), or use Terraform (the aws_eks_cluster resource, usually via the terraform-aws-modules/eks/aws module from Post 29). At a bank the answer is Terraform in a pipeline; eksctl is for sandboxes; the console is for looking. The raw call shows every knob the wrappers hide:

$ aws eks create-cluster --name prod-ca --kubernetes-version 1.33 \
    --role-arn arn:aws:iam::111122223333:role/eks-cluster-role \
    --resources-vpc-config subnetIds=subnet-0a1,subnet-0b2,subnet-0c3,endpointPublicAccess=false,endpointPrivateAccess=true \
    --kubernetes-network-config serviceIpv4Cidr=172.20.0.0/16,ipFamily=ipv4 \
    --access-config authenticationMode=API,bootstrapClusterCreatorAdminPermissions=true \
    --encryption-config '[{"resources":["secrets"],"provider":{"keyArn":"arn:aws:kms:ca-central-1:111122223333:key/abcd-..."}}]' \
    --logging '{"clusterLogging":[{"types":["api","audit","authenticator"],"enabled":true}]}' \
    --upgrade-policy supportType=STANDARD
{
    "cluster": {
        "name": "prod-ca",
        "status": "CREATING",
        ...

The cluster IAM role (eks-cluster-role) needs the AWS-managed AmazonEKSClusterPolicy; that is the identity the control plane uses to create ENIs and load balancers in your account. Creation takes 10 to 15 minutes. Note the serviceIpv4Cidr: it is the range for ClusterIP Services (Post 5), it must not overlap the VPC or anything reachable from it, and it cannot be changed after creation.

VPC and subnet design

Because the data plane lives in your VPC, VPC design is EKS design. Get this wrong and nothing above it can be fixed cheaply; the cluster's networking is baked in at creation.

Subnets per Availability Zone

The standard shape is three AZs, and per AZ: a public subnet (has a route to an Internet Gateway; holds internet-facing load balancers and, in non-bank setups, NAT gateways), a private subnet for nodes (routes to a NAT gateway or, at a bank, to a Transit Gateway that leads to a centralized egress VPC), and optionally a separate pod subnet from a secondary CIDR, which we will get to under IP planning. Nodes go in private subnets without exception. Load balancers go where their audience is: an internal ALB in the private subnets, an internet-facing ALB in the public ones.

The AWS Load Balancer Controller (covered below) discovers which subnets to use through tags, so tagging is not optional:

NAT gateways and VPC endpoints

Nodes in private subnets still need to pull images, call STS, and talk to the EKS API. The simple answer is a NAT gateway per AZ (one per AZ, not one shared, or you have a cross-AZ dependency and cross-AZ data charges). The bank answer is that nodes have no route to the internet at all: egress goes through the Transit Gateway to an inspection or egress VPC with a firewall, and AWS services are reached through VPC endpoints, which are private ENIs in your subnets that front an AWS service. A private EKS cluster needs at least these:

• com.amazonaws.<region>.ecr.api and ecr.dkr (interface endpoints) plus an S3 gateway endpoint, because ECR stores image layers in S3. Without the S3 endpoint, pulls authenticate and then hang.
• ec2 (the VPC CNI calls EC2 to attach ENIs and IPs), sts (IRSA and every SDK credential exchange), eks (so aws eks describe-cluster works from a private runner), and eks-auth if you use EKS Pod Identity, because the Pod Identity agent exchanges tokens against that service.
• elasticloadbalancing for the Load Balancer Controller, autoscaling if you still run Cluster Autoscaler, logs and monitoring for CloudWatch, ssm, ssmmessages, ec2messages for Session Manager access to nodes, kms if you encrypt with customer-managed keys.

Each interface endpoint costs roughly 0.01 USD per hour per AZ plus data processing, so a fully private three-AZ cluster carries a few hundred dollars a month in endpoint fees before any workload runs. Interviewers at banks like hearing that you know the cost and still consider it obviously correct.

IP planning: the number one EKS design mistake

The fact that surprises people coming from OpenShift: with the default Amazon VPC CNI, every pod consumes a real IP address from your VPC subnet, not an overlay address. A cluster running 3,000 pods needs 3,000 subnet IPs plus the ones the CNI keeps warm on every node, plus nodes, load balancers and endpoints. Teams given a /24 per AZ hit IP exhaustion: new pods sit in ContainerCreating with the CNI logging failed to assign an IP address to container, and no amount of node capacity helps.

You rarely get large routable ranges at a bank, because the corporate plan already fills 10.0.0.0/8. The standard fix is a secondary VPC CIDR from 100.64.0.0/10 (the carrier-grade NAT range, not routed on the corporate network) with one large pod subnet per AZ, and custom networking in the VPC CNI so pods draw from those subnets while nodes keep the small routable ones. Pods still reach on-prem because traffic leaving the VPC is source-NATed to the node's routable IP (controlled by AWS_VPC_K8S_CNI_EXTERNALSNAT). Two more levers: prefix delegation hands each node /28 blocks instead of single IPs, raising pod density and cutting EC2 API calls, and IPv6, where each node gets a /80 and exhaustion becomes impossible. IPv6 must be chosen at creation (ipFamily=ipv6, a dual-stack VPC; pods reach IPv4-only destinations through NAT on the node), and a bank whose on-prem tooling is IPv4-only will usually not choose it yet.

The Amazon VPC CNI: how pods get IPs

The Amazon VPC CNI = the default Container Network Interface plugin on EKS, running as the aws-node DaemonSet in kube-system. Its job is to make every pod a first-class citizen of the VPC. It does that by attaching extra ENIs to each node and pre-allocating secondary IP addresses on them; when a pod is scheduled, the CNI picks one of those addresses, creates a veth pair into the pod, and adds the routes. Every EC2 instance type has a fixed maximum of ENIs and of IPs per ENI, which is where the famous max pods per instance number comes from: an m5.large allows 3 ENIs with 10 IPs each, and since one IP per ENI is the primary, the formula ENIs × (IPs per ENI − 1) + 2 gives 29 pods. A t3.medium gets 17, an m5.xlarge 58, an m5.4xlarge 234. The kubelet's --max-pods is set from this table at boot, so the scheduler will never place a 30th pod on an m5.large regardless of CPU left over.

$ kubectl get ds aws-node -n kube-system
NAME       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
aws-node   6         6         6       6            6           <none>          41d

$ kubectl get ds aws-node -n kube-system -o jsonpath='{.spec.template.spec.containers[0].env}' | jq -r '.[] | "\(.name)=\(.value)"' | sort
AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
AWS_VPC_K8S_CNI_EXTERNALSNAT=false
ENABLE_POD_ENI=true
ENABLE_PREFIX_DELEGATION=true
ENI_CONFIG_LABEL_DEF=topology.kubernetes.io/zone
MINIMUM_IP_TARGET=20
WARM_IP_TARGET=5
WARM_PREFIX_TARGET=1
...

Those environment variables are the CNI's whole personality. Learn the important ones:

• WARM_ENI_TARGET (default 1): keep one entire spare ENI, fully populated with IPs, attached and ready. Fast pod starts, but on an m5.4xlarge that is 30 idle IPs per node, and across 60 nodes that is 1,800 addresses doing nothing.
• WARM_IP_TARGET and MINIMUM_IP_TARGET: replace the ENI-level pool with an IP-level one. "Keep at least 20 IPs on this node, and always 5 more than are in use." This is what you tune when subnets are tight; the trade is more EC2 API calls as pods churn.
• ENABLE_PREFIX_DELEGATION=true with WARM_PREFIX_TARGET: instead of individual IPs, each ENI slot holds a /28 prefix (16 addresses). An m5.large jumps from 29 to 110 pods (EKS caps small instances at 110 and larger ones at 250). It needs Nitro instances and it needs the subnet to still contain contiguous /28 blocks, so a fragmented subnet can fail prefix allocation while showing plenty of free addresses. Enable it before creating node groups: managed node groups compute the new max-pods automatically only for nodes launched afterwards.
• AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true plus ENI_CONFIG_LABEL_DEF=topology.kubernetes.io/zone: the custom networking switch. Pod ENIs are created in the subnet named by an ENIConfig object whose name matches the node's zone label, so you create one ENIConfig per AZ. The node's primary ENI is then reserved for the node itself, which lowers max-pods by one ENI's worth unless prefix delegation is on too.
• ENABLE_POD_ENI=true: turns on security groups for pods. The CNI attaches a trunk ENI to the node and gives selected pods their own branch ENI with its own security group, selected by a SecurityGroupPolicy object. Use it for the handful of workloads that must talk to an RDS instance whose security group only allows a named source; do not use it for everything, because branch ENIs are limited per instance and t-family instances cannot do it.

apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
  name: ca-central-1a          # must equal the node's topology.kubernetes.io/zone value
spec:
  subnet: subnet-0pod1a        # 100.64.0.0/18 pod subnet in AZ a
  securityGroups:
    - sg-0nodegroup             # SG applied to pod ENIs in this AZ

Network policy: built in, or Calico and Cilium

For years the VPC CNI did no policy enforcement and everyone bolted on Calico. Since VPC CNI 1.14 the add-on ships a network policy agent (an eBPF program managed by the aws-eks-nodeagent container in the same DaemonSet) that enforces the standard Kubernetes NetworkPolicy API. You switch it on through add-on configuration, enableNetworkPolicy: "true", and optionally enablePolicyEventLogs to write allow/deny decisions to /var/log/aws-routed-eni/network-policy-agent.log. It supports only the upstream API, so no GlobalNetworkPolicy, no DNS-name rules, no L7. Teams that need those, or who want a single policy language across OpenShift and EKS, run Calico or Cilium in policy-only mode alongside the VPC CNI. Cilium can also fully replace the VPC CNI (in "ENI mode" it still uses VPC IPs, or in overlay mode it does not), which is the option to mention when the interviewer asks how you would get past VPC IP limits without custom networking.

How this differs from OVN-Kubernetes on OpenShift

Diagnosing "no IPs available"

The symptom is a pod stuck in ContainerCreating. The method, in order:

$ kubectl describe pod api-7c9f5b-x2kd7 -n payments | tail -5
  Warning  FailedCreatePodSandBox  12s (x8 over 2m)  kubelet  Failed to create pod sandbox: rpc error:
    code = Unknown desc = failed to setup network for sandbox "3f1a...": plugin type="aws-cni"
    name="aws-cni" failed (add): add cmd: failed to assign an IP address to container

$ kubectl logs -n kube-system ds/aws-node -c aws-node --tail=3
{"level":"error","msg":"DataStore has no available IP/Prefix addresses"}
{"level":"warn","msg":"Failed to allocate IPs on ENI eni-0abc: InsufficientFreeAddressesInSubnet"}

$ aws ec2 describe-subnets --filters Name=tag:kubernetes.io/role/cni,Values=1 \
    --query 'Subnets[].{az:AvailabilityZone,id:SubnetId,free:AvailableIpAddressCount}' --output table
|  ca-central-1a | subnet-0pod1a |  0   |
|  ca-central-1b | subnet-0pod1b |  412 |
|  ca-central-1d | subnet-0pod1d |  388 |

Read it as a chain: the kubelet reports the CNI failed to assign an IP; the CNI's IP address manager (ipamd) says its local store is empty and EC2 refused to hand out more with InsufficientFreeAddressesInSubnet; the subnet in AZ a is at zero. Fixes, cheapest first: lower the warm targets so other nodes release hoarded IPs, cordon and drain nodes in the exhausted AZ so the scheduler uses the others, add a secondary CIDR and switch that AZ to custom networking, or enable prefix delegation. A different variant is ipamd reporting it cannot attach more ENIs (AttachmentLimitExceeded): the node is at its ENI ceiling, and the fix is bigger instances or prefix delegation. The full ipamd log lives on the node at /var/log/aws-routed-eni/ipamd.log, and the cni-metrics-helper deployment publishes awscni_total_ip_addresses and awscni_assigned_ip_addresses to CloudWatch so you can alert before hitting zero.

Data plane options: managed nodes, Fargate, Karpenter, Auto Mode

You have four ways to give the control plane something to schedule onto, and a real platform usually mixes two of them.

Managed node groups

A managed node group = an EC2 Auto Scaling group that EKS creates and operates: it picks the EKS-optimized AMI for your version, joins nodes, and rolls them (draining each, respecting PodDisruptionBudgets) when you change AMI or version. You choose the AMI family (AL2023_x86_64_STANDARD, AL2023_ARM_64_STANDARD, BOTTLEROCKET_x86_64 and GPU variants; Amazon Linux 2 AMIs stopped being published for new versions in 2025, so treat AL2_x86_64 as legacy), the capacity type (ON_DEMAND or SPOT with several instance types listed so Spot has choices), scaling bounds, labels and taints, and an update config (maxUnavailable or a percentage; since 2025 also a "minimal" update strategy that launches replacements more conservatively). A launch template adds what EKS does not expose: instance metadata options (IMDSv2, hop limit 1), root volume size and encryption, custom user data (on AL2023 a NodeConfig document for nodeadm, replacing bootstrap.sh), and a custom AMI if your bank bakes hardened images. Bottlerocket is the minimal, image-based, API-configured OS with no shell or package manager, the closest thing in AWS to RHCOS and the right default for a regulated environment. With the eks-node-monitoring-agent add-on and node auto repair enabled, EKS replaces nodes reporting kernel, network or storage faults without a human paging in.

$ aws eks describe-nodegroup --cluster-name prod-ca --nodegroup-name system-a \
    --query 'nodegroup.{ami:amiType,cap:capacityType,types:instanceTypes,ver:version,rel:releaseVersion,scale:scalingConfig,upd:updateConfig,taints:taints}'
{
    "ami": "BOTTLEROCKET_x86_64",
    "cap": "ON_DEMAND",
    "types": ["m6i.xlarge"],
    "ver": "1.33",
    "rel": "1.33.3-2e3c4f5d",
    "scale": {"minSize": 3, "maxSize": 6, "desiredSize": 3},
    "upd": {"maxUnavailable": 1},
    "taints": [{"key": "CriticalAddonsOnly", "value": "true", "effect": "NO_SCHEDULE"}]
}

That node group is the canonical "system" pool: small, on-demand, one per AZ or spread across three, tainted so only cluster components (CoreDNS, Karpenter, the LB controller, monitoring agents) land there. Application capacity comes from Karpenter.

Self-managed nodes

You can also run your own Auto Scaling groups, bootstrap nodes yourself, and register them with an access entry of type EC2_LINUX. Once the only option, today it is for the rare case managed node groups forbid, such as an unsupported OS. Mention it exists, then move on.

AWS Fargate

Fargate = serverless pods. A Fargate profile maps namespaces and label selectors to Fargate; each matching pod runs alone in its own micro-VM sized to its requests (rounded up, with about 256 MB reserved for the Fargate kubelet). No nodes to patch, strong isolation. The limits are the interview material: no DaemonSets (log shipping is configured through a ConfigMap in the aws-observability namespace instead), no privileged containers, no hostNetwork or hostPort, no EBS (EFS only), no GPUs, a 16 vCPU and 120 GB ceiling, private subnets only, slower starts, higher per-vCPU price. Good for bursty batch jobs, small clusters where nobody wants to own nodes, and historically for running Karpenter itself; poor for high-density microservices.

Karpenter

Karpenter = an open-source node autoscaler, created by AWS and now a CNCF project, that replaced Cluster Autoscaler for most EKS teams. Cluster Autoscaler works through Auto Scaling groups: it can only add "one more of what that group already is," it needs a group per instance shape, and it decides slowly. Karpenter watches for unschedulable pods, reads their requirements (CPU, memory, architecture, zone, taints, GPU), and calls the EC2 Fleet API directly to launch the cheapest instance that fits the batch, in seconds, without any Auto Scaling group. It also does consolidation: continuously looking for nodes that can be emptied or replaced by a cheaper one, and doing so within limits you define. The two objects you write are a NodePool (what kind of nodes, how much, how disruption is allowed) and an EC2NodeClass (the AWS specifics: AMI family, subnets, security groups, node IAM role, block devices, metadata options).

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: apps-general
spec:
  template:
    metadata:
      labels:
        pool: apps-general
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: bottlerocket-private
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]              # add "spot" for non-prod pools
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["m", "c", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["5"]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["ca-central-1a", "ca-central-1b", "ca-central-1d"]
      expireAfter: 720h                       # rotate nodes every 30 days for patching
      terminationGracePeriod: 48h
  limits:
    cpu: "800"
    memory: 3200Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 5m
    budgets:
      - nodes: "10%"                          # never disrupt more than 10% at once
      - nodes: "0"                            # and nothing during business hours
        schedule: "0 9 * * mon-fri"
        duration: 9h

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: bottlerocket-private
spec:
  amiSelectorTerms:
    - alias: bottlerocket@latest             # pin to a version id in prod, e.g. bottlerocket@v1.40.0
  role: eks-prod-ca-node-role
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: prod-ca
        tier: private
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: prod-ca
  metadataOptions:
    httpTokens: required
    httpPutResponseHopLimit: 1
  blockDeviceMappings:
    - deviceName: /dev/xvdb
      ebs:
        volumeSize: 100Gi
        volumeType: gp3
        encrypted: true

The disruption budget block is the part a bank cares about: it turns Karpenter's aggressive optimizer into something a change advisory board can sign off on. Karpenter needs its own IAM (a controller role via Pod Identity, a node role, an SQS queue subscribed to EC2 Spot interruption and health events), and it must run somewhere it does not manage: the tainted system node group above, or Fargate.

EKS Auto Mode

EKS Auto Mode (generally available December 2024) = AWS operates the data plane too: it runs Karpenter, the VPC CNI, CoreDNS, kube-proxy, the EBS CSI driver and the Load Balancer Controller as part of the control plane, launches Bottlerocket-based nodes it owns and rotates within 21 days, and exposes just a NodePool and NodeClass (two pools, general-purpose and system, exist by default). You cannot SSH or SSM into the nodes, bring a custom AMI or a different CNI, and you pay a management fee on top of EC2. For a team that wants EKS to feel fully managed it is compelling; for a bank with a hardened-image mandate, a mature Karpenter setup, or mandated node-level agents, standard mode with managed node groups plus Karpenter remains the answer. Explaining that trade-off is worth more than picking a side.

Authentication and authorization: IAM identity to cluster identity

EKS has no user database. Every kubectl call is authenticated by AWS IAM, and the question the cluster must answer is: "this IAM principal (a user or, almost always, an assumed role) maps to which Kubernetes username and groups?" Once that mapping exists, ordinary Kubernetes RBAC from Post 11 takes over. Two mechanisms have existed for that mapping, and interviewers ask about both because most real clusters are mid-migration.

The old way: the aws-auth ConfigMap

For the first six years of EKS, the mapping lived in a ConfigMap named aws-auth in kube-system, read by the aws-iam-authenticator inside the control plane:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::111122223333:role/eks-prod-ca-node-role
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
    - rolearn: arn:aws:iam::111122223333:role/AWSReservedSSO_PlatformAdmin_a1b2c3
      username: platform-admin:{{SessionName}}
      groups:
        - platform-admins
  mapUsers: |
    - userarn: arn:aws:iam::111122223333:user/break-glass
      username: break-glass
      groups:
        - system:masters

Its foot-guns are legendary, and naming them shows experience. A YAML indentation mistake in mapRoles breaks authentication for everyone, nodes included, so a bad kubectl edit could take the data plane offline. The cluster creator held system:masters invisibly, outside the ConfigMap, so nobody could audit who had root. IAM Identity Center roles have a path (/aws-reserved/sso.amazonaws.com/) you had to strip by hand or the match silently failed. And if you removed your own role, you were locked out until the creator came back.

The current way: access entries and access policies

Since late 2023 the mapping is an EKS API resource. An access entry = a record on the cluster saying "this IAM principal ARN is a cluster identity, with this Kubernetes username and these Kubernetes groups." An access policy = an AWS-managed bundle of Kubernetes permissions you associate with an access entry, scoped to the whole cluster or to a list of namespaces, so that common cases need no RBAC objects at all. The four you will name are AmazonEKSClusterAdminPolicy (equivalent to cluster-admin), AmazonEKSAdminPolicy (roughly the admin ClusterRole, meant for namespace scope), AmazonEKSEditPolicy (edit) and AmazonEKSViewPolicy (view). The cluster's authentication mode chooses the mechanism: CONFIG_MAP (only aws-auth), API_AND_CONFIG_MAP (both consulted, access entries win on conflict; the migration mode and the default for new clusters in most tooling) and API (access entries only). You can move a cluster forward through those modes but never back, so migrate deliberately: create access entries for every principal in aws-auth, test, then flip to API.

$ aws eks create-access-entry --cluster-name prod-ca \
    --principal-arn arn:aws:iam::111122223333:role/aws-reserved/sso.amazonaws.com/ca-central-1/AWSReservedSSO_PaymentsDev_9f8e7d \
    --type STANDARD --kubernetes-groups payments-developers
{
    "accessEntry": {
        "principalArn": "arn:aws:iam::111122223333:role/AWSReservedSSO_PaymentsDev_9f8e7d",
        "kubernetesGroups": ["payments-developers"],
        "username": "arn:aws:sts::111122223333:assumed-role/AWSReservedSSO_PaymentsDev_9f8e7d/{{SessionName}}",
        "type": "STANDARD"
    }
}

$ aws eks associate-access-policy --cluster-name prod-ca \
    --principal-arn arn:aws:iam::111122223333:role/AWSReservedSSO_PaymentsDev_9f8e7d \
    --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSEditPolicy \
    --access-scope type=namespace,namespaces=payments-dev,payments-test

$ aws eks list-access-entries --cluster-name prod-ca
{
    "accessEntries": [
        "arn:aws:iam::111122223333:role/AWSReservedSSO_PlatformAdmin_a1b2c3",
        "arn:aws:iam::111122223333:role/AWSReservedSSO_PaymentsDev_9f8e7d",
        "arn:aws:iam::111122223333:role/eks-prod-ca-node-role",
        "arn:aws:iam::111122223333:role/gha-deploy-runner"
    ]
}

Notice that EKS stripped the SSO path from the ARN for you, and that the username defaults to the assumed-role ARN with the session name, so the audit log shows the human's SSO username. Nodes are access entries too, of type EC2_LINUX, mapped to system:nodes without any ConfigMap. The pattern for a team: one Identity Center permission set per role, one access entry per resulting role, and either an access policy scoped to the team's namespaces or a Kubernetes group that your GitOps repo binds with a RoleBinding. Access entries are Terraform resources (aws_eks_access_entry, aws_eks_access_policy_association), so onboarding a team (Post 26 for the OpenShift version) becomes a pull request.

How the token exchange works

$ aws eks update-kubeconfig --name prod-ca --region ca-central-1 --profile platform-admin
Updated context arn:aws:eks:ca-central-1:111122223333:cluster/prod-ca in /Users/you/.kube/config

$ kubectl config view --minify -o jsonpath='{.users[0].user.exec}' | jq .
{
  "apiVersion": "client.authentication.k8s.io/v1beta1",
  "command": "aws",
  "args": ["--region", "ca-central-1", "eks", "get-token", "--cluster-name", "prod-ca", "--output", "json"],
  "env": [{"name": "AWS_PROFILE", "value": "platform-admin"}]
}

$ aws eks get-token --cluster-name prod-ca | jq -r .status.token | cut -c1-40
k8s-aws-v1.aHR0cHM6Ly9zdHMuY2EtY2VudHJhbC0

update-kubeconfig writes no credentials, only an exec credential plugin stanza: each time kubectl needs a token it runs aws eks get-token, which uses your AWS credential chain to build a presigned sts:GetCallerIdentity URL with the cluster name in a header, base64-encoded and prefixed k8s-aws-v1.. The authenticator calls that URL against STS, learns the caller's ARN, and looks it up in the access entries. The token lasts 15 minutes and is never stored. So "Unauthorized" on EKS is almost always an AWS-side problem (expired SSO session, wrong AWS_PROFILE, no access entry): run aws sts get-caller-identity first; if that works, the access entry is missing or mapped to groups no RBAC binding references.

Pod-level IAM: IRSA and EKS Pod Identity

Pods need AWS permissions: read a bucket, put a CloudWatch metric, decrypt with a KMS key. The lazy way is the node instance role: every process on the node, including every pod, can call the EC2 Instance Metadata Service (IMDS) at 169.254.169.254 and receive the node's credentials. That means a compromised pod in the marketing namespace holds the same permissions as the payments service and as the CNI itself (which can attach ENIs). Banks forbid it, and the JD's "IAM integration" bullet is really asking whether you know the two mechanisms that fix it.

IRSA: IAM Roles for Service Accounts

IRSA (2019) = each cluster publishes an OpenID Connect issuer; you register that issuer as an IAM OIDC identity provider; then an IAM role can trust one specific Kubernetes ServiceAccount through the token's sub claim, and pods using that ServiceAccount exchange their projected token for AWS credentials with sts:AssumeRoleWithWebIdentity. Three pieces make it work:

$ aws eks describe-cluster --name prod-ca --query cluster.identity.oidc.issuer --output text
https://oidc.eks.ca-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

$ eksctl utils associate-iam-oidc-provider --cluster prod-ca --region ca-central-1 --approve
[✔]  created IAM Open ID Connect provider for cluster "prod-ca" in "ca-central-1"

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.ca-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "oidc.eks.ca-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com",
        "oidc.eks.ca-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:payments:statement-exporter"
      }
    }
  }]
}

apiVersion: v1
kind: ServiceAccount
metadata:
  name: statement-exporter
  namespace: payments
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/payments-statement-exporter
    eks.amazonaws.com/sts-regional-endpoints: "true"

$ kubectl exec -n payments deploy/statement-exporter -- env | grep AWS_
AWS_ROLE_ARN=arn:aws:iam::111122223333:role/payments-statement-exporter
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
AWS_STS_REGIONAL_ENDPOINTS=regional
AWS_DEFAULT_REGION=ca-central-1

$ kubectl exec -n payments deploy/statement-exporter -- aws sts get-caller-identity
{
    "Arn": "arn:aws:sts::111122223333:assumed-role/payments-statement-exporter/botocore-session-1757..."
}

The pod never wrote those variables. A mutating admission webhook in the control plane (pod-identity-webhook) sees the annotation and injects them plus a projected volume holding a short-lived token with audience sts.amazonaws.com; the AWS SDKs look for exactly those variables, so no code changes. The sub condition is the whole security model: loosen it to system:serviceaccount:payments:* with StringLike and every ServiceAccount in the namespace can assume the role; omit it and every pod in the cluster can. Review trust policies as carefully as permission policies.

EKS Pod Identity

EKS Pod Identity (November 2023) = the same outcome with less ceremony. You install the eks-pod-identity-agent add-on (a DaemonSet that listens on the link-local address 169.254.170.23), give the IAM role a trust policy for the service principal pods.eks.amazonaws.com (one trust policy, identical for every cluster, no OIDC provider, no per-cluster issuer URL), and create a pod identity association that binds a namespace and ServiceAccount to the role in the EKS API rather than in an annotation:

$ aws eks create-addon --cluster-name prod-ca --addon-name eks-pod-identity-agent

$ aws iam create-role --role-name payments-ledger-reader --assume-role-policy-document '{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "pods.eks.amazonaws.com"},
    "Action": ["sts:AssumeRole", "sts:TagSession"]
  }]}'

$ aws eks create-pod-identity-association --cluster-name prod-ca \
    --namespace payments --service-account ledger-reader \
    --role-arn arn:aws:iam::111122223333:role/payments-ledger-reader
{
    "association": {
        "associationId": "a-0k3j2h1g",
        "namespace": "payments",
        "serviceAccount": "ledger-reader",
        "roleArn": "arn:aws:iam::111122223333:role/payments-ledger-reader"
    }
}

$ kubectl exec -n payments deploy/ledger-reader -- env | grep AWS_CONTAINER
AWS_CONTAINER_CREDENTIALS_FULL_URI=http://169.254.170.23/v1/credentials
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token

The same webhook injects these variables; the SDK calls the node agent with the projected token; the agent calls EKS Auth (eks-auth:AssumeRoleForPodIdentity, hence the eks-auth VPC endpoint in private clusters) and returns credentials. The session is tagged with cluster, namespace, ServiceAccount and pod name, so policies can use attribute-based conditions such as "only buckets tagged with this namespace". Because the trust policy is generic, one role serves the same ServiceAccount in twenty clusters through twenty associations, which makes Pod Identity the multi-cluster answer. It does not work on Fargate and needs SDKs from late 2023 or newer.

Least privilege and blocking IMDS

Pod roles should be tiny: one role per workload, one policy naming the specific bucket ARNs, queue ARNs and KMS keys, with conditions (aws:SourceVpce to require the VPC endpoint, s3:prefix for a directory, the Pod Identity session tags). Then close the back door. Set the node's instance metadata options to IMDSv2 required with a hop limit of 1 (in the launch template or the EC2NodeClass shown earlier): IMDSv2 responses carry an IP TTL of 1, and a pod's traffic crosses one extra hop through the veth, so the response dies before it reaches the pod. Only hostNetwork pods, which are already privileged by definition, can still reach it; a NetworkPolicy denying egress to 169.254.169.254/32 is the belt to that suspender. Verify from inside an ordinary pod: curl -s -m 2 http://169.254.169.254/latest/meta-data/ should time out.

On OpenShift the same problem is solved by the Cloud Credential Operator in manual/STS mode (Post 22): ccoctl publishes the cluster's OIDC documents to an S3 bucket, creates one IAM role per operator CredentialsRequest with the same sub-conditioned trust policy, and the cluster ships the identical pod-identity-webhook so application ServiceAccounts annotated with eks.amazonaws.com/role-arn work on OpenShift on AWS exactly as on EKS. If you can say "OpenShift STS mode and IRSA are the same OIDC federation pattern with different tooling around it," you have connected the two halves of the JD.

Managed add-ons

An EKS add-on = a cluster component that AWS packages, versions per Kubernetes release, installs through the EKS API, and upgrades for you on request. It is the closest EKS gets to OpenShift's cluster operators (Post 23), with a crucial difference: an add-on is upgraded when you ask, not continuously reconciled, and there is no oc get co that reports its health. The core four are vpc-cni, coredns, kube-proxy and eks-pod-identity-agent. The ones a real platform adds next: aws-ebs-csi-driver and aws-efs-csi-driver for storage, snapshot-controller for volume snapshots, amazon-cloudwatch-observability (Container Insights, the CloudWatch agent and Fluent Bit), aws-guardduty-agent (runtime threat detection, deployed automatically once GuardDuty EKS Runtime Monitoring is on), eks-node-monitoring-agent (feeds node auto repair), aws-mountpoint-s3-csi-driver, and since 2025 a set of "community add-ons" such as metrics-server, cert-manager, external-dns, kube-state-metrics and prometheus-node-exporter that AWS builds and hosts but does not otherwise support.

$ aws eks list-addons --cluster-name prod-ca
{
    "addons": [
        "amazon-cloudwatch-observability",
        "aws-ebs-csi-driver",
        "aws-efs-csi-driver",
        "aws-guardduty-agent",
        "coredns",
        "eks-node-monitoring-agent",
        "eks-pod-identity-agent",
        "kube-proxy",
        "metrics-server",
        "snapshot-controller",
        "vpc-cni"
    ]
}

$ aws eks describe-addon-versions --kubernetes-version 1.33 --addon-name vpc-cni \
    --query 'addons[].addonVersions[:3].{v:addonVersion,default:compatibilities[0].defaultVersion}' --output table
|  v1.20.1-eksbuild.1   |  False  |
|  v1.20.0-eksbuild.1   |  True   |
|  v1.19.6-eksbuild.7   |  False  |

$ aws eks describe-addon --cluster-name prod-ca --addon-name vpc-cni \
    --query 'addon.{v:addonVersion,status:status,pi:podIdentityAssociations,cfg:configurationValues}'
{
    "v": "v1.20.0-eksbuild.1",
    "status": "ACTIVE",
    "pi": ["arn:aws:eks:ca-central-1:111122223333:podidentityassociation/prod-ca/a-9x8y7z"],
    "cfg": "{\"enableNetworkPolicy\":\"true\",\"env\":{\"ENABLE_PREFIX_DELEGATION\":\"true\",\"WARM_PREFIX_TARGET\":\"1\"}}"
}

Three operating rules. Add-ons that call AWS (the CNI, CSI drivers, the CloudWatch agent) need IAM: --service-account-role-arn with an IRSA role, or preferably --pod-identity-associations so EKS creates the association itself. Configuration goes through --configuration-values, validated against a schema you can print with aws eks describe-addon-configuration; edit the DaemonSet by hand and the next update will fight you. That fight is governed by conflict resolution: --resolve-conflicts OVERWRITE on create resets hand edits to defaults, PRESERVE on update keeps values you set through configuration, and NONE fails if anything differs, the safe pipeline default because it makes drift visible. Pin versions in Terraform (aws_eks_addon with addon_version); add-on upgrades are part of every upgrade runbook in Post 28.

Load balancing and ingress

The AWS Load Balancer Controller = the open-source controller (installed with Helm, or built in with Auto Mode) that turns Ingress objects into Application Load Balancers and Service objects into Network Load Balancers, and since version 2.13 also implements the Gateway API; the legacy in-tree cloud provider could only make Classic Load Balancers. It runs in kube-system with an IRSA or Pod Identity role holding the policy from its GitHub release, and it needs the subnet tags from earlier:

$ helm repo add eks https://aws.github.io/eks-charts
$ helm upgrade --install aws-load-balancer-controller eks/aws-load-balancer-controller \
    -n kube-system --version 1.13.3 \
    --set clusterName=prod-ca \
    --set serviceAccount.create=false \
    --set serviceAccount.name=aws-load-balancer-controller \
    --set region=ca-central-1 --set vpcId=vpc-0abc123 \
    --set defaultTargetType=ip
Release "aws-load-balancer-controller" has been upgraded. Happy Helming!

$ kubectl get deploy -n kube-system aws-load-balancer-controller
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           41d

Ingress to ALB

An ALB is layer 7: TLS termination with an ACM certificate, host and path routing, WAF and Shield, and optional Cognito or OIDC authentication in front of your pod. Annotations configure it. target-type: ip registers pod IPs directly (possible because pods have VPC IPs), skipping the NodePort hop and letting readiness gates hold a rollout until the ALB sees the new pod healthy; target-type: instance is the older NodePort path, the only option on an overlay CNI. group.name is the cost control: every Ingress in the group shares one ALB, so a namespace-per-team platform does not pay for 80 of them.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: statements-api
  namespace: payments
  annotations:
    alb.ingress.kubernetes.io/scheme: internal                 # never internet-facing at a bank without review
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/group.name: payments-internal    # one shared ALB for the namespace
    alb.ingress.kubernetes.io/group.order: "10"
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:ca-central-1:111122223333:certificate/3f2a...
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS13-1-2-2021-06
    alb.ingress.kubernetes.io/wafv2-acl-arn: arn:aws:wafv2:ca-central-1:111122223333:regional/webacl/internal-baseline/8b1c...
    alb.ingress.kubernetes.io/healthcheck-path: /healthz
    alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=true,access_logs.s3.bucket=bmo-alb-logs-ca,idle_timeout.timeout_seconds=60
    alb.ingress.kubernetes.io/subnets: subnet-0priv1a,subnet-0priv1b,subnet-0priv1d
    external-dns.alpha.kubernetes.io/hostname: statements.payments.internal.example.ca
spec:
  ingressClassName: alb
  rules:
    - host: statements.payments.internal.example.ca
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: statements-api
                port:
                  number: 8080

$ kubectl get ingress -n payments statements-api
NAME             CLASS   HOSTS                                     ADDRESS                                                         PORTS   AGE
statements-api   alb     statements.payments.internal.example.ca   internal-k8s-paymentsinter-1a2b3c-456789.ca-central-1.elb.amazonaws.com   80      3m

The ADDRESS column filling in is your signal of success; if it stays empty, kubectl describe ingress shows the controller's event, most often "couldn't auto-discover subnets" (missing tags) or an IAM denial. An IngressClassParams object on the alb IngressClass can enforce scheme: internal cluster-wide, which is how a platform team stops a developer from accidentally publishing an internet-facing ALB.

Service to NLB

An NLB is layer 4: TCP and UDP passthrough, static IPs per AZ, millions of connections, no HTTP awareness, and the only option for non-HTTP protocols or when a firewall team needs fixed addresses to allow-list. Ask for one with loadBalancerClass: service.k8s.aws/nlb rather than the old aws-load-balancer-type: external annotation:

apiVersion: v1
kind: Service
metadata:
  name: mq-broker
  namespace: messaging
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internal
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true,deletion_protection.enabled=true
    service.beta.kubernetes.io/aws-load-balancer-private-ipv4-addresses: 10.40.1.20,10.40.2.20,10.40.3.20
spec:
  type: LoadBalancer
  loadBalancerClass: service.k8s.aws/nlb
  selector:
    app: mq-broker
  ports:
    - port: 5671
      targetPort: 5671
      protocol: TCP

Two more objects complete the picture. A TargetGroupBinding attaches an existing target group (created by Terraform on a load balancer the network team owns) to a Service, so the controller only registers pod IPs and never touches the load balancer; that division of ownership is common at banks. ExternalDNS watches Ingress and Service objects and writes Route 53 records for their hostnames, so the annotation above becomes an ALIAS record in a private hosted zone without a ticket.

Private-only ingress at a bank

The end-to-end path for an internal API: a user on the corporate network resolves statements.payments.internal.example.ca through corporate DNS, which forwards the zone to a Route 53 Resolver inbound endpoint; the answer is the internal ALB's private IPs; the packet rides Direct Connect into the Transit Gateway, into the cluster VPC's private subnets, through the WAF-attached ALB listener with a certificate from the bank's private CA, and to a pod IP target. No public hosted zone, no internet gateway, no public certificate, and every hop logs (VPC Flow Logs, ALB access logs, WAF logs). Say that sentence on a whiteboard and you have answered "how do you expose services on EKS in a regulated environment."

DNS and service discovery

Inside the cluster, CoreDNS resolves Service names as in Post 5; the EKS twist is scaling it. The coredns add-on ships two replicas regardless of cluster size, and a 500-node cluster will melt them. Since 2024 the add-on has built-in autoscaling (autoScaling.enabled: true with minReplicas and maxReplicas in its configuration values), replacing the hand-deployed cluster-proportional-autoscaler. For DNS-heavy workloads add NodeLocal DNSCache, a DaemonSet answering from a per-node cache at 169.254.20.10, so most lookups never leave the node and the conntrack race behind the infamous five-second DNS timeouts is avoided.

Outside the cluster, EKS delegates to Route 53: private hosted zones associated with the VPC hold internal names (ExternalDNS writes them), Resolver inbound endpoints let on-prem DNS query them, and outbound endpoints with forwarding rules (or a forward stanza in the CoreDNS Corefile) let pods resolve corporate names. Between VPCs and accounts you have three tools: Transit Gateway for routed connectivity, AWS PrivateLink for exposing exactly one service to another account with no routing (an NLB in front of the Service, a VPC endpoint service, an interface endpoint on the consumer side), and VPC Lattice with the AWS Gateway API Controller for application-level service networking across accounts. Transit Gateway for "these VPCs should route", PrivateLink for "this account may call exactly this API".

Storage on EKS, briefly

The EBS CSI driver add-on provisions block volumes: ReadWriteOnce, fast, and bound to one AZ, which is the detail that bites: a PVC provisioned before its pod is scheduled can land in AZ a while the scheduler wants b, leaving the pod Pending with a volume affinity conflict. volumeBindingMode: WaitForFirstConsumer delays provisioning until the pod has a node. The in-tree kubernetes.io/aws-ebs provisioner is gone, so the pre-created gp2 StorageClass does nothing until the CSI add-on is installed; create your own gp3 class and make it default:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3-encrypted
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  encrypted: "true"
  kmsKeyId: arn:aws:kms:ca-central-1:111122223333:key/9c2d-...   # bank CMK, not the AWS-managed key
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete

The EFS CSI driver gives ReadWriteMany NFS volumes that span AZs; its dynamic provisioning creates an access point per PVC (a root directory with fixed POSIX owner and permissions) on one shared file system, which is how you isolate tenants on EFS. FSx (Lustre for HPC and ML scratch, ONTAP for teams migrating NetApp workloads, OpenZFS) has its own CSI drivers. Mountpoint for Amazon S3 exposes a bucket as a volume for read-heavy, sequential workloads; it is not POSIX-complete and is no substitute for a file system. Snapshots, backup with Velero, and the operational side of all this live in Part 2.

A reference architecture for a bank's EKS platform

Now put it together the way you would on a whiteboard, in the order you would draw it. Start with accounts, then network, then clusters, then identity, then the platform services around them; each layer only makes sense on top of the previous one.

1. Accounts (the landing zone). An AWS Organization with management, log-archive, security-tooling and shared-services accounts, plus one workload account per environment per line of business (payments-dev, payments-prod). Service control policies deny regions outside ca-central-1 and ca-west-1 (data residency in Canada is a real requirement, so say it), deny public S3, and deny EKS clusters with a public endpoint.
2. Network. A Transit Gateway in a network account, attached to every workload VPC, to an inspection VPC running AWS Network Firewall or the bank's appliance, to a centralized egress VPC with NAT gateways, and to Direct Connect (two circuits, two locations) with VPN backup. Workload VPCs have no internet gateway; their default route is the Transit Gateway. Each cluster VPC has three-AZ private node subnets on routable space, 100.64.0.0/16 pod subnets with custom networking, internal ALB subnets and the full set of VPC endpoints. Route 53 Resolver endpoints bridge corporate DNS.
3. Clusters. One cluster per environment per account (blast radius and change windows argue against one giant shared cluster), private endpoint only, API authentication mode, secrets encrypted with a customer-managed KMS key, all five control plane log types to CloudWatch, a Bottlerocket managed node group for the system pool, Karpenter with business-hours disruption budgets for application capacity, the VPC CNI with prefix delegation and network policy on, IMDSv2 hop limit 1 everywhere. All built by Terraform from a versioned module in a pipeline (Post 29); nobody clicks.
4. Identity. Humans authenticate through IAM Identity Center federated to the bank's directory; permission sets become IAM roles, roles become access entries, access entries map to Kubernetes groups, and the groups are bound to namespaced roles by manifests in Git. Workloads use Pod Identity with one role per ServiceAccount. Pipelines assume a dedicated deploy role through OIDC federation (GitHub Actions or the bank's runner) and hold an access entry scoped to their namespaces.
5. Delivery. Argo CD (Post 30) in a shared management cluster reconciles every workload cluster from Git: namespaces, quotas, RBAC bindings, network policies, non-managed add-ons and the applications. Images live in ECR in the shared-services account with tag immutability, Inspector scanning, a pull-through cache for approved upstream registries and replication to ca-west-1 for DR; admission policy (Kyverno or Gatekeeper) blocks images from anywhere else.
6. Observability. The CloudWatch Observability add-on for Container Insights and logs, Amazon Managed Prometheus scraped by the AWS Distro for OpenTelemetry collector, Amazon Managed Grafana federated to the same Identity Center, and alerts into the bank's incident tooling. Part 2 goes deep on this.
7. Security and compliance. GuardDuty EKS Protection (audit log analysis plus the runtime agent), Security Hub aggregating findings into the security-tooling account, AWS Config rules and conformance packs for the CIS EKS benchmark, CloudTrail in the log-archive account, and the compliance-reporting automation from Post 31.

OpenShift to EKS: the translation table

This is the table to have in your head when the interviewer switches from one half of the JD to the other mid-sentence.

Likely interview questions

Next up: Amazon EKS Part 2, where the cluster you just designed has to be upgraded across three minor versions without an outage, kept patched and observable, hardened to a CIS benchmark, and explained to a finance team that wants to know why the bill doubled.