AWS EKS
Managed Kubernetes — control plane, node groups, VPC CNI, Pod Identity, Karpenter, and observability
AWS Account ┌────────────────────────────────────────────────────────────────┐ │ EKS Managed Control Plane (AWS-owned account) │ │ kube-apiserver · etcd · scheduler · controller-manager │ │ Endpoint: public | private | both │ └───────────────────────┬────────────────────────────────────────┘ │ kubectl / aws-iam-authenticator ┌───────────────────────▼────────────────────────────────────────┐ │ Your VPC │ │ ┌─────────────────────┐ ┌──────────────────────────────┐ │ │ │ Managed Node Group │ │ Fargate Profile │ │ │ │ EC2 (AL2 / Bottler) │ │ Serverless pods │ │ │ │ kubelet · VPC CNI │ │ (one pod per Fargate task) │ │ │ └─────────────────────┘ └──────────────────────────────┘ │ │ │ │ Karpenter (NodePool) · Add-ons · Load Balancer Controller │ └────────────────────────────────────────────────────────────────┘
EKS ArchitectureCore

EKS manages the Kubernetes control plane in an AWS-owned account. You only manage the data plane (nodes). The control plane runs across multiple AZs with automated backups and patching.

Control plane access modes

ModeWho can reach the API serverNotes
Public + PrivateInternet (public endpoint) and nodes (private)Default. kubectl from anywhere; node traffic stays in VPC.
Public onlyInternet onlyNode-to-API traffic traverses internet (not recommended).
Private onlyWithin VPC onlyMost secure. kubectl requires VPN/DX or bastion in VPC.
Shell
# Create a cluster
eksctl create cluster \
  --name my-cluster \
  --region us-east-1 \
  --version 1.30 \
  --vpc-private-subnets subnet-1a,subnet-1b \
  --vpc-public-subnets subnet-pub-1a,subnet-pub-1b \
  --without-nodegroup

# Update kubeconfig
aws eks update-kubeconfig --name my-cluster --region us-east-1

# Restrict API endpoint to private only
aws eks update-cluster-config \
  --name my-cluster \
  --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true

# Check cluster status
aws eks describe-cluster --name my-cluster \
  --query 'cluster.{Status:status,Version:version,Endpoint:endpoint}'
Node GroupsCompute

EKS supports three compute models: managed node groups (AWS manages ASG and node lifecycle), self-managed nodes (you control the ASG), and Fargate (serverless, per-pod billing).

Managed vs self-managed vs Fargate

Managed Node GroupSelf-ManagedFargate
Node provisioningAWS via ASGYou manage ASGAWS per pod
Drain on updateYes (automatic)ManualN/A
AMIEKS-optimised AL2/BottlerocketAnyAWS-managed
GPU / customYesYesNo
DaemonSetsYesYesNo
Cost modelEC2 on-demand/spotEC2 on-demand/spotvCPU+GB-hour
Shell
# Create managed node group
eksctl create nodegroup \
  --cluster my-cluster \
  --name general \
  --node-type m5.xlarge \
  --nodes 2 --nodes-min 1 --nodes-max 10 \
  --subnet-ids subnet-private-1a,subnet-private-1b \
  --asg-access \
  --managed

# Create spot node group (Karpenter preferred, but available via MNG)
eksctl create nodegroup \
  --cluster my-cluster \
  --name spot-workers \
  --node-type m5.xlarge,m5a.xlarge,m4.xlarge \
  --spot \
  --nodes-min 0 --nodes-max 20

# Create Fargate profile
eksctl create fargateprofile \
  --cluster my-cluster \
  --name fp-default \
  --namespace default \
  --labels env=prod
VPC CNINetworking

The AWS VPC CNI plugin assigns real VPC IP addresses to pods. Each pod gets a secondary IP from the node's ENI — pods are first-class VPC citizens. This limits pod density to ENI × IP-per-ENI per instance type.

How it works

Flow
Node boots → ipamd daemon starts
  │
  ├─ Attaches additional ENIs to the instance
  │    (up to max ENIs for instance type)
  ├─ Pre-warms secondary IPs on each ENI
  │
Pod scheduled → kubelet calls CNI
  │
  ├─ ipamd assigns a pre-warmed secondary IP to the pod
  ├─ Creates a veth pair: pod netns ↔ host netns
  └─ Adds iptables/ip rules to route pod traffic via ENI

Pod IPs are real VPC IPs — visible in VPC routing, SGs apply directly

IP exhaustion and prefix delegation

Shell
# Enable prefix delegation (assigns /28 = 16 IPs per slot instead of 1)
# Dramatically increases pod density
kubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true
kubectl set env daemonset aws-node -n kube-system WARM_PREFIX_TARGET=1

# Check available IPs on a node
kubectl describe node <node> | grep -A5 "Allocatable"

# Check ipamd status
kubectl exec -n kube-system ds/aws-node -- /app/grpc-health-probe -addr=:50051

# Custom networking — assign pod IPs from a different CIDR
# (useful when VPC CIDR is exhausted)
kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
# Then create ENIConfig CRDs per AZ
Pod IdentityIAM

EKS Pod Identity (and its predecessor IRSA) lets pods assume IAM roles without static credentials. The kubelet projects a short-lived token into the pod; the AWS SDK exchanges it for IAM credentials.

EKS Pod Identity vs IRSA

EKS Pod Identity (newer)IRSA
How it worksEKS agent intercepts AWS SDK calls, provides credentials via credential processOIDC federation: pod token exchanged via STS AssumeRoleWithWebIdentity
SetupAssociate IAM role with service account via EKS APIOIDC provider + trust policy on IAM role
Credential refreshAutomatic via agentSDK refreshes before expiry
Cross-accountYesYes (via STS)
RecommendationPreferred for new clustersEstablished, widely supported
Shell
# EKS Pod Identity setup
# 1. Enable the Pod Identity agent add-on
aws eks create-addon --cluster-name my-cluster \
  --addon-name eks-pod-identity-agent

# 2. Create IAM role with trust policy for EKS Pod Identity
# Trust principal: pods.eks.amazonaws.com

# 3. Associate role with service account
aws eks create-pod-identity-association \
  --cluster-name my-cluster \
  --namespace my-namespace \
  --service-account my-sa \
  --role-arn arn:aws:iam::123456789:role/my-pod-role

# IRSA setup (legacy)
eksctl utils associate-iam-oidc-provider --cluster my-cluster --approve

eksctl create iamserviceaccount \
  --cluster my-cluster \
  --namespace my-namespace \
  --name my-sa \
  --role-name my-pod-role \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \
  --approve
KarpenterAutoscaling

Karpenter is an open-source node autoscaler for Kubernetes. It watches for unschedulable pods and directly provisions EC2 instances that exactly fit the workload — faster and more flexible than Cluster Autoscaler.

Key resources

ResourceRole
NodePoolDefines constraints for nodes Karpenter can provision: instance families, AZs, capacity types (spot/on-demand), taints, limits.
EC2NodeClassAWS-specific config: AMI family, subnets, security groups, instance profile, user data.
NodeClaimCreated by Karpenter when a node is provisioned. Tracks lifecycle.
YAML
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: [spot, on-demand]
      - key: karpenter.k8s.aws/instance-family
        operator: In
        values: [m5, m5a, m6i, m6a]
      - key: karpenter.k8s.aws/instance-size
        operator: NotIn
        values: [nano, micro, small]
  limits:
    cpu: 1000           # total CPU across all Karpenter nodes
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m
Shell
# Install Karpenter via Helm
helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter \
  --namespace kube-system \
  --set settings.clusterName=my-cluster \
  --set settings.interruptionQueue=my-cluster \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi

# Watch node provisioning
kubectl get nodeclaims -w
kubectl get nodes -w

# Trigger consolidation manually
kubectl annotate nodepool general karpenter.sh/do-not-disrupt-
Cluster Add-onsAdd-ons

EKS managed add-ons are installed and updated by AWS, with version management via the EKS API. Critical add-ons should be managed this way rather than manually.

Essential add-ons

Add-onPurpose
vpc-cniPod networking — assigns VPC IPs to pods
corednsCluster DNS resolution
kube-proxyService networking (iptables/ipvs). Not needed with Cilium.
aws-ebs-csi-driverProvision EBS volumes as PersistentVolumes
aws-efs-csi-driverMount EFS as ReadWriteMany PersistentVolumes
eks-pod-identity-agentEnable EKS Pod Identity credential injection
amazon-cloudwatch-observabilityContainer Insights metrics and logs
Shell
# List installed add-ons
aws eks list-addons --cluster-name my-cluster

# Get latest available version for an add-on
aws eks describe-addon-versions --addon-name aws-ebs-csi-driver \
  --kubernetes-version 1.30 \
  --query 'addons[0].addonVersions[0].addonVersion'

# Install EBS CSI driver
aws eks create-addon \
  --cluster-name my-cluster \
  --addon-name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::123456789:role/ebs-csi-role

# Update an add-on
aws eks update-addon \
  --cluster-name my-cluster \
  --addon-name coredns \
  --addon-version v1.11.1-eksbuild.4 \
  --resolve-conflicts OVERWRITE
NetworkingNetworking

The AWS Load Balancer Controller (LBC) provisions ALBs for Ingress resources and NLBs for LoadBalancer Services. Security groups for pods allow attaching SGs directly to pods.

Load Balancer Controller

Shell
# Install AWS Load Balancer Controller
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=my-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller
YAML
# Ingress → ALB
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip        # pod IPs directly
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
spec:
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-app
            port:
              number: 8080
---
# Service → NLB
apiVersion: v1
kind: Service
metadata:
  name: my-tcp-service
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
  type: LoadBalancer
  ports:
  - port: 443
    targetPort: 8443

Security groups for pods

Security groups for pods require a supported instance type (nitro-based) and the VPC CNI ENABLE_POD_ENI=true setting. Each pod with an SG gets a dedicated branch ENI — counts against the instance's ENI limit.
ObservabilityObservability
Shell
# Enable Container Insights (metrics + logs)
aws eks create-addon \
  --cluster-name my-cluster \
  --addon-name amazon-cloudwatch-observability

# Or via CloudWatch agent DaemonSet + Fluent Bit
ClusterName=my-cluster
RegionName=us-east-1
FluentBitHttpPort='2020'
FluentBitReadFromHead='Off'
curl -s https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluent-bit-quickstart.yaml | \
  sed "s/{{cluster_name}}/${ClusterName}/g;s/{{region_name}}/${RegionName}/g" | kubectl apply -f -

# Control plane logging
aws eks update-cluster-config \
  --name my-cluster \
  --logging '{"clusterLogging":[{"types":["api","audit","authenticator","controllerManager","scheduler"],"enabled":true}]}'

# Query control plane logs in CloudWatch
# Log group: /aws/eks/my-cluster/cluster
SecuritySecurity

EKS access entries (new auth model)

Shell
# New model: EKS Access Entries (replaces aws-auth ConfigMap)
# Enable on cluster creation or migration
aws eks update-cluster-config \
  --name my-cluster \
  --access-config authenticationMode=API_AND_CONFIG_MAP

# Grant IAM role cluster-admin access
aws eks create-access-entry \
  --cluster-name my-cluster \
  --principal-arn arn:aws:iam::123456789:role/admin-role \
  --type STANDARD

aws eks associate-access-policy \
  --cluster-name my-cluster \
  --principal-arn arn:aws:iam::123456789:role/admin-role \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
  --access-scope type=cluster

# Legacy: aws-auth ConfigMap
kubectl edit configmap aws-auth -n kube-system

Image scanning

Shell
# Enable enhanced scanning on ECR (Inspector v2)
aws ecr put-registry-scanning-configuration \
  --scan-type ENHANCED \
  --rules '[{"repositoryFilters":[{"filter":"*","filterType":"WILDCARD"}],"scanFrequency":"CONTINUOUS_SCAN"}]'

# Get scan findings for an image
aws ecr describe-image-scan-findings \
  --repository-name my-app \
  --image-id imageTag=latest \
  --query 'imageScanFindings.findings[?severity==`CRITICAL`]'
Cheat SheetReference

Cluster management

aws eks update-kubeconfig --name cluster
eksctl create nodegroup ...
aws eks list-addons --cluster-name x
aws eks update-addon ...

VPC CNI

Pod IPs = real VPC IPs
Prefix delegation: ENABLE_PREFIX_DELEGATION=true
SGs for pods: ENABLE_POD_ENI=true
Check: kubectl describe node | grep Allocatable

Pod Identity

Prefer EKS Pod Identity over IRSA
aws eks create-pod-identity-association ...
Verify: aws sts get-caller-identity in pod
Token at: /var/run/secrets/...

Karpenter

NodePool = constraints
EC2NodeClass = AWS config
kubectl get nodeclaims
Disruption: consolidationPolicy: WhenEmptyOrUnderutilized

Debugging

kubectl describe pod x — events
kubectl logs -p x — previous container
kubectl get events --sort-by=.lastTimestamp
aws eks describe-nodegroup ...

Security

Use Access Entries (not aws-auth)
Pod Identity for AWS API calls
Enable ECR enhanced scanning
Pod Security Admission on namespaces