EKS manages the Kubernetes control plane in an AWS-owned account. You only manage the data plane (nodes). The control plane runs across multiple AZs with automated backups and patching.
Control plane access modes
| Mode | Who can reach the API server | Notes |
|---|---|---|
| Public + Private | Internet (public endpoint) and nodes (private) | Default. kubectl from anywhere; node traffic stays in VPC. |
| Public only | Internet only | Node-to-API traffic traverses internet (not recommended). |
| Private only | Within VPC only | Most secure. kubectl requires VPN/DX or bastion in VPC. |
# Create a cluster eksctl create cluster \ --name my-cluster \ --region us-east-1 \ --version 1.30 \ --vpc-private-subnets subnet-1a,subnet-1b \ --vpc-public-subnets subnet-pub-1a,subnet-pub-1b \ --without-nodegroup # Update kubeconfig aws eks update-kubeconfig --name my-cluster --region us-east-1 # Restrict API endpoint to private only aws eks update-cluster-config \ --name my-cluster \ --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true # Check cluster status aws eks describe-cluster --name my-cluster \ --query 'cluster.{Status:status,Version:version,Endpoint:endpoint}'
EKS supports three compute models: managed node groups (AWS manages ASG and node lifecycle), self-managed nodes (you control the ASG), and Fargate (serverless, per-pod billing).
Managed vs self-managed vs Fargate
| Managed Node Group | Self-Managed | Fargate | |
|---|---|---|---|
| Node provisioning | AWS via ASG | You manage ASG | AWS per pod |
| Drain on update | Yes (automatic) | Manual | N/A |
| AMI | EKS-optimised AL2/Bottlerocket | Any | AWS-managed |
| GPU / custom | Yes | Yes | No |
| DaemonSets | Yes | Yes | No |
| Cost model | EC2 on-demand/spot | EC2 on-demand/spot | vCPU+GB-hour |
# Create managed node group eksctl create nodegroup \ --cluster my-cluster \ --name general \ --node-type m5.xlarge \ --nodes 2 --nodes-min 1 --nodes-max 10 \ --subnet-ids subnet-private-1a,subnet-private-1b \ --asg-access \ --managed # Create spot node group (Karpenter preferred, but available via MNG) eksctl create nodegroup \ --cluster my-cluster \ --name spot-workers \ --node-type m5.xlarge,m5a.xlarge,m4.xlarge \ --spot \ --nodes-min 0 --nodes-max 20 # Create Fargate profile eksctl create fargateprofile \ --cluster my-cluster \ --name fp-default \ --namespace default \ --labels env=prod
The AWS VPC CNI plugin assigns real VPC IP addresses to pods. Each pod gets a secondary IP from the node's ENI — pods are first-class VPC citizens. This limits pod density to ENI × IP-per-ENI per instance type.
How it works
Node boots → ipamd daemon starts │ ├─ Attaches additional ENIs to the instance │ (up to max ENIs for instance type) ├─ Pre-warms secondary IPs on each ENI │ Pod scheduled → kubelet calls CNI │ ├─ ipamd assigns a pre-warmed secondary IP to the pod ├─ Creates a veth pair: pod netns ↔ host netns └─ Adds iptables/ip rules to route pod traffic via ENI Pod IPs are real VPC IPs — visible in VPC routing, SGs apply directly
IP exhaustion and prefix delegation
# Enable prefix delegation (assigns /28 = 16 IPs per slot instead of 1) # Dramatically increases pod density kubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true kubectl set env daemonset aws-node -n kube-system WARM_PREFIX_TARGET=1 # Check available IPs on a node kubectl describe node <node> | grep -A5 "Allocatable" # Check ipamd status kubectl exec -n kube-system ds/aws-node -- /app/grpc-health-probe -addr=:50051 # Custom networking — assign pod IPs from a different CIDR # (useful when VPC CIDR is exhausted) kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true # Then create ENIConfig CRDs per AZ
EKS Pod Identity (and its predecessor IRSA) lets pods assume IAM roles without static credentials. The kubelet projects a short-lived token into the pod; the AWS SDK exchanges it for IAM credentials.
EKS Pod Identity vs IRSA
| EKS Pod Identity (newer) | IRSA | |
|---|---|---|
| How it works | EKS agent intercepts AWS SDK calls, provides credentials via credential process | OIDC federation: pod token exchanged via STS AssumeRoleWithWebIdentity |
| Setup | Associate IAM role with service account via EKS API | OIDC provider + trust policy on IAM role |
| Credential refresh | Automatic via agent | SDK refreshes before expiry |
| Cross-account | Yes | Yes (via STS) |
| Recommendation | Preferred for new clusters | Established, widely supported |
# EKS Pod Identity setup # 1. Enable the Pod Identity agent add-on aws eks create-addon --cluster-name my-cluster \ --addon-name eks-pod-identity-agent # 2. Create IAM role with trust policy for EKS Pod Identity # Trust principal: pods.eks.amazonaws.com # 3. Associate role with service account aws eks create-pod-identity-association \ --cluster-name my-cluster \ --namespace my-namespace \ --service-account my-sa \ --role-arn arn:aws:iam::123456789:role/my-pod-role # IRSA setup (legacy) eksctl utils associate-iam-oidc-provider --cluster my-cluster --approve eksctl create iamserviceaccount \ --cluster my-cluster \ --namespace my-namespace \ --name my-sa \ --role-name my-pod-role \ --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \ --approve
Karpenter is an open-source node autoscaler for Kubernetes. It watches for unschedulable pods and directly provisions EC2 instances that exactly fit the workload — faster and more flexible than Cluster Autoscaler.
Key resources
| Resource | Role |
|---|---|
| NodePool | Defines constraints for nodes Karpenter can provision: instance families, AZs, capacity types (spot/on-demand), taints, limits. |
| EC2NodeClass | AWS-specific config: AMI family, subnets, security groups, instance profile, user data. |
| NodeClaim | Created by Karpenter when a node is provisioned. Tracks lifecycle. |
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: general
spec:
template:
spec:
nodeClassRef:
group: karpenter.k8s.aws
kind: EC2NodeClass
name: default
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: [spot, on-demand]
- key: karpenter.k8s.aws/instance-family
operator: In
values: [m5, m5a, m6i, m6a]
- key: karpenter.k8s.aws/instance-size
operator: NotIn
values: [nano, micro, small]
limits:
cpu: 1000 # total CPU across all Karpenter nodes
disruption:
consolidationPolicy: WhenEmptyOrUnderutilized
consolidateAfter: 1m
# Install Karpenter via Helm helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter \ --namespace kube-system \ --set settings.clusterName=my-cluster \ --set settings.interruptionQueue=my-cluster \ --set controller.resources.requests.cpu=1 \ --set controller.resources.requests.memory=1Gi # Watch node provisioning kubectl get nodeclaims -w kubectl get nodes -w # Trigger consolidation manually kubectl annotate nodepool general karpenter.sh/do-not-disrupt-
EKS managed add-ons are installed and updated by AWS, with version management via the EKS API. Critical add-ons should be managed this way rather than manually.
Essential add-ons
| Add-on | Purpose |
|---|---|
| vpc-cni | Pod networking — assigns VPC IPs to pods |
| coredns | Cluster DNS resolution |
| kube-proxy | Service networking (iptables/ipvs). Not needed with Cilium. |
| aws-ebs-csi-driver | Provision EBS volumes as PersistentVolumes |
| aws-efs-csi-driver | Mount EFS as ReadWriteMany PersistentVolumes |
| eks-pod-identity-agent | Enable EKS Pod Identity credential injection |
| amazon-cloudwatch-observability | Container Insights metrics and logs |
# List installed add-ons aws eks list-addons --cluster-name my-cluster # Get latest available version for an add-on aws eks describe-addon-versions --addon-name aws-ebs-csi-driver \ --kubernetes-version 1.30 \ --query 'addons[0].addonVersions[0].addonVersion' # Install EBS CSI driver aws eks create-addon \ --cluster-name my-cluster \ --addon-name aws-ebs-csi-driver \ --service-account-role-arn arn:aws:iam::123456789:role/ebs-csi-role # Update an add-on aws eks update-addon \ --cluster-name my-cluster \ --addon-name coredns \ --addon-version v1.11.1-eksbuild.4 \ --resolve-conflicts OVERWRITE
The AWS Load Balancer Controller (LBC) provisions ALBs for Ingress resources and NLBs for LoadBalancer Services. Security groups for pods allow attaching SGs directly to pods.
Load Balancer Controller
# Install AWS Load Balancer Controller
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=my-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller
# Ingress → ALB apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: my-app annotations: kubernetes.io/ingress.class: alb alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: ip # pod IPs directly alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:... alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]' spec: rules: - host: api.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-app port: number: 8080 --- # Service → NLB apiVersion: v1 kind: Service metadata: name: my-tcp-service annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing spec: type: LoadBalancer ports: - port: 443 targetPort: 8443
Security groups for pods
ENABLE_POD_ENI=true setting. Each pod with an SG gets a dedicated branch ENI — counts against the instance's ENI limit.# Enable Container Insights (metrics + logs) aws eks create-addon \ --cluster-name my-cluster \ --addon-name amazon-cloudwatch-observability # Or via CloudWatch agent DaemonSet + Fluent Bit ClusterName=my-cluster RegionName=us-east-1 FluentBitHttpPort='2020' FluentBitReadFromHead='Off' curl -s https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluent-bit-quickstart.yaml | \ sed "s/{{cluster_name}}/${ClusterName}/g;s/{{region_name}}/${RegionName}/g" | kubectl apply -f - # Control plane logging aws eks update-cluster-config \ --name my-cluster \ --logging '{"clusterLogging":[{"types":["api","audit","authenticator","controllerManager","scheduler"],"enabled":true}]}' # Query control plane logs in CloudWatch # Log group: /aws/eks/my-cluster/cluster
EKS access entries (new auth model)
# New model: EKS Access Entries (replaces aws-auth ConfigMap) # Enable on cluster creation or migration aws eks update-cluster-config \ --name my-cluster \ --access-config authenticationMode=API_AND_CONFIG_MAP # Grant IAM role cluster-admin access aws eks create-access-entry \ --cluster-name my-cluster \ --principal-arn arn:aws:iam::123456789:role/admin-role \ --type STANDARD aws eks associate-access-policy \ --cluster-name my-cluster \ --principal-arn arn:aws:iam::123456789:role/admin-role \ --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \ --access-scope type=cluster # Legacy: aws-auth ConfigMap kubectl edit configmap aws-auth -n kube-system
Image scanning
# Enable enhanced scanning on ECR (Inspector v2) aws ecr put-registry-scanning-configuration \ --scan-type ENHANCED \ --rules '[{"repositoryFilters":[{"filter":"*","filterType":"WILDCARD"}],"scanFrequency":"CONTINUOUS_SCAN"}]' # Get scan findings for an image aws ecr describe-image-scan-findings \ --repository-name my-app \ --image-id imageTag=latest \ --query 'imageScanFindings.findings[?severity==`CRITICAL`]'
Cluster management
aws eks update-kubeconfig --name clustereksctl create nodegroup ...aws eks list-addons --cluster-name xaws eks update-addon ...
VPC CNI
Pod IPs = real VPC IPs
Prefix delegation: ENABLE_PREFIX_DELEGATION=true
SGs for pods: ENABLE_POD_ENI=true
Check: kubectl describe node | grep Allocatable
Pod Identity
Prefer EKS Pod Identity over IRSAaws eks create-pod-identity-association ...
Verify: aws sts get-caller-identity in pod
Token at: /var/run/secrets/...
Karpenter
NodePool = constraints
EC2NodeClass = AWS configkubectl get nodeclaims
Disruption: consolidationPolicy: WhenEmptyOrUnderutilized
Debugging
kubectl describe pod x — eventskubectl logs -p x — previous containerkubectl get events --sort-by=.lastTimestampaws eks describe-nodegroup ...
Security
Use Access Entries (not aws-auth)
Pod Identity for AWS API calls
Enable ECR enhanced scanning
Pod Security Admission on namespaces