• Core Value Proposition:

  • Combines Amazon EKS Auto Mode (automating infrastructure and compute layer management) with Istio Ambient Mesh (automating service-to-service networking and security) to significantly reduce manual operational overhead and strengthen security.

• Amazon EKS Auto Mode Key Components:

  • Managed Instances: AWS fully controls the lifecycle, patching, and security configurations of nodes; direct SSH access is removed in favor of Kubernetes-native troubleshooting.
  • Bottlerocket-based OS: Nodes utilize Bottlerocket, a minimal, immutable, and container-optimized Linux distribution enforcing strict security boundaries via SELinux.
  • Built-in System Components: Core add-ons—including Amazon VPC CNI, kube-proxy, Amazon EBS CSI driver, CoreDNS, and AWS Load Balancer Controller—are managed directly by AWS as system processes to eradicate version compatibility friction and minimize the threat surface.
  • Karpenter-powered Scaling: Employs a custom integrated version of Karpenter that dynamically provisions right-sized instances based on pod requests, continuously evaluates consolidation opportunities, and optimizes workloads onto Spot instances where possible.

• Istio Ambient Mesh Capabilities:

  • Sidecarless Architecture: Shifts traffic security and policy enforcement out of individual application pods into a split infrastructure model, decoupling service networking from application lifecycles and significantly cutting down resource overhead.
  • Layer 4 Security (ztunnel): Uses a secure overlay network via node-level ztunnel proxies to enforce zero-trust capabilities like automatic mutual TLS (mTLS) encryption, L4 authorization policies, and TCP-level observability.
  • Layer 7 Capabilities (Waypoint proxies): Provisions optional Layer 7 Waypoint proxies externally to implement advanced traffic routing, circuit breaking, and rich cryptographic or application-layer policy enforcement without sidecars.

• Integration and Workload Onboarding:

  • Unified Automation: Offloads data-plane management entirely to AWS while simultaneously removing sidecar proxy complexities from the application architecture.
  • Incremental Mesh Onboarding: Workloads can be added selectively to the ambient mesh simply by applying the label istio.io/dataplane-mode=ambient at either the namespace level or to individual target pods.

• AWS EKS costs can escalate due to massive, parallel workloads in life sciences/drug development (e.g., genomic sequencing, molecular modeling).
• Default Kubernetes scheduler uses leastAllocated strategy, spreading pods across many nodes for fairness/high availability.
• leastAllocated strategy causes many partially utilized nodes, preventing autoscalers from scaling down idle nodes, increasing costs.
• mostAllocated scheduling strategy "packs" pods onto fewer nodes, maximizing utilization and enabling autoscalers like Karpenter to remove idle nodes.
• Switching to mostAllocated can reduce runtime costs significantly (e.g., ~10% in UAT, 43% in PROD environments).
• Custom scheduler deployment on AWS EKS requires creating a service account, ClusterRoleBindings, RoleBinding, a ConfigMap with the mostAllocated scoring strategy, and a deployment with a matching Kubernetes version container image.
• Resource weights can prioritize packing of expensive resources (e.g., high weight on GPUs for ML workloads).
• Testing in non-production environments is recommended before full rollout.
• Implementing mostAllocated scheduling can dramatically optimize costs by enabling cluster autoscalers to shut down unused nodes.

See the summary below this highlight

• Introduction

  • Containers are widely used for scalable applications but face challenges with startup times for large images (e.g., AI/ML workloads).
  • Pulling large images from Amazon Elastic Container Registry (ECR) can take several minutes, impacting performance.
  • Bottlerocket, an AWS open-source Linux OS optimized for containers, offers a solution to reduce container startup time.

• Solution Overview

  • Bottlerocket's data volume feature allows prefetching container images locally, eliminating the need for downloading during startup.
  • Prefetching is achieved by creating an Amazon Elastic Block Store (EBS) snapshot of Bottlerocket's data volume and mapping it to new Amazon EKS nodes.
  • Steps to implement:
  • Spin up an Amazon EC2 instance with Bottlerocket AMI.
  • Pull application images from the repository.
  • Create an EBS snapshot of the data volume.
  • Map the snapshot to Amazon EKS node groups.

• Benefits of Bottlerocket

  • It separates OS and container data volumes, ensuring consistency and security during updates.
  • Prefetched images significantly reduce startup times for large containers.

• Implementation Walkthrough

  • Step 1: Build EBS Snapshot
    • Automate snapshot creation using a script.
    • Prefetch images like Jupyter-PyTorch and Kubernetes pause containers.
    • Export the snapshot ID for use in node group configuration.
  • Step 2: Setup Amazon EKS Cluster
    • Create two node groups:
    • no-prefetch-mng: Without prefetched images.
    • prefetch-mng: With prefetched images mapped via EBS snapshot.
  • Step 3: Deploy Pods
    • Test deployment on both node groups.
    • Prefetched nodes start pods in just 3 seconds, compared to 49 seconds without prefetching.

• Results

  • Prefetching reduced container startup time from 49 seconds to 3 seconds, improving efficiency and user experience.

• Further Enhancements

  • Use Karpenter for automated scaling with Bottlerocket nodes.
  • Automate snapshot creation in CI pipelines using GitHub Actions.

• Cleaning Up

  • Delete AWS resources (EKS cluster, Cloud9 environment, EBS snapshots) to avoid charges after testing.

• Conclusion

  • Bottlerocket's data volume prefetching dramatically enhances container startup performance for large workloads on Amazon EKS.

In this case, IaC is favored over using EKS directly or manually deploying on EC2

Why use EKS

Cost of EKS

Benefits of using Kubernetes on AWS: - scalability - cost efficiency - high availability

Karpenter

Key limitation of CAS