The main difference between the Amazon EKS-optimized AMI (amazon-eks-node-1.29) and the Bottlerocket AMI (bottlerocket-aws-k8s-1.29) lies in their purpose
See the summary below this highlight
13 Matching Annotations
- Mar 2025
-
repost.aws repost.aws
-
-
aws.amazon.com aws.amazon.com
-
Reduce container startup time on Amazon EKS with Bottlerocket data volume
-
Introduction
- Containers are widely used for scalable applications but face challenges with startup times for large images (e.g., AI/ML workloads).
- Pulling large images from Amazon Elastic Container Registry (ECR) can take several minutes, impacting performance.
- Bottlerocket, an AWS open-source Linux OS optimized for containers, offers a solution to reduce container startup time.
-
Solution Overview
- Bottlerocket's data volume feature allows prefetching container images locally, eliminating the need for downloading during startup.
- Prefetching is achieved by creating an Amazon Elastic Block Store (EBS) snapshot of Bottlerocket's data volume and mapping it to new Amazon EKS nodes.
- Steps to implement:
- Spin up an Amazon EC2 instance with Bottlerocket AMI.
- Pull application images from the repository.
- Create an EBS snapshot of the data volume.
- Map the snapshot to Amazon EKS node groups.
-
Benefits of Bottlerocket
- It separates OS and container data volumes, ensuring consistency and security during updates.
- Prefetched images significantly reduce startup times for large containers.
-
Implementation Walkthrough
- Step 1: Build EBS Snapshot
- Automate snapshot creation using a script.
- Prefetch images like Jupyter-PyTorch and Kubernetes pause containers.
- Export the snapshot ID for use in node group configuration.
- Step 2: Setup Amazon EKS Cluster
- Create two node groups:
no-prefetch-mng: Without prefetched images.prefetch-mng: With prefetched images mapped via EBS snapshot.
- Step 3: Deploy Pods
- Test deployment on both node groups.
- Prefetched nodes start pods in just 3 seconds, compared to 49 seconds without prefetching.
- Step 1: Build EBS Snapshot
-
Results
- Prefetching reduced container startup time from 49 seconds to 3 seconds, improving efficiency and user experience.
-
Further Enhancements
- Use Karpenter for automated scaling with Bottlerocket nodes.
- Automate snapshot creation in CI pipelines using GitHub Actions.
-
Cleaning Up
- Delete AWS resources (EKS cluster, Cloud9 environment, EBS snapshots) to avoid charges after testing.
-
Conclusion
- Bottlerocket's data volume prefetching dramatically enhances container startup performance for large workloads on Amazon EKS.
-
-
- Jun 2024
-
spacelift.io spacelift.io
-
Neither of the methods shown above are ideal in environments where you require several clusters or need them to be provisioned in a consistent way by multiple people.
In this case, IaC is favored over using EKS directly or manually deploying on EC2
-
Running a cluster directly on EC2 also gives you the choice of using any available Kubernetes distribution, such as Minikube, K3s, or standard Kubernetes as deployed by Kubeadm.
-
EKS is popular because it’s so simple to configure and maintain. You don’t need to understand the details of how Kubernetes works or how Nodes are joined to your cluster and secured. The EKS service automates cluster management procedures, leaving you free to focus on your workloads. This simplicity can come at a cost, though: you could find EKS becomes in-flexible as you grow, and it might be challenging to migrate from if you switch to a different cloud provider.
Why use EKS
-
The EKS managed Kubernetes engine isn’t included in the free tier. You’ll always be billed $0.10 per hour for each cluster you create, in addition to the EC2 or Fargate costs associated with your Nodes. The basic EKS charge only covers the cost of running your managed control plane. Even if you don’t use EKS, you’ll still need to pay to run Kubernetes on AWS. The free tier gives you access to EC2 for 750 hours per month on a 12-month trial, but this is restricted to the t2.micro and t3.micro instance types. These only offer 1 GiB of RAM so they’re too small to run most Kubernetes distributions.
Cost of EKS
-
Some of the other benefits of Kubernetes on AWS include
Benefits of using Kubernetes on AWS: - scalability - cost efficiency - high availability
Tags
Annotators
URL
-
- Apr 2024
-
-
To address the issues of CAS, Karpenter uses a different approach. Karpenter directly interacts with the EC2 Fleet API to manage EC2 instances, bypassing the need for autoscaling groups.
Karpenter
-
The problem occurs when you want to move the pod to another node, in cases such as cluster rebalancing, spot interruptions, and other events. This is because the EBS volumes are zonal bound and can only be attached to EC2 instances within the zone they were originally provisioned in.This is a key limitation that CAS is not able to take into an account when provisioning a new node.
Key limitation of CAS
-
Since Karpenter can schedule nodes quicker, it will most often win this race and provide a new node for the pending workload. CAS will still attempt to create a new node, however will be slower and will most likely have to remove the node after some time, due to emptiness. This brings unnecessary costs to your cloud bill
-
It’s worth mentioning that Cluster Autoscaler and Karpenter can co-exist within the same cluster.
-
- Nov 2023
-
aws.amazon.com aws.amazon.com
-
You can now run Amazon EKS clusters on a Kubernetes version for up to 26 months from the time the version is generally available on Amazon EKS.
-
- Mar 2023
-
tomaszdudek.substack.com tomaszdudek.substack.com
-
You can freely replace SageMaker services with other components as your project grows and potentially outgrows SageMaker.
-