Add support for EC2 bandwidth weighting configuration (NetworkPerformanceOptions.BandwidthWeighting) in the EC2NodeClass spec
Tell us about the problem you're trying to solve
8th-gen EC2 instances (M8, C8, R8, X8 families) support configurable bandwidth weighting, where you can shift baseline bandwidth between networking and EBS using vpc-1 (more network) or ebs-1 (more EBS). This is set at launch time via the NetworkPerformanceOptions parameter in the launch template.
For workloads that use local NVMe (instance store) and don't need EBS bandwidth - such as Spark on EKS with NVMe shuffle — vpc-1 provides free additional network headroom for S3 and shuffle traffic at no extra cost.
Currently there is no way to configure this in Karpenter:
- EC2NodeClass does not expose a networkPerformanceOptions field
- Custom launch templates were removed in v0.33+
- modify-instance-network-performance-options API requires the instance to be in Stopped state, so userData-based modification is not possible
How are you currently working around this?
No workaround available. We are tuning S3A connection parallelism at the application level to stay within the default bandwidth envelope, but cannot take advantage of the increased networking baseline that vpc-1 provides.
Proposed solution
Add spec.networkPerformanceOptions to EC2NodeClass:
yaml apiVersion: karpenter.k8s.aws/v1 kind: EC2NodeClass metadata: name: high-network spec: networkPerformanceOptions: bandwidthWeighting: vpc-1
Include NetworkPerformanceOptions in the generated launch template when this field is set. The EC2 CreateLaunchTemplateVersion API already supports NetworkPerformanceOptions.
Additional context
- Supported instance families: M8a, M8g, M8gd, M8i, M8id, M8i-flex, C8a, C8g, C8gd, C8i, C8id, C8i-flex, R8a, R8g, R8gd, R8i, R8id, R8i-flex, X8g, X8aedz, X8i
- No additional cost
- Use case: EMR on EKS with Spark workloads on R8gd instances using NVMe for shuffle, where network bandwidth for S3 access is the bottleneck
Source: aws/karpenter-provider-aws