Supporting cross-account/ multi-account data plane nodes with Karpenter
Description
What problem are you trying to solve?
We want to support Kubernetes clusters whose EKS control plane is in a different AWS account from their EC2 data-plane nodes, and ideally enable one Karpenter installation to manage nodes across multiple data-plane accounts.
This would reduce the cost and operational complexity of running a separate control plane for every data-plane account. The initial idea is to add a NodeClass configuration for the target AWS account and a role Karpenter can assume. Since EC2 instances belong to the account whose credentials create them, Karpenter could use account-specific credentials selected by the NodeClass to create and manage those instances. That identity should also be used to discover the appropriate subnet, VPC, security groups, AMIs, and related resources in the target account.
There are several feasibility and design questions that need to be resolved as part of this feature:
EKS and network topology
- Confirm that EC2 instances in a data-plane account can join and operate as nodes in an EKS cluster owned by a separate control-plane account. This should not be conflated with EKS Hybrid Nodes, which target external/on-premises nodes and do not enable Karpenter to provision cross-account EC2 instances.
- Define the supported network topology: independently owned VPCs connected via peering/TGW, shared VPCs, or another model.
- Define the required API-endpoint reachability, DNS, routes, and security-group rules between the target-account VPC and the EKS control-plane ENIs.
- Define node bootstrap and authentication: how nodes retrieve the cluster endpoint/CA and how the target-account node IAM role is authorized through EKS access entries or
aws-auth, as applicable.
API, IAM, and security model
- Define a per-NodeClass target account/region and controller assume role. Make the resolved account visible in NodeClass status and events.
- Specify trust policies, ExternalId/session tags, permission boundaries,
iam:PassRole, and an allowlist of accounts/roles that Karpenter may assume. - Ensure a user able to create or modify a NodeClass cannot cause Karpenter to assume an arbitrary role/account.
- Define how Karpenter creates, refreshes, caches, and isolates account-scoped AWS clients, including throttling and partial-account failure behavior.
- Ensure the target account owns the node role and instance profile; cover instance-profile creation/adoption, tagging, collision avoidance, garbage collection, and cleanup.
Resource discovery and lifecycle
- Resolve all account-scoped resources using target-account credentials: subnets, security groups, AMIs, KMS keys, capacity reservations/capacity blocks, launch-template-related resources, and tags/IAM conditions.
- Handle availability-zone identity correctly. Names such as
us-east-1aare account-specific aliases and do not reliably identify the same physical AZ across accounts; use AZ IDs or resolve zone names in the target account. - Retain the target account for the entire NodeClaim lifecycle, not only launch: instance discovery, termination, drift, consolidation, disruption, node repair, controller restart recovery, and orphan cleanup.
Interruption handling and operability
- Design multi-account handling for Spot interruption, rebalance recommendation, and scheduled-maintenance events. These are account-local, so the feature needs either one queue/event source per target account or secure cross-account EventBridge/SQS forwarding to a central consumer.
- Include target account identity in NodeClaim status, Kubernetes events, logs, and metrics.
- Account for per-account EC2 quotas, API throttling, lost target-role permissions, and target-account outages.
Suggested acceptance criteria
- A documented, supported EKS and network topology for cross-account EC2 nodes.
- A NodeClass API and authorization model for an allowlisted target account and assume role.
- Target-account discovery and provisioning of all EC2/IAM resources required by a node.
- Successful node bootstrap and Kubernetes authentication from a target account.
- Complete lifecycle support: launch, interruption handling, drift/consolidation, termination, and cleanup.
- Multi-account interruption-event ingestion.
- Status, events, logs, and metrics that identify the target account.
- E2E coverage with at least two data-plane accounts, including target-account permission loss and failure cases.
How important is this feature to you?
This feature would reduce cost and complexity for deployments that need to isolate compute/data-plane resources by AWS account while operating a shared Kubernetes control plane.
cc @jmdeal @DerekFrank
- Please vote on this issue by adding a reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Source: aws/karpenter-provider-aws