AWS Clusters

Create a self-managed kubeadm or k3s cluster on EC2 instances in your AWS account, in a VPC Ankra creates or one you already own, with Ankra installing and operating Kubernetes.

Choose self-managed EC2 over Amazon EKS when you want the control plane on instances you own, with no EKS control-plane fee, or when the cluster has to join a VPC you already run. Ankra creates the network (or adopts yours), a bastion, the control plane and worker instances, installs Kubernetes with the AWS cloud controller manager and EBS CSI driver, and then operates the cluster - this is an Ankra Managed cluster. The comparison table sets the two side by side.

Before you start

• An AWS credential whose role carries the Provisioning scope (the one write role for EKS and self-managed clusters) or the EC2-only Self-managed scope; access keys with the equivalent policy also work. A provisioning stack launched before the self-managed statements were added must be updated. See AWS Credentials.
• An SSH key credential - your own public key, or one Ankra generates. See SSH Key Credentials.
• A decision on the network. Nothing needs to exist in the account beforehand unless you adopt a VPC.

Created or adopted VPC

• Created VPC (the default; leave out vpc_id). Ankra creates a VPC from network_ip_range (default 10.0.0.0/16, any /16 to /20) with, per availability zone, a private /20 for the nodes and a public /24 for the bastion and NAT gateway, an internet gateway and route tables. It uses one zone when the control plane has fewer than three nodes and three zones otherwise, or the availability_zones you name; more than one zone needs at least three control planes, so etcd keeps quorum when a zone is lost. Teardown removes all of it.
• Adopted VPC (vpc_id). Ankra builds inside a VPC you already run: private node_subnet_ids for the control plane and workers (spread them across zones for a multi-AZ cluster) and a public bastion_subnet_id. Ankra never creates, modifies or deletes your VPC, subnets, internet gateway, NAT gateways, DHCP options or existing route tables. network_ip_range, availability_zones and nat_gateway_single_zone are refused alongside vpc_id, and the subnet fields without it.

Nodes never get a public IP, so they need an egress path (egress_mode):

Mode	Where	What it is	Cost
nat_gateway	Created VPC (the default there)	An Ankra-created NAT gateway per zone, so egress survives a zone outage; nat_gateway_single_zone: true shares one	About USD 32 a month per gateway plus per-GB processing (VPC pricing)
bastion_nat	Either	The bastion forwards node traffic. Egress is capped by the bastion’s network performance, and stops while it restarts or resizes	The bastion and its Elastic IP only
existing	Adopted VPC	Your NAT gateway, NAT instance or transit path already on the node subnets’ route table; Ankra changes nothing	What you already pay

For an adopted VPC, preflight picks the mode when you leave it out: subnets that already have egress get existing, empty subnets without it get bastion_nat. bastion_nat is always refused when a node subnet holds an instance Ankra did not create, because re-pointing that subnet’s default route would take over its egress; create a NAT gateway and use existing instead. Egress modes has the full rules.

Create the cluster

Dashboard

1. Open the create dialog

   Go to Clusters, click Create cluster and pick AWS under Ankra Managed (the AWS card under Cloud Managed creates EKS).

2. Basics

   Name the cluster and pick the AWS credential, the SSH key and the region. A read-only (cost) credential, or a provisioning role whose stack predates the self-managed statements, is listed but refused at preflight with the missing permission named.

3. Network

   Keep Create a new VPC (recommended) - optionally with your own network range and availability zones - or switch to Use an existing VPC and pick the VPC, the node subnets and a public bastion subnet; the wizard shows each subnet’s zone and route. Choose the egress. Bastion allowed IPs starts as 0.0.0.0/0: SSH is key-only, password login is off and sshd is rate-limited on the host, but replace it with your own CIDRs to close the port to everyone else. Ankra’s own addresses stay admitted.

4. Compute

   Pick the control plane count (1, 3 or 5) and instance type, one or more worker node groups, the bastion instance type (a burstable type such as t3.small is enough, unless it is the NAT), the root volume size (an encrypted gp3 volume, 40 GiB by default), the Ubuntu series (24.04 by default) and what happens to EBS volumes on teardown. Instance types load live with prices; only amd64 types are offered.

5. Kubernetes

   Keep kubeadm (the default, Cilium only) or pick k3s, the version, and the etcd topology or CNI. The CNI defaults to Cilium; a k3s cluster can pick Calico or flannel, but only Cilium and Calico enforce the IMDS guard, so flannel gets a preflight warning. See Choices fixed at create time.

6. GitOps

   Optionally connect a Git repository. Two checkboxes are on by default:

   • Include Networking Stack - Traefik, cert-manager and a Let’s Encrypt ClusterIssuer, with Traefik behind a Network Load Balancer the AWS cloud controller creates.
   • Include DNS Integration - a delegated subdomain on ankra.cc with external-dns wired, so Ingress hostnames publish their DNS records automatically.
7. Review

   Click Run preflight. It reports the network ownership, the zones it resolved, the egress mode, the AMI, and the vCPU, VPC, NAT gateway, Elastic IP and security-group quotas. Once nothing is an error, click Create cluster and follow the progress view.

CLI

# Provisioning role: print the external ID and CloudFormation link, launch the stack, then store the role
ankra credentials aws onboarding --scope provisioning      # or --scope self_managed for the EC2-only role
ankra credentials aws create-role --name my-aws --role-arn <role-arn> --external-id <external-id> \
  --region eu-north-1 --scope provisioning
# Or access keys (the secret is asked for on a masked prompt)
ankra credentials aws create-keys --name my-aws-keys --access-key-id <access-key-id> --region eu-north-1

ankra credentials hetzner ssh-key create --name my-ssh-key --generate   # any SSH key credential works

The external ID must be the one onboarding printed and the stack was launched with; a role stored at the cost scope is refused at create. Look up what the credential can reach with ankra cluster aws regions, availability-zones, instance-types, images and pricing (all take --credential-id and --region), and vpcs and subnets --vpc-id <vpc-id> when you adopt a VPC.

preflight takes the same flags as create and creates nothing. A three-zone cluster in a created VPC:

ankra cluster aws preflight \
  --name my-cluster \
  --credential-id <aws-credential-id> \
  --ssh-key-credential-id <ssh-key-credential-id> \
  --region eu-north-1 \
  --bastion-allowed-ips 203.0.113.10/32 \
  --control-plane-count 3 \
  --control-plane-type t3.medium \
  --worker-count 3 \
  --worker-type t3.large
# When it reports no errors, run the same command with create instead of preflight

--network-ip-range, --availability-zones, --egress-mode nat_gateway|bastion_nat and --nat-gateway-single-zone shape a created network. To adopt a VPC instead:

ankra cluster aws create \
  --name my-cluster \
  --credential-id <aws-credential-id> \
  --ssh-key-credential-id <ssh-key-credential-id> \
  --region eu-north-1 \
  --vpc-id <vpc-id> \
  --node-subnet-ids <subnet-a>,<subnet-b>,<subnet-c> \
  --bastion-subnet-id <public-subnet> \
  --egress-mode existing \
  --bastion-allowed-ips 203.0.113.10/32 \
  --control-plane-count 3 \
  --worker-count 3

--distribution k3s, --cni, --cni-features, --retention-policy retain|delete, --environment, --criticality and the --gitops-* flags cover the other choices. Every flag is in the CLI reference.

API

Preflight takes the same body as create and answers with findings - can_proceed, network_ownership, resolved_availability_zones and resolved_egress_mode - instead of a cluster:

curl -X POST https://platform.ankra.app/api/v1/clusters/aws/preflight \
  -H "Authorization: Bearer $ANKRA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-cluster",
    "credential_id": "<aws-credential-id>",
    "ssh_key_credential_id": "<ssh-key-credential-id>",
    "region": "eu-north-1",
    "availability_zones": ["eu-north-1a", "eu-north-1b", "eu-north-1c"],
    "egress_mode": "nat_gateway",
    "bastion_allowed_ips": ["203.0.113.10/32"],
    "control_plane_count": 3,
    "control_plane_type": "t3.medium",
    "node_groups": [
      {"name": "default", "instance_type": "t3.large", "count": 3}
    ],
    "retention_policy": "retain"
  }'

Send the same body to POST /api/v1/clusters/aws to create the cluster. It answers cluster_id, kind: "aws", state: "creating" and an operation_id that tracks provisioning. To adopt a VPC, replace the network fields with "vpc_id", "node_subnet_ids", "bastion_subnet_id" and "egress_mode": "existing" or "bastion_nat"; the created-network fields are then refused with 422. Every field and default is in the AWS Reference. Ankra’s AI can create the cluster in chat too, running the preflight for you.

Verify

1. The cluster moves from Provisioning to Online in the clusters list once the Ankra Agent has connected.

2. Point kubectl at it through Ankra (this needs a Cluster Access grant) and check the nodes:

   ankra cluster kubeconfig add my-cluster --use
   kubectl get nodes
   

   Every control plane and worker should be Ready. See Accessing Clusters with kubectl.

What Ankra created in your account

• The network, in a created VPC: the VPC, subnets, internet gateway, route tables and, in nat_gateway mode, NAT gateways with their Elastic IPs. In an adopted VPC, at most one route table (bastion_nat mode).
• Two security groups (nodes and bastion), an imported key pair, one Elastic IP for the bastion, and the bastion, control plane and worker instances with encrypted gp3 root volumes and no public IPs on the nodes.
• Two IAM roles with instance profiles, ankra-k3s-<cluster>-cp and ankra-k3s-<cluster>-node, that the cloud controller manager and EBS CSI driver use. The credential you gave Ankra is never copied into the cluster.
• The AWS cloud-provider stack: the cloud controller manager (load balancers for LoadBalancer Services), the EBS CSI driver (encrypted gp3 volumes by default) and the IMDS guard.

Everything carries ankra.cloud/cluster-id=<id>, ankra.cloud/managed=true and kubernetes.io/cluster/<id>=owned, and the credential’s write permissions are scoped to those tags. What Ankra creates has the full list and the network diagram.

AWS specifics

• Node groups follow the node subnets. New nodes are balanced across the cluster’s zones - each takes the zone with the fewest instances - unless the group is pinned with --availability-zone (API availability_zone). Pin a group that runs zonal storage: an EBS volume cannot attach from another zone. Every node carries topology.kubernetes.io/zone and topology.kubernetes.io/region, so topologySpreadConstraints work as they are.
• An instance type change replaces nodes. EC2 cannot resize a running instance, so each node of the group is drained, terminated and recreated on the new type, one at a time.
• Control plane. A cluster whose nodes span more than one zone needs three control planes and cannot go below three. ankra cluster aws control-plane get|set-count|set-instance-type manage it.
• Restart is an EC2 reboot; if the instance does not respond it falls back to a stop and start, which keeps the private IP. ankra cluster aws nodes cloud-init-log reads a node’s cloud-init status over the bastion. OS-level diagnosis and repair over SSH run through Ankra’s AI (ssh_diagnose_node, ssh_repair_node).
• Bastion. Resize it from Nodes → Bastion & VMs or with PUT /api/v1/clusters/aws/{cluster_id}/bastion/instance-type; there is no CLI verb on AWS. The instance is stopped, changed and started, keeping its Elastic IP. SSH as ubuntu through it (ankra cluster aws access-info); only bastion_allowed_ips and Ankra’s addresses reach it.
• Upgrades move the AWS cloud controller manager with the nodes: its minor version is pinned to the cluster’s Kubernetes minor.

In bastion_nat mode the bastion is every node’s default route, so a bastion resize or restart stops image pulls, the Ankra Agent connection and the cloud controller’s AWS calls until it is back.

Stopping and starting

A stop stops every node and the bastion rather than terminating them, so the instances keep their root volumes and private IPs, and the bastion keeps its Elastic IP. Start brings the same instances back and etcd rejoins on the same addresses, so the cluster returns with its state, not as a rebuild. Ankra records each instance’s private IP at create and checks it after every start; a difference is reported as drift. While stopped, compute is not billed, but EBS storage, the Elastic IP, load balancers and - in a created VPC - the NAT gateways are, so a cluster parked for weeks on nat_gateway egress is cheaper to terminate. A stop never deletes the volumes the CSI driver provisioned, whatever the retention_policy.

Terminating

A terminate terminates the instances, releases the bastion’s Elastic IP, and deletes the security groups, key pair and IAM roles. In a created VPC it then removes the network in reverse order - NAT gateways with their Elastic IPs, route tables, subnets, internet gateway, VPC - by the tags Ankra set. In an adopted VPC it restores each node subnet’s previous route table association before deleting the one route table it created, and leaves the rest of your network alone.

EBS volumes the CSI driver provisioned, and load balancers the cloud controller created, follow the cluster’s retention_policy, chosen at create:

• retain (the default) keeps them in your account, where they keep billing. The teardown result lists their IDs, and they carry the cluster ID tag. --force still deletes the load balancers, never the volumes, and no volume question is asked.
• delete removes those tagged ankra.cloud/managed=true for this cluster - the EBS volumes only once you accept it.

With a created VPC, load balancers are always deleted, because they cannot outlive the VPC; volumes still follow the policy.

ankra cluster aws deprovision <cluster>                            # names the EBS volumes a delete cluster loses, and asks
ankra cluster aws deprovision <cluster> --accept-volume-data-loss  # for scripts

Operate it

Day-2 tasks work the same way on every Ankra Managed provider, with the CLI name aws:

• Node groups and legacy worker scaling
• Control plane
• Restart a node and the bastion
• SSH access and keys
• Upgrade Kubernetes
• Stop and start
• Terminate a cluster

Troubleshooting

Issue	Solution
Preflight refuses the credential	It is read-only (cost scope), or its provisioning stack predates the self-managed statements. Add a provisioning credential or update the stack to the published template
The preflight cidr item is an error	network_ip_range does not hold a private /20 and a public /24 per zone - a /20 VPC fits one zone. Widen the range (up to /16) or use fewer zones
Preflight refuses the zones	A named zone does not exist or is not available, or several zones were given with fewer than three control planes. Check ankra cluster aws availability-zones
Preflight reports the VPC or NAT gateway quota	The account is at its VPCs-per-region (L-F678F1CE) or NAT-gateways-per-zone (L-FE5A380F) limit. Request an increase, remove unused clusters, or use bastion_nat or nat_gateway_single_zone
Preflight refuses bastion_nat (adopted VPC)	A node subnet holds an instance Ankra did not create. Pick an empty subnet, or create a NAT gateway and use existing
Preflight refuses existing (adopted VPC)	The node subnets have no 0.0.0.0/0 route to a NAT gateway, NAT instance or transit path. Add one, or use empty subnets with bastion_nat
node_subnet_ids or network_ip_range refused with 422	Subnet fields belong to an adopted VPC and network fields to a created one - send vpc_id with subnets, or neither
bastion_allowed_ips refused	An entry is not an IPv4 CIDR. 0.0.0.0/0 and ::/0 are accepted, and an empty list means 0.0.0.0/0
Cluster stuck in provisioning	Check the vCPU quota for the instance family, the Elastic IP and NAT gateway quotas, and - in an adopted VPC - that the bastion subnet routes to the internet gateway
Nodes stay NotReady with the node.cloudprovider.kubernetes.io/uninitialized taint	The cloud controller cannot match the node to its instance - see below
LoadBalancer Service stuck in Pending	Check the cloud controller pods in kube-system. On a fresh account the first load balancer also creates the Elastic Load Balancing service-linked role, which the credential allows
PVC stuck in Pending	Check the EBS CSI controller pods; a pod pinned to a zone with no volume there needs a node group in that zone
Cannot SSH to the bastion	Your address is not in bastion_allowed_ips, or the security group was edited outside Ankra
Nodes lost egress after a bastion resize	Expected in bastion_nat mode while the bastion restarts
Create result reports imds_guard: unavailable	The cluster runs flannel, which cannot enforce the IMDS guard - see below

Cloud controller cluster ID and node names. The cloud controller finds a cluster’s instances by the kubernetes.io/cluster/<id>=owned tag and matches each node to an instance by name. Ankra sets both: the tag on every instance, and each node name to the instance’s EC2 private DNS name as DescribeInstances reports it (k3s gets --node-name), so a VPC whose DHCP options set carries a custom domain-name does not break the match, and changing the DHCP options later affects no node. Removing or editing the tag in the console makes the controller treat the instance as foreign until the tag is back. If a node was renamed or re-registered by hand, delete its Kubernetes node object and restart the node from Nodes; it registers again under the right name.

Instance metadata (IMDS). Every instance requires IMDSv2, so tools that use IMDSv1 fail on the nodes. Workers and the bastion have a hop limit of 1, so pods cannot reach the instance role. Control planes have a hop limit of 3 so the EBS CSI controller can use the control-plane role; the IMDS guard, a network policy rendered for Cilium and Calico, blocks every other pod from 169.254.169.254. On flannel there is no policy engine, so a pod scheduled onto a control plane can read that role’s credentials: prefer Cilium or Calico, or keep the control planes tainted. Give a workload that needs AWS access its own credentials; do not raise the hop limits or re-enable IMDSv1 - Ankra restores the hardening on reconciliation.

AWS service quotas. Preflight checks the quotas that most often block a create, but a quota can be used up between preflight and create, and the per-cluster quota report answers 404 for AWS. If provisioning fails with a quota error, check the Service Quotas console for the region - On-Demand vCPUs for the instance family, VPCs per region, NAT gateways per zone, Elastic IPs (one for the bastion plus one per NAT gateway), security groups per interface and rules per group, and Network Load Balancers - then create again. A failed create leaves only tagged objects, which Ankra removes on the next attempt.