undefined | Better HN

0 pointslobster_johnson9y ago0 comments

Kubernetes does this for you IPs. For example, if your VPC subnet is 172.16.0.0/16, then you can tell K8s to use 10.0.0.0/16.

AWS won't know this IP range and won't route it. So K8s automatically populates your routing table with the routes every time a node changes or is added/removed.

K8s will give a /24 CIDR to each minion host, so the first will get 10.0.1.0/24, the next 10.0.2.0/24, and so on. Each pod will get 10.0.1.1, 10.0.1.2, etc.

Obviously having an additional IP/interface per box adds complexity, but I don't know if K8s supports any other automatic mode of operation on AWS.

(Note: Kubernetes expects AWS objects that it can control — security groups, instances, etc. — to be tagged with KubernetesCluster=<your cluster name>. This also applies to the routing table.)

0 comments

bboreham9y ago

OK, I see this is the same as what Flannel does in its aws-vpc backend, but I though you were saying you could do better. Maybe I mis-parsed what you said.

If you're adding a routing rule for every minion then you will also hit the 50 limit in AWS routing tables.

lobster_johnsonOP9y ago

Sorry about the confusion — yes, absolutely. One option is to ask AWS to increase it.

Flannel is just one of many different options if you need to go beyond 50 nodes. It seems some people use Flannel to create an overlay network, but this isn't necessary. You can use the host-gw mode to accomplish the same thing as Kubernetes' default routing-table-updating behaviour, but with routing tables maintained on each node instead.

j / k navigate · click thread line to collapse