from Hacker News

A brief guide to Kubernetes networking

by thewizl on 3/16/23, 12:59 PM with 19 comments

• by jmillikin on 3/16/23, 1:59 PM

  For folks interested in the Kubernetes networking model, I recommend looking elsewhere. This post appears to be content marketing wrapped in a thin shell of introductory tutorial for an approach that isn't used by non-trivial deployments.

  First, if you want to do a networking tutorial, start with something simple -- an HTTP server and curl, for example. You want to be able to tcpdump the traffic to understand what's going on under the hood. The first half of the blog post is some massively complex magic -- CRDs? An operator? Just why? -- and there's no reason to run an Elasticsearch instance just to test out packet routing.

  Second, you almost certainly don't want to use kube-proxy. It uses (abuses?) iptables/nftables in a way that will make your sysadmins cry tears of blood. For small deployments, every major cloud provider (AWS, Azure, GCP, etc) has a CNI plugin that lets you allocate pod IPs out of a dedicated NAT prefix. For larger or bare-metal deployments, either use IPv6 natively (if available) or 6to4 (if on an IPv4-only network). I wrote a tutorial on the 6to4 approach[0], but honestly if you have someone on staff who is familiar with the Linux kernel network configs they'll probably have a better idea of how to set it up to work with your system.

  Third, you probably want to avoid getting super-magical with your DNS. Approaches like that described in the article (coredns configured to directly resolve non-namespaced Kubernetes service names) have poor performance once you get beyond toy-sized clusters, and having to hunt down all the places your code does a single-name lookup is not fun. Instead, configure a "normal" DNS server (or equivalent non-DNS address resolver) to read Kubernetes-announced endpoints in bulk (with caching, etc), and use hostnames like `myservice.mynamespace.mycluster.yourproddomain.com`, which lets you (1) figure out where your packets are getting routed to, and (2) provision mTLS certificates to pods that let them authenticate themselves as a given service identity. Yes, it's longer, but your future self (or future underlings) will thank you.

  [0] https://john-millikin.com/stateless-kubernetes-overlay-netwo...

• by ilovecaching on 3/16/23, 2:41 PM

  This is definitely just an ad for ergomake, there's barely any networking knowledge here at all.

• by ianpurton on 3/16/23, 2:03 PM

  Does anyone know how they make those hand drawn style kubernetes diagrams?

• by revskill on 3/16/23, 2:40 PM

  Wait, if the documentation failed at explaining core technologies for a 5 years old baby to understand, it's a scam.

• by lifty on 3/16/23, 2:13 PM

  Question to the author. How does ergomake do multi tenancy on K8s?

• by Already__Taken on 3/16/23, 2:41 PM

  honestly I learnt a lot about k8s networking from reading how ciliums bgp replacement for kube proxy works. real nice docs and some good diagrams.