by closeparen on 3/2/21, 5:07 PM
This is such an interesting marketing strategy, I had never thought of selling B2B production infrastructure under the aesthetic of, “Can you believe this shit actually works?”
by majke on 3/2/21, 3:27 PM
by tptacek on 3/2/21, 3:01 PM
I added some example code to the post, because, again, I kind of can't get over how easy this turns out to be. And if you follow the link into Jason's `wireguard-go` code, until you hit gVisor itself, it's not much more complicated under the hood.
Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique.
The whole thing is kind of a vindication for Go's standard library network interface, which I have always hated.
by azalemeth on 3/2/21, 3:51 PM
I know it's not really an HN thing to say, but this is just cool. Reverse ssh tunnels on Wireguard through my VPN are cool enough; the amount of magic here (albeit I think perhaps not totally strictly required magic…) is definitely interesting++.
by vlmutolo on 3/2/21, 4:18 PM
> Normally, this big balloon thingy would be an elaborate scheme to get you to check out our product, but here it's just pointing out some new source code we haven't talked about elsewhere.
I really enjoy this style of writing from a company.
Regarding the article, it seems like Fly has pulled off some insane networking nonsense, but I don’t know enough about networking yet to understand it. Saving this page for later and gonna get back to the TCP/IP Guide.
by anderspitman on 3/2/21, 7:24 PM
This is fantastic. I maintain a list[0] of tunneling software. One of the few downsides of WireGuard is the inability to run it in unprivileged situations. The complexity and performance overhead here might still be too much to edge out solutions like SSH tunnels, but I love that the space is being explored.
I'm hopeful we'll also see some robust QUIC-based tunneling tools over the next couple years.
[0]: https://github.com/anderspitman/awesome-tunneling
by ptomato on 3/2/21, 4:25 PM
Not having been previously familiar with fly's network setup, I gotta say I find it delightful; derived-prefix IPv6 + WG to give you basically static routing + ability to auth on IP is very elegant. I've actually been working on a toy stupid-simple clustering thing that does something similar, and I'm absolutely going to steal the userspace tcp stack over wireguard thing for API access.
by chrisweekly on 3/2/21, 7:50 PM
Amazing. The client API is profoundly simple.
Also, this post prompted me to look closer at Fly.io, and it's leapfrogged to the top of my shortlist for an imminent client "edge proxy" project.
by smithclay on 3/2/21, 11:11 PM
Hacking stuff together using a userspace networking stack is an incredibly fun side project and significantly easier with the gVisor networking libraries written in Go.
Last year I implemented TCP/IP over AWS Cloudwatch. Tons of "can you believe that actually works?" stuff possible with it:
https://medium.com/clog/tcp-ip-over-amazon-cloudwatch-logs-c...
by CyberRabbi on 3/2/21, 4:35 PM
Running networking stacks in user mode really opens up a lot of interesting solutions. Wireguard is sort of an enabling technology for this.
Just realized this was written by security guru tptacek, nice. What is the contextual meaning of “AFFIANT SAYS NOTHING FURTHER.”?
by abrookewood on 3/2/21, 11:55 PM
Man, some people are just next level productive:
"How hard could it be to put together a tiny user-mode TCP, just for the purposes of doing pure-userland WireGuard networking, so people could SSH into instances on Fly without installing WireGuard? I made the mistake of musing about this on a Slack channel I share with Jason Donenfeld. I mused about it just before I went to bed. I woke up. Jason had implemented it, using gVisor, and made it part of the WireGuard library."
by bluesign on 3/2/21, 3:04 PM
Is this super complex infrastructure for fairly simple thing or am I missing something?
by rileymichael on 3/2/21, 3:52 PM
Pretty cool write up. It mentions that every host is running a DNS server that instances have access to, which is being utilized to store the public key (neat!)... is there any way for customers to consume this for other purposes, say out of the box service (instance) discovery?
by tarasglek on 3/2/21, 5:39 PM
The prefix for ssh command looks good for commandline. However, is there a way to hide with some settings in .ssh/config so one can have normal-looking "ssh host" cmdline without special prefixes?
by im3w1l on 3/2/21, 4:03 PM
I read this but I didn't get it at all. I can't see the forest for all the excited talk about particular trees. In simple words, what problem are they trying to solve?
by kerng on 3/2/21, 4:54 PM
Isn't this bad for privacy? Encoding app, org and such information in IP address?
by spockz on 3/2/21, 3:08 PM
> We take Docker-type containers from users and transmogrify them into Firecracker micro-VMs
What is the relationship with micro kernels? Is the feature available separate from the deployment/hosting?
by 0xbadcafebee on 3/2/21, 7:58 PM
> I’ve written a bunch about private networking at Fly. Long story short: it’s like a simpler, IPv6 version of GCP or AWS “Virtual Private Clouds”; we call it “6PN”. When an app instance (a Firecracker micro-VM) is started at Fly, we assign it a special IPv6 prefix; the prefix encodes the app’s ID, the ID of its organization, and an identifier for the Fly hardware it’s running on. We use a tiny bit of eBPF code to statically route those IPv6 packets along our internal WireGuard mesh, and to make sure that customers can’t hop into different organizations.My first thought was "Wow, can we make this _more_ complicated please?", and then I read the rest of the post.
I hate technology.
by sdevonoes on 3/2/21, 4:51 PM
Sounds very cool and all but at the same time it sounds like a terrible thing to maintain in the future.
Perhaps it's just me, but this is something I would accept as a "hey, I was bored and worked on something on my free time. It's probably broken but nobody cares because it's a toy thing, but it's sooo cool". I wouldn't accept it as " Fly.io OKR 1.3 (2021): SSH and User-mode iP WireGuard"... it's sounds pretty much like a hack.
by tasssko on 3/2/21, 11:08 PM
Nice work, I love WireGuard, what it needs is more recognition and definitely more integrations like this.
by boundlessdreamz on 3/2/21, 3:12 PM
Off-topic: What's the software used for the blog?
by resoluteteeth on 3/2/21, 3:04 PM
Maybe you should put this up on github so everyone can use it rather than just talking about how easy it is?