by spacebuffer on 3/30/24, 7:35 AM with 192 comments
by lopkeny12ko on 3/30/24, 11:16 AM
For example, there's no reasonable world where that 200 MB blob is not cached and served over CDN. I can't imagine someone would be so proud that their application server isn't reading 200 MB from disk and copying those bytes to the client on every download; it's just so obviously poor design.
by ThePhysicist on 3/30/24, 9:01 AM
by oefrha on 3/30/24, 9:51 AM
If I were to be hit by such an "attack" myself I probably wouldn't even notice until Cloudflare sends me that monthly "X TB of data transferred, something close to 100% bandwidth saved" email.
I like the app btw, can recommend.
by PreInternet01 on 3/30/24, 9:31 AM
As long as such abuse doesn't cause monetary or resource exhaustion concerns, it's quite OK to ignore it, but stories like "whelp, turns out that 80% of the capacity of our auto-scaling fleet is not doing anything useful" are depressingly common enough to at least keep an eye on things.
My annoyance with this kind of abuse revolves mostly around logging: a majority of logs just showing the same set of hosts displaying the same pointless behavior over and over again. Again, not a huge issue if your log storage is cheap and plentiful (as it should be), but having some kind of way to automatically classify certain traffic as abusive and suppress routine handling of that is definitely a good idea.
It's also a lot harder than it sounds! I can't count the number of times I've added classification logic to my inbound SMTP server that should pick up on outright, silly abuse (of which there is a lot when dealing with email), only to have it triggered by some borderline-valid scenario as well.
Spending way too much time on going down successive rabbit holes is a great way not to get any real work done -- a great reason to outsource, or, if that's too much work as well or just too expensive, indeed just ignore the abuse, annoying though it is...
by sethammons on 3/30/24, 10:29 AM
Their advice isn't bad per se, but their numbers are not a testament to it. I expect for my Go HTTP API services to handle 5k requests per second on a small to medium VPS when there is some DB activity and some JSON formatting without doing any optimizations. This is based on deploying dozens of similar services while working at a place that got multiple billions of requests per day, spiking to over 500k rps.
by headmelted on 3/30/24, 9:09 AM
They’re advocating deploying a binary as preferable to using docker, fair enough, but what about the host running the binary? One of the reasons for using containers is to wrap your security hardening into your deployment so that anytime you do need to scale out you have confidence your security settings are identical across nodes.
On that, the monolith talked about here can be hosted on a single VPS, again that’s great (and cheap!), but if it crashes or the hardware fails for any reason that’s potentially substantial downtime.
The other worry I’d have is that tying everything into the monolith means losing any defence in depth in the application stack - if someone does breach your app through the frontend then they’ll be able to get right through to the backend data-store. This is one of the main reasons people put their data store behind an internal web service (so that you can security group it off in a private network away from the front-end to limit the attack surface to actions they would only have been able to perform through a web browser anyway).
by sameoldtune on 3/30/24, 10:51 AM
by mastermedo on 3/30/24, 8:53 AM
by NKosmatos on 3/30/24, 8:43 AM
“… => Thus, we build a monolith service for each app, which is easy to deploy and maintain. No Docker, no Kubernetes, no dependencies, no runtime environment - just a binary file that can be deployed on any newly created VPS. …”
by filleokus on 3/30/24, 9:36 AM
Depending on the distribution of the traffic they might have survived well on VPS's without Cloudflare anyways, doesn't seem that large. Would be interesting to see more detailed stats of rps and how much (if any) Cloudflare stopped before they got it.
Russian layer7 ddos'es that I know of targeting Swedish companies have been large enough that major providers run into capacity problems and fall over (including Verizon, Azure Frontdoor, Cloudflare, GCP's Load balancer). This strategy would absolutely not work against those volumes.
by pheatherlite on 3/30/24, 10:48 AM
by kopos on 3/30/24, 9:07 AM
Also Java jar files give you the same benefit.
by trickpa1 on 4/10/24, 10:51 PM
by bun_terminator on 3/30/24, 9:34 AM
by ckdarby on 3/30/24, 11:41 AM
I've got nodejs lambda code that is doing 388B/month and only at this point have we even considered changing the language for performance because the cost savings have a net positive ROI.
It took 5 years to get to this point.
by andrewmackrodt on 3/30/24, 10:58 AM
by dugmartin on 3/30/24, 10:13 AM
by vasco on 3/30/24, 9:43 AM
I've had 3 situations where my place of work was under DoS attack, in the 3 cases I managed to identify an email address and reached out asking why they are doing it, and if they want to talk about our backend. In 1 case, the "attack" was a broken script by someone learning how to program, the other two were real attacks and one of them just immediately stopped once they knew we knew who they were, the other actually wanted to chat and we emailed back and forward a bit.
99.99% of the time a DoS is someone who is bored. Talking to them tends to work.
Edit: there's some questions about the situations so I'll expand:
- The first was not a real attack, and they were doing the network calls through their authenticated API key. This was early days of a YC startup so of course there was no rate limiting in place. In this case I exchanged 2 or 3 emails and after they sent me their python script I sent them back a patch and they finished their scraping without bringing us down. Never heard from them again
- The second was at a different company, we were getting targeted to distribute email spam, because at the time we'd allow people to invite their colleagues as members of their account, and some people associated with casinos based out of Macau automated a way to spam their casinos by putting the URL in the name of the account, which went out in the email notification. I contacted one of the admin emails of one of the casinos I found and they stopped and disappeared. In this case we also locked all their accounts and prevented further logins + emailed them to reach out to support if they thought it was a mistake.
- The third one was more difficult, they weren't using any account, so all we had was network. At some point on the second day though they changed how they were sending some of the calls, and by mistake or not leaked their Telegram username. I installed telegram and talked to them, they trolled me a little bit, but stopped very quickly and didn't start it again. This one was very amusing to people in my company because I had told them this approach would work but a few of the big wigs didn't want me to do it (they didnt have any reason other than "obviously won't work to just talk"). I just did it anyway.
To be clear, you shouldn't reach out with some threats or how you're so good that you found them. My approach is of genuine curiosity, and my literal first message to the telegram person was:
"Hello, how is it going? I work at <companyname> and we're seeing a load of requests originating from your user here on telegram. Does this make any sense to you or do you think I might have the wrong person?"
That's it!
by tromp on 3/30/24, 9:38 AM
> we keep things as minimal as possible
Wonder what's in that file that makes it need to be that large...
by KingOfCoders on 3/30/24, 10:01 AM
4TB/200mb = 5000.
by ddorian43 on 3/30/24, 8:50 AM
by AtNightWeCode on 3/30/24, 10:16 AM
by welzel on 3/31/24, 8:40 PM
Anyhow, doing the same with a high traffic application would be a very very different animal, specially when the app has 100k+ active daily users and is doing actual stuff. The advice is not bad, but it sounds so silly. From experience every time a commercial web application was build as a monolith it became very hard or even unmaintainable in a few years, specially when 15+ Teams are constantly contribution. So pick the right hammer for the problem you have, but pretending a simple marketing webpage + payment/subscription is a good example for architecture is just a bit much.
by vintermann on 3/30/24, 9:08 AM
This sounds like a dream, both in the sense that it's wonderful, and that I'm not quite sure I believe it.
by razodactyl on 3/30/24, 9:03 AM
by block_dagger on 3/30/24, 9:17 AM
by d_burfoot on 3/30/24, 1:20 PM
by PaulHoule on 3/30/24, 12:33 PM
by ur-whale on 3/30/24, 9:19 AM
Not entirely sure it's a wise approach given the deeply asymmetric infrastructure costs of DDoS attacks, especially if the attacker has access to a botnet.
[EDIT]:
in other words, there is a non-zero probability that the attacker, piqued by the boasting, might be able at the flick of a switch to increase the intensity of the attack by a factor 1M.
by dsign on 3/30/24, 11:55 AM
“Systemctl” instead of “systemd” ? Hm, do I detect reticence to publicly admit the undeniable, vast superiority of systemd by confusingly using the name of the utility?
by neya on 3/30/24, 3:22 PM
"We do nothing..because we can."
This speaks volumes about your attitude towards security as a business. If I was your enterprise client I wouldn't really be happy reading this.
by hntddt1 on 3/30/24, 9:44 AM
by CanaryLayout on 3/30/24, 9:34 AM
by wigster on 3/30/24, 11:18 AM
by _ache_ on 3/30/24, 10:16 AM
by memothon on 3/30/24, 3:34 PM
by samyar on 3/30/24, 9:11 AM
What is it and how can one learn about it.
by dewey on 3/30/24, 11:51 AM
by tluyben2 on 3/30/24, 8:51 AM
by vdddv on 3/30/24, 9:03 AM
by pknerd on 3/30/24, 9:46 AM
by b0x68 on 3/30/24, 1:40 PM
by sylware on 3/30/24, 11:32 AM
DDoS attacks do benefit some specific corps, for instance cloudflare.
What's very important is to build DDoS resistant infrastructure without them, to rid of the incentive to shadow-hire hackers to DDoS and force some infrastructures to move there and pay them.
There is too much suspicion in the digital world nowdays. Like current crypto is not mainly for shaddy ops and mafia? Really?
by kbar13 on 3/30/24, 9:58 AM
by aoeusnth1 on 3/30/24, 5:17 PM
Wake me up when you have hundreds of millions of QPS of DOS load.