from Hacker News

Ask HN: Can we collaborate on a IP Address or Regex blacklist?

by usernamebias on 4/7/21, 2:33 AM with 43 comments

• by thephyber on 4/7/21, 3:12 AM

  Project Honeypot has been doing this (IP address reputation scoring) for something like 16 years (disclaimer: I once worked for a sibling company and developed an early Apache module for it).

  What you propose is very similar to what happens with email IP reputation. If you look at all of the effort that goes into verifying as few false positives and false negatives as possible, you should probably consider why that effort is put in. Example: what happens if a malicious user who works on behalf of a rival company to yours creates a Pull Request to your list with your customer’s IP addresses? Could you realistically identify the issue and the malicious user before it hurt your corporate reputation?

  I don’t think your idea is bad, but you have to realize that the concept of an IP address as a proxy for an actor/reputation is not as valuable in recent years as it used to be. With IPv6 and cheap botnet access, your list will fill up with junk when the attacker spends very little effort to add new GET/POST rules and new clients.

  I would recommend you spend some time considering how much you care about this particular cat and mouse game when CDNs and WAFs have already made products which cater to this need.

• by ggm on 4/7/21, 2:40 AM

  What are you going to do, when the addresses are SAAS and your blacklist is now impinging your own use of FAANG and DC cloud hosted services?

  What are you going to do when the addresses belong to the US mil and are being promiscuously misused by lots of ISPs?

  What are you going to do about politically motivated and other non benign influences on the blacklist like wanting to boycott China?

  (I work in a regional internet registry so I should declare my interest i guess)

• by trinovantes on 4/7/21, 2:50 AM

  There was a GitHub repo posted sometime ago that contained a list of ASNs (basically ids of datacenters) where most attackers/spammers come from. Simply blocking those ASNs helped the author stop almost all of their bad requests. I wish I bookmarked it but maybe somebody else can chime in...

• by mjbrownie on 4/7/21, 3:07 AM

  I usually set up nginx to "default ignore" and only respond to specific paths which I can name... works for api only domains at any rate. just use an explicit subfolder like /api/ ... cuts down on the noise.

      location / {
              return 444;
              access_log off;
      }
  
      location /a/ {
         ...
      }

• by bradknowles on 4/7/21, 5:30 AM

  I’ve been in various security adjacent jobs over the past 30+ years, and many times I’ve been working with security experts to try to secure some service or another that my team supports.

  IP address reputation based blocking was a concept that we saw back in the mid-90s when I was fighting spam as the Senior Internet Mail Administrator at AOL. It worked okay, for a while. It quickly became a game of cat-and-mouse, where some spammers wouldn’t care that we blocked them, but plenty others found various ways around the blocks we were implementing.

  More than 25 years later, and the problem really hasn’t changed that much. You still get lots of people who think they can just block stuff by IP addresses and that will solve all the problems.

  The best modern WAFs that I’ve seen in the past five to ten years are probabilistic at best. Set the rejection threshold too low, and you start getting way too many false positive hits. Set the rejection threshold too low, and too many attacks just skip right past the WAF. They are a tool you need to have in your toolbox and you need to make use of them, but they are weak protection, at best. They’re table stakes, which set a low bar for your attackers to clear.

  Mod_Security is an excellent example of a free and relatively low effort WAF that you can implement, but there are alternatives. Fastly is a well known commercial CDN/WAF provider, but Cloudflare has their WAF service, AWS has a built-in service, etc....

  If you really want to be secure against attackers, you need to make sure that every layer of your code is secure. Do all the standard network scanning and fuzzing tools. Have someone play red team against your system and see if they can penetrate your defenses. Use the source code analysis tools that are appropriate for your language — Fortify might not always be the right answer. Use the dynamic application security tools like the stuff from Contrast Security, where they can scan your object code as it is running in real time and monitor for all known vulnerabilities and attack patterns, and then update that list of things to scan for in real time.

  Make sure you actually fix the weaknesses that are turned up by these tools. It doesn’t help you to identify a bunch of problems and then just leave them unfixed.

  The OWASP stuff is a start, but they’re just skimming the surface. This is a true deepness with no bottom.

• by zamalek on 4/7/21, 4:51 AM

  The problem with IP addresses is that many ISPs rotate IPs between customers, especially IPv4 addresses. If an ISP starts running out of addresses, they may have to start using NAT. "Privacy" VPNs also do the same (by design).

  A banned IP may be rotated to a legitimate user under many scenarios. Only ban/blackhole IPs for a limited duration.

• by jrockway on 4/7/21, 3:24 AM

  Good idea. It would be nice to feed these bad requests into the per-IP rate limiter and just count them as being more than one request. Fetch index.html, that counts towards the rate limit as 1 request. Fetch DROP DATABASE users.html, that counts as 1000 requests. If your quota is 120 requests per minute (the arbitrary value I picked for my personal website), you're gone for 8 minutes.

  How useful it is to rate limit on known attacks, I don't actually know. I feel like you really only need one request to exploit a 0day, so it probably provides no protection.

• by AndroidKitKat on 4/7/21, 3:48 AM

  Is this not sorta like https://www.dronebl.org ?

• by indymike on 4/7/21, 3:19 AM

  You might want to head over to Shadow Server and take a look at their networking reporting. https://www.shadowserver.org/what-we-do/network-reporting/

• by kjrose on 4/7/21, 4:11 AM

  Is this not essentially what cloudflare does with the IPs it is tracking? If your ip is showing abuse or problems you get knocked to the "boat or bike" system to at least slow you down a notch?

• by TameAntelope on 4/7/21, 3:53 AM

  Maybe it's just me but I tend not to worry much about these scans.

• by rognjen on 4/7/21, 8:59 AM

  Another issue is that these are all just scripts that scan things randomly and it takes a minute to set it up on a new server.

  That means whenever a server is compromised it'd have this type of stuff installed and it'd start running it immediately.

  That means two things: the list would indefinitely block servers that have been compromised but then cleaned up, and you'd never get a list comprehensive enough because servers are constantly being compromised.

• by farazzz123 on 4/7/21, 5:08 AM

  http://googledrawing.com/ http://howtohub.co/ http://4dresult.org/

• by farazzz123 on 4/7/21, 5:07 AM

  http://4dresult.org/

• by nickphx on 4/7/21, 5:14 AM

  Why? It's a fruitless endeavor. Keep internet facing things patched, limit exposure of internet facing things. If the idea of seeing those events in a log bothers you, restrict your logging to the paths you care about. :)

• by jpmoral on 4/7/21, 5:11 AM

  Dumb question: how does the checking for .env work?

• by ev1 on 4/7/21, 3:10 AM

  abuseipdb.com

• by sgrinich on 4/7/21, 3:55 AM

  What about a denylist?