• 0 Posts
  • 64 Comments
Joined 3 years ago
cake
Cake day: June 25th, 2023

help-circle





  • You mean for the referer part? Of course you don’t want it for all urls and there’s some legitimate cases. I have that on specific urls where it’s highly unlikely, not every url. E.g. a direct link to a single comment in lemmy, and whitelisting logged-in users. Plus a limit, like >3 times an hour before a ban. It’s already pretty unusual to bookmark a link to a single comment

    It’s a pretty consistent bot pattern, they will go to some subsubpage with no referer with no prior traffic from that ip, and then no other traffic from that ip after that for a bit (since they cycle though ip’s on each request) but you will get a ton of these requests across all ips they use. It was one of the most common patterns i saw when i followed the logs for a while.

    of course having some honeypot url in a hidden link or something gives more reliable results, if you can add such a link, but if you’re hosting some software that you can’t easily add that to, suspicious patterns like the one above can work really well in my experience. Just don’t enforce it right away, have it with the ‘dummy’ action in f2b for a while and double check.

    And I mostly intended that as an example of seeing suspicious traffic in the logs and tailoring a rule to it. Doesn’t take very long and can be very effective.


  • This is the way. I also have rules for hits to url, without a referer, that should never be hit without a referer, with some threshold to account for a user hitting F5. Plus a whitelist of real users (ones that got a 200 on a login endpoint). Mostly the Huawei and Tencent crawlers have fake user agents and no referer. Another thing crawlers don’t do is caching. A user would never download that same .js file 100s of times in a hour, all their devices’ browsers would have cached it. There’s quite a lot of these kinds of patterns that can be used to block bots. Just takes watching the logs a bit to spot them.

    Then there’s ratelimiting and banning ip’s that hit the ratelimit regularly. Use nginx as a reverse proxy, set rate limits for URLs where it makes sense, with some burst set, ban IPs that got rate-limited more than x times in the past y hours based on the rate limit message in the nginx error.log. Might need some fine tuning/tweaking to get the thresholds right but can catch some very spammy bots. Doesn’t help with those that just crawl from 100s of ips but only use each ip once every hour, though.

    Ban based on the bot user agents, for those that set it. Sure, theoretically robots.txt should be the way to deal with that, for well behaved crawlers, but if it’s your homelab and you just don’t want any crawlers, might as well just block those in the firewall the first time you see them.

    Downloading abuse ip lists nightly and banning those, that’s around 60k abusive ip’s gone. At that point you probably need to use nftables directly though instead of iptables or going through ufw, for the sets, as having 60k rules would be a bad idea.

    there’s lists of all datacenter ip ranges out there, so you could block as well, though that’s a pretty nuclear option, so better make sure traffic you want is whitelisted. E.g. for lemmy, you can get a list of the ips of all other instances nightly, so you don’t accidentally block them. Lemmy traffic is very spammy…

    there’s so much that can be done with f2b and a bit of scripting/writing filters



  • In a perfect world, yes.

    In reality, i knew what i did and why i did it, two years ago, after which i never had to touch it again until now, and it takes me 2 hours of searching/fiddling until i remember that weird thing i did 2 years ago…

    and it’s still totally worth it

    Oh or e.g. random env vars in .profile that I’m sure where needed for nvidia on wayland at some point, no clue if they’re still necessary but i won’t touch them unless something breaks. and half of them were probably not neccessary to begin with, but trying all differen’t combinations is tedious…




  • Actually, the optimism in ministry for the future depressed me and made me not finish it. Even though at the time I was wanting for some optimistic climate fiction.

    Here we have this huge threat to humanity and way too little is done about it. But then all the ‘solutions’ in the book are so unrealistic, like russians using oil equipment in antarctica to help the world… it just made me more depressed about climate change that the solutions he came up with are more fever dreams.

    the first chapter was very well done though and should be required reading



  • I’d be really curios to see some sort of study done on this. I mean, it’s not just americans and most of the west is not insulated from america, either, at least not online. and you don’t know from talking to someone online where they’re from. At the same time, there’s rising fascism and neoliberalism bullshit in europe, too.

    I’d love to know how much of it is people getting antsier in general because they’re in a shit situation and how much it’s ‘infectious’ from talking with people in shit situations elsewhere, spreading bad vibes. Is this also happening in the chinese web? How about other countries that are more politically/economically aligned with the west but culturally less part of the english speaking web?

    There has to be some sociologist out there somewhere studying this, no? But i wouldn’t know where to look. if anyone knows of something along those lines, i’d love to hear it.




  • It’d be more accurately titled Star Trek: Burnham

    I always called it ‘The Burnham Show, starring Michael Burnham’.

    It was crazy to me how they could make every plot line revolve around her in some way, have her always be part of figuring out the solution, everyone else fawning over how great she is and what they’d do without her, just the lengths the writers went to to insert her everywhere. It’s just so on your nose and gets really tiring after like 3 seasons.

    Compared with like DS9 where you could have whole episodes where the main character, Quark, only has like 1-2 lines and they focus more on supporting cast like Cisco or just Bashir and Garrek (sorry, I couldn’t resist :) )


  • I don’t disagree. I meant for users it is incidental. Most users probably wouldn’t buy them with spying as the main purpose(they just also don’t really care that it can spy). making them much more widespread than something where spying was the main use-case, making the problem worse.

    And as someone else mentioned, once you did get it, the temptation for using it for spying is there for a user. Making it worse than e.g. a spy pen imo, as with that you’d need the intent to spy first, and then buy it, but with this, you buy it for whatever reason and then think “oh, I could just spy now” since you already own the device, which I’d argue leads to more overall spying, so to speak. Maybe you see a video online and go “oh, I can just do that, right now, no effort on my part, since I already own this device”.

    And for Meta it’s like tracking cookies on crack



  • I remember reading that hotel TVs are an option. They also have an ad platform, but one intended for the hotel owner to send ads from, not some 3rd party. Not exactly dumb but also not as bad as regular TVs.

    And of course a beamer or PC screen connected to some cheap small form factor PC is always an option, with Kodi or similar on it, i haven’t owned a TV in like 10 years, just using a small linux pc with beamer, and a tv tuner card in the past (nowadays my ISP offers all public channels on IPTV)