anubis

What's the difference with https://github.com/TecharoHQ/anubis? Considering sibuna it's anubis backwards.

That doesn't match my experience. For example, I recently had Meta scrape robots.txt disallowed paths. On some page they claimed they respect it but they don't. And yes, all the IPs they used to connect to me (they used a different /64 network for each connection) were owned by Meta, it was not someone pretending to be them. Some other reports of this: https://github.com/TecharoHQ/anubis/issues/1565 I ended up just fully closing connections with no response from these assholes on any URL.

Nice analogy, wonderful art. Something I've found interesting is that LLMs tend to act like this on the small scale, too. Claude Code in particular appears to be absolutely relentless in trying to come up with workarounds when faced with a block page.[1] As an experiment, a developer for Anubis recently added a feature to give the LLMs an "easier" problem to solve instead of straight-up blocking them, and it seemed quite effective.[2] (It was later removed[3] due to what I assume to be privacy concerns.) I think, beyond HuggingFace, there ought to be a push to align LLMs so they understand that they should not proceed when access is blocked and/or their presence is unwanted. [1]: https://blog.xkeeper.net/the-cutting-room-floor/self-hosting... [2]: https://github.com/TecharoHQ/anubis/pull/1895/commits/de2312... [3]: https://github.com/Tech

Doubtful. You’re not going to mine a bitcoin in 30 seconds. It’s been pretty popular lately to utilize a proof of work problem to deter AI scrapers. This is likely what you’re seeing. Anubis is the one I’ve seen most often, but not the only one. https://github.com/techaroHQ/anubis

Yes, that seems to work for Altcha: https://altcha.org/docs/integration/proof-of-work-captcha/ Edit: this is already in discussion for Anubis: https://github.com/TecharoHQ/anubis/issues/94

Forgot about that. I intially thought you were referring to CloudFlare Captcha's, which Google and Firefox are partnering with: https://www.techtimes.com/articles/318891/20260623/cloudflar... It's remarkable how much more time one has to wait just to access the same level of information (Cookie Policy, EULAs, etc). Even with "lighter" weight HTTP Firewall utilities such as Anubis (https://github.com/techaroHQ/anubis), the average user has to pay for other's (DDoSers) misdeeds. Now no one can visit a site without a firewall unless it's static and rate-limited by IP address to avoid crashing a small home server. Some impressive LMDBs might be able to serve up to 100,000 requests per second on a lightweight PC, but then again they could still be knocked offline from a super resourceful organization. I like not putting all my eggs in one bask

I'm almost ready to ship the wasm feature in the next version of Anubis after the one that's about to come out. The big blocker is that testing against dozens of googles chrome to ensure functionality on abandoned smart TV oses takes a nontrivial amount of time. As an example of the level of debugging and the like required: https://github.com/TecharoHQ/anubis/pull/1684/changes/67621f... My office gets very warm when chromesweep runs. This is something that is complicated enough that even though LLM tools can help, it's not a magic bullet. It's just complicated in general.

The prompt used for Opus 4.8 was: write a implementation of the anubis proof of work in native c code, optimized for speed above all else. use every trick available to make the proof of work as efficient and fast as possible, including modern processor tricks on the x86 platform. your code should avoid using external libraries where possible, include tests, and be readable and concise. a reference for what needs to be met is in this repository. https://github.com/TecharoHQ/anubis Then let’s develop this more. turn this solver into a local HTTP server that can be given work in the request, and it returns solved work. make an end to end tester that sends test work to the solver and waits for a valid response. add support for solving with a GPU using cuda. Then it was done more or less, it happily made a local server that supports

> that stupid anime girl who is on way too many open source web sites I assume you're referring to https://github.com/TecharoHQ/anubis. As I understand, this is a tool for blocking bot traffic and is sometimes used as DDOS protection. In my experience it's less obtrusive than Cloudflare, and it's open source and competes with Cloudflare's near monopoly on DDOS protection. I don't think we should be discouraging people from protecting their sites from unwanted traffic.

Adding the link to GitHub here if anyone is curious: https://github.com/techaroHQ/anubis

For what it's worth, Anubis supports LibreJS: https://github.com/TecharoHQ/anubis/blob/main/web/build.sh#L...

The "benefit of someone else" in this scenario is the site operator not having their website down or their hosting bills unsustainable because of misbehaving (which are 99% of current) web scrapers from AI companies. The README itselfs admit that this is an nuclear option. https://github.com/TecharoHQ/anubis

Tell me the flags I'm missing then: https://github.com/TecharoHQ/anubis/blob/Xe/wasm4/utils/wasm...

Anubis: https://github.com/TecharoHQ/anubis blog post (pretty sure I've seen it on HN before) on the topic: https://lock.cmpxchg8b.com/anubis.html

I think you'd definitely find some interest, e.g. anyone that intentionally avoids "the cloud" will want something local. Honestly I assume there are some of these already, monitoring apache/nginx/etc logs. Anubis is arguably similar and has been exploding lately, for example, though I'm not sure if it auto-updates its rules at all: https://github.com/TecharoHQ/anubis As to if it'd get enough interest: yea no idea at all. I wish you luck tho! Clearly there's a need for this kind of thing.