./www/anubis, Proof-of-work check to stop AI bots

[ Image CVSweb ] [ Image Homepage ] [ Image RSS ] [ Image Required by ] [ Image Add to tracker ]


Branch: CURRENT, Version: 1.24.0nb4, Package name: anubis-1.24.0nb4, Maintainer: bsiegert

Anubis weighs the soul of your connection using a sha256 proof-of-work
challenge in order to protect upstream resources from scraper bots.

Installing and using this will likely result in your website not being
indexed by some search engines. This is considered a feature of Anubis,
not a bug.

This is a bit of a nuclear response, but AI scraper bots scraping so
aggressively have forced my hand. I hate that I have to do this, but
this is what we get for the modern Internet because bots don't conform
to standards like robots.txt, even when they claim to.

In most cases, you should not need this and can probably get by using
Cloudflare to protect a given origin. However, for circumstances where
you can't or won't use Cloudflare, Anubis is there for you.


Master sites:

Filesize: 16144.47 KB

Version history: (Expand)


CVS history: (Expand)


   2026-04-08 07:45:14 by Benny Siegert | Files touched by this commit (203) | Package updated
Log message:
Revbump all Go packages after security update
   2026-03-06 22:08:08 by Benny Siegert | Files touched by this commit (203) | Package updated
Log message:
Revbump all Go packages after go126 update
   2026-02-24 09:45:43 by Benny Siegert | Files touched by this commit (203)
Log message:
Revbump all Go packages after default version bump
   2026-01-26 20:49:21 by Benny Siegert | Files touched by this commit (21)
Log message:
Move non-pattern Go flags to GOFLAGS.

GO_BUILD_PATTERN is only for patterns, as announced on tech-pkg@.
   2026-01-15 20:55:00 by Benny Siegert | Files touched by this commit (202) | Package updated
Log message:
Revbump all Go packages after go125 update
   2026-01-10 18:38:34 by Benny Siegert | Files touched by this commit (3) | Package updated
Log message:
anubis: update to 1.24.0

Anubis is back and better than ever! Lots of minor fixes with some big
ones interspersed.

- Fix panic when validating challenges after privacy-mode browsers strip
  headers and the follow-up request matches an ALLOW threshold.
- Expose WEIGHT rule matches as Prometheus metrics.
- Allow more OCI registry clients based on feedback.
- Expose services directory in the embedded (data) filesystem.
- Add Ukrainian locale (#1044).
- Allow Renovate as an OCI registry client.
- Properly handle 4in6 addresses so that IP matching works with those
  addresses.
- Add support to simple Valkey/Redis cluster mode
- Open Graph passthrough now reuses the configured target Host/SNI/TLS
  settings, so metadata fetches succeed when the upstream certificate
  differs from the public domain. (1283)
- Stabilize the CVE-2025-24369 regression test by always submitting an
  invalid proof instead of relying on random POW failures.
- Refine the check that ensures the presence of the Accept header to
  avoid breaking docker clients.
- Removed rules intended to reward actual browsers due to abuse in the wild.
   2025-12-02 20:25:26 by Benny Siegert | Files touched by this commit (202) | Package updated
Log message:
Revbump all Go packages after go125 update
   2025-11-28 18:38:31 by Ryo ONODERA | Files touched by this commit (3)
Log message:
www/anubis: Update to 1.23.1

Changelog:
## v1.23.1: Lyse Hext - Echo 1

- Fix `SERVE_ROBOTS_TXT` setting after the double slash fix broke it.

### Potentially breaking changes

#### Remove default Tencent Cloud block rule

v1.23.0 added a default rule to block Tencent Cloud. After an email from their \ 
abuse team where they promised to take action to clean up their reputation, I \ 
have removed the default block rule. If this network causes you problems, please \ 
contact [abuse@tencent.com](mailto:abuse@tencent.com) and supply the following \ 
information:

- Time of abusive requests.
- IP address, User-Agent header, or other unique identifiers that can help the \ 
abuse team educate the customer about their misbehaving infrastructure.
- Does the abusive IP address request robots.txt? If not, be sure to include \ 
that information.
- A brief description of the impact to your system such as high system load, \ 
pages not rendering, or database system crashes. This helps the provider \ 
establish the fact that their customer is causing you measurable harm.
- Context as to what your service is, what it does, and why they should care.

Mention that you are using Anubis or BotStopper to protect your services. If \ 
they do not respond to you, please [contact me](https://xeiaso.net/contact) as \ 
soon as possible.

#### Docker / OCI registry clients

Anubis v1.23.0 accidentally blocked Docker / OCI registry clients. In order to \ 
explicitly allow them, add an import for `(data)/clients/docker-client.yaml`:

```yaml
bots:
  - import: (data)/meta/default-config.yaml
  - import: (data)/clients/docker-client.yaml
```

This is technically a regression as these clients used to work in Anubis \ 
v1.22.0, however it is allowable to make this opt-in as most websites do not \ 
expect to be serving Docker / OCI registry client traffic.

## v1.23.0: Lyse Hext

- Add default tencent cloud DENY rule.
- Added `(data)/meta/default-config.yaml` for importing the entire default \ 
configuration at once.
- Add `-custom-real-ip-header` flag to get the original request IP from a \ 
different header than `x-real-ip`.
- Add `contentLength` variable to bot expressions.
- Add `COOKIE_SAME_SITE_MODE` to force anubis cookies SameSite value, and \ 
downgrade automatically from `None` to `Lax` if cookie is insecure.
- Fix lock convoy problem in decaymap \ 
([#1103](https://github.com/TecharoHQ/anubis/issues/1103)).
- Fix lock convoy problem in bbolt by implementing the actor pattern \ 
([#1103](https://github.com/TecharoHQ/anubis/issues/1103)).
- Remove bbolt actorify implementation due to causing production issues.
- Document missing environment variables in installation guide: `SLOG_LEVEL`, \ 
`COOKIE_PREFIX`, `FORCED_LANGUAGE`, and `TARGET_DISABLE_KEEPALIVE` \ 
([#1086](https://github.com/TecharoHQ/anubis/pull/1086)).
- Add validation warning when persistent storage is used without setting signing \ 
keys.
- Fixed `robots2policy` to properly group consecutive user agents into `any:` \ 
instead of only processing the last one \ 
([#925](https://github.com/TecharoHQ/anubis/pull/925)).
- Make the `fast` algorithm prefer purejs when running in an insecure context.
- Add the [`s3api` storage backend](./admin/policies.mdx#s3api) to allow Anubis \ 
to use S3 API compatible object storage as its storage backend.
- Fix a "stutter" in the cookie name prefix so the auth cookie is \ 
named `techaro.lol-anubis-auth` instead of `techaro.lol-anubis-auth-auth`.
- Make `cmd/containerbuild` support commas for separating elements of the \ 
`--docker-tags` argument as well as newlines.
- Add the `DIFFICULTY_IN_JWT` option, which allows one to add the `difficulty` \ 
field in the JWT claims which indicates the difficulty of the token \ 
([#1063](https://github.com/TecharoHQ/anubis/pull/1063)).
- Ported the client-side JS to TypeScript to avoid egregious errors in the future.
- Fixes concurrency problems with very old browsers \ 
([#1082](https://github.com/TecharoHQ/anubis/issues/1082)).
- Randomly use the Refresh header instead of the meta refresh tag in the \ 
metarefresh challenge.
- Update OpenRC service to truncate the runtime directory before starting Anubis.
- Make the git client profile more strictly match how the git client behaves.
- Make the default configuration reward users using normal browsers.
- Allow multiple consecutive slashes in a row in application paths \ 
([#754](https://github.com/TecharoHQ/anubis/issues/754)).
- Add option to set `targetSNI` to special keyword 'auto' to indicate that it \ 
should be automatically set to the request Host name \ 
([424](https://github.com/TecharoHQ/anubis/issues/424)).
- The Preact challenge has been removed from the default configuration. It will \ 
be deprecated in the future.
- An open redirect when in subrequest mode has been fixed.

### Potentially breaking changes

#### Multiple checks at once has and-like semantics instead of or-like semantics

Anubis lets you stack multiple checks at once with blocks like this:

```yaml
name: allow-prometheus
action: ALLOW
user_agent_regex: ^prometheus-probe$
remote_addresses:
  - 192.168.2.0/24
```

Previously, this only returned ALLOW if _any one_ of the conditions matched. \ 
This behaviour has changed to only return ALLOW if _all_ of the conditions \ 
match. I expect this to have some issues with user configs, however this fix is \ 
grave enough that it's worth the risk of breaking configs. If this bites you, \ 
please let me know so we can make an escape hatch.

### Better error messages

In order to make it easier for legitimate clients to debug issues with their \ 
browser configuration and Anubis, Anubis will emit internal error detail in base \ 
64 so that administrators can chase down issues. Future versions of this may \ 
also include a variant that encrypts the error detail messages.

### Bug Fixes

Sometimes the enhanced temporal assurance in \ 
[#1038](https://github.com/TecharoHQ/anubis/pull/1038) and \ 
[#1068](https://github.com/TecharoHQ/anubis/pull/1068) could backfire because \ 
Chromium and its ilk randomize the amount of time they wait in order to avoid a \ 
timing side channel attack. This has been fixed by both increasing the amount of \ 
time a client has to wait for the metarefresh and preact challenges as well as \ 
making the server side logic more permissive.