Skip to content

Bluesky reply bot checker - #348

Merged
simonw merged 2 commits into
mainfrom
claude/bluesky-ai-bot-detector-tthj2m
Sep 27, 2026
Merged

simonw merged 2 commits into
mainfrom
claude/bluesky-ai-bot-detector-tthj2m

Conversation

@simonw

@simonw simonw commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Build a tool to help evaluate if a user on Bluesky is likely an AI reply bot (inspired by previous bluesky tools in this repo)

It should start by showing their follow and follower counts, and how many posts they have made, and if possible how many of those posts are relies as opposed to top level posts (if that cannot be done efficiently it should show that just for the past 100 posts)

It should evaluate their replies. I want it to list their most recent 50 replies to other people and for each one note:

  • how long after the original post was the reply sent?
  • does the reply include a question mark?
  • time of day - I want to spot if accounts post 24 hours a day or if they have more human looking patterns

Before we start building, any suggestions for more signals?

Yes scan 1000 - and yes to account age, quiet window, and length consistency

Don't bother with those LLM style markers yet

Yea to considering follower count of parents

Look for the posting mix, and how many likes their posts have had overall and avg number of likes

Do UTC

Build this tool so it starts with headline stats and a one paragraph of why the tool thinks it may be a bot (or not), then it shows some examples of recent bot-like replies, then it has much more detail, and it finishes with all of the recent posts and replies with an indicator on each one for if it has bot signals

The tool must reveal its working, it needs to be very clear about the signals it uses so it doesn't accuse people without justification

Claude Opus 5.5: https://claude.ai/code/session_01Fjfk7d7jqcP2Sd74UTooFP

New tool that scans up to 1,000 of an account's most recent posts and
evaluates it for signs of being an automated (often AI) reply bot:
reply delays, replies posted faster than anyone could type them, daily
breaks, reply length consistency, posting mix, who it replies to and
more. Every signal shows its measured value, rule, rationale and
innocent explanations, plus per-reply indicators on every post.

Includes Playwright tests with a mocked API for a bot-like and a
human-like account.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fjfk7d7jqcP2Sd74UTooFP
@claude

claude Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Claude encountered an error —— View job


I'll analyze this and get back to you.

Prompted by an account that replies hours after the original post, in
bursts, and so scored "few signals" despite being clearly automated.

- New "Replies posted by software" signal: timestamps not in the app's
  millisecond format (e.g. Python microseconds) or missing language tags
- "Faster than typing" now also counts time since the account's own
  previous post, catching bursts of replies to different posts; replies
  with repeated (pasted) text are skipped
- New "Never replies back" signal (strong) and 2 points for 95%+
  drive-by replies; uniform reply length worth 3 points below 0.25
- Scoring is now additive points split into direct evidence of
  automation vs behaviour; "strong pattern" requires automation
  evidence, so behaviour alone can't produce it
- One-way signals like reply speed show as "not triggered" instead of
  counting as evidence of a human

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fjfk7d7jqcP2Sd74UTooFP
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Deploying tools with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3e44940
Status: ✅  Deploy successful!
Preview URL: https://802afb05.tools-b1q.pages.dev
Branch Preview URL: https://claude-bluesky-ai-bot-detect.tools-b1q.pages.dev

View logs

@claude

claude Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Claude encountered an error —— View job


I'll analyze this and get back to you.

@simonw
simonw merged commit 631f951 into main Sep 27, 2026
1 of 4 checks passed
@arodericks

Copy link
Copy Markdown

any findings so far? Happy to review any bots you find. Feel free to ping me at aaron.bsky.team or I can also share an email to send your findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants