restrict paywalled useragents; use dumps

[mekarpeles]

This pull request updates the Nginx configuration to further restrict access from specific user agents, particularly targeting paywalled commercial services and bots. The main changes involve identifying and blocking requests from the VibXchange user agent.

User agent restrictions:

• Added a rule to flag requests from the VibXchange user agent as suspicious, grouping it with ancient browsers and other restricted clients in the map $http_user_agent $is_sus_user_agent block.
• Updated the conditional block in the server configuration to explicitly block requests from VibXchange, alongside existing bot user agents like bytespider and meta-externalagent, returning a 403 Forbidden status.

[Copilot]

Pull request overview

This PR adds restrictions for the VibXchange user agent, identifying it as a paywalled commercial service that should use data dumps instead of scraping the website. The changes implement blocking at both the general traffic level and the API endpoint level.

Changes:

• Added VibXchange to the suspicious user agent map with explanatory comments
• Updated the API location block to explicitly block VibXchange requests with 403 Forbidden

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mekarpeles]

discussed on ABC