Inspiration
AI is known to make mistakes—from fake medical advice to invented citations—all of which can harm real people. Companies can't catch every error, but millions of users see them every day. We empower everyday users to become AI safety testers, democratizing the process of identifying and correcting AI hallucinations to make AI systems safer for everyone.
What it does
Hallucination Tracker is a crowdsourced platform that identifies and documents AI accuracy issues across major models like ChatGPT, Claude, and Gemini.
Core Features:
- Hallucination Reporting: Submit detailed reports with correct information and verified sources
- Model Analytics: View accuracy trends and statistics across AI providers
- Vendor Response System: Official feedback and acknowledgment from AI companies
- Critical Bounty Program: LLM vendors post crypto bounties for discovering critical hallucinations in high-impact domains (medical, military, safety-critical systems)
How we built it
Users discover AI mistakes and submit detailed reports with corrections and credible sources. The community reviews submissions while vendors validate claims. For critical hallucinations that match active vendor bounties—such as dangerous medical misinformation, military/security issues, or errors affecting large populations—users receive crypto payments directly to their wallet once the claim is approved.
Challenges we ran into
Dual-Layer Validation: We built a system combining community peer review with vendor validation to prevent spam while maintaining efficient processing.
Quality Control: Balancing incentives required creating tiered bounties for critical issues while keeping the platform accessible for all users to contribute to AI safety.
Accuracy Standards: We implemented linked source requirements, reputation systems for trusted contributors, and standardized categories to ensure fair cross-model comparisons.
Accomplishments that we're proud of
- Built a working platform with community submissions, real-time analytics, and the concept of crypto bounty integration for critical findings
- Created a community-driven design that scales without heavy moderation
- Kept the experience simple and accessible for non-technical users
- Developed data visualizations that help vendors quickly identify and address systematic issues
What we learned
Community moderation works. Public responses from AI companies create accountability that benefits everyone.
Incentive structures matter. Crypto bounties for critical issues attract serious safety researchers, while reputation systems encourage consistent, quality participation across all submissions.
Transparency drives improvement. Even small recognition significantly boosts participation in making AI safer.
What's next for Hallucination Tracker
Our vision is to become the industry standard for AI accuracy benchmarking through:
- Governance tokens for community-led platform decisions
- Academic partnerships for research collaboration and validation
- Insurance products based on hallucination risk profiles
- Automated testing suites that vendors integrate directly into their development pipelines
Our ultimate goal: Highlight the real-world costs of AI hallucinations and demonstrate why safety must be the default, not the exception. By democratizing hallucination detection while incentivizing critical safety discoveries, we're building a future where AI systems are accountable to the communities they serve.
Log in or sign up for Devpost to join the conversation.