PhishFilter

Scam who? Not you.

Comment

Inspiration 💫

Over 70% of college students actively seek jobs and internships, but nearly 1 in 3 fall for phishing emails. In the U.S., over 68 million people receive scam calls each month, with billions lost to fraud annually. Our team experienced this firsthand - one teammate shared their resume and phone number in a fake job offer, while another’s mother lost money to a scam call.

These moments weren't just frustrating - they were wake-up calls. We realized that while AI is evolving rapidly, so are social engineering scams, and many users (especially college students and older adults) are left unprotected. These moments inspired us to build PhishFilter - an AI-powered app that detects threats in real time and educates users to stay safe.

What it does 🧠

PhishFilter is a React Native mobile application designed to protect users from two common attack vectors:

  • 1)Real-time Scam Call Detection
  • Using Vapi, we analyze active phone calls for suspicious patterns, tone, and keywords to determine if the caller is likely a scammer.
  • After the call, the app gives users a breakdown of the risk level, what red flags were detected, and what to watch for next time.

Email Phishing Detection (Job Scam Focused)

  • Users upload or screenshot suspicious emails (especially job recruitment emails).
  • Using Claude's Opus 4 OCR and language understanding, the app extracts and analyzes the content for phishing indicators like urgency, mismatched domains, and fake sender identities.
  • The app flags the message and provides an explanation, educating the user.

How we built it 🛠️

  • Frontend: Built with React Native for cross-platform mobile support.
  • Voice Call Analysis: Integrated Vapi to stream real-time call audio to the app, detect scams mid-call, and alert the user live.
  • Phishing Detection: Used Claude Opus 4 with OCR to analyze email screenshots. Fine-tuned prompts to catch job scam patterns targeting students and job seekers.
  • Backend:
  • FastAPI to interface with Claude for phishing detection.

Challenges we ran into 🥲

  • Claude OCR Confusion: It took around 5 hours to get OCR to work! We were using Claude without realizing we needed to route requests specifically to Claude Opus 4. We did the good ol' debug print statements to figure out issues. Once we figured that out, everything sped up.
  • Frontend UI Design: Creating a clean, intuitive UI in React Native that could show real-time scam detection — without overwhelming the user — took several design iterations. We wanted it to feel informative, not technical.
  • Connecting Frontend and Backend: Getting Claude’s output (especially from OCR) to talk smoothly with the frontend was tricky. Parsing the text results and rendering them clearly in-app required a lot of back-and-forth debugging.
  • Vapi Integration: Reading Claude’s response programmatically and passing it into Vapi’s voice synthesis was a challenge. We had to map the scam content and safety explanation into natural-sounding speech while syncing it with the app UI.
  • WebSocket Stability: Our server WebSocket kept timing out during longer OCR processing. We had to rework the socket flow to support longer wait times and reconnect if needed — not fun at 2am!

Accomplishments that we're proud of

  • Reduced scam detection latency by 80%: Our final pipeline detects and responds to scam signals significantly faster than our initial baseline.
  • OCR time improved by over 5 seconds: By switching to Claude Opus 4 and optimizing our routing, we made OCR processing much faster and more efficient.
  • Successfully integrated Vapi for live scam call detection: We connected real-time voice output with scam detection results, creating an immersive and interactive user experience.

What we learned 📚

  • How to work with the Vapi API, including voice output, call forwarding, and call logs.
  • How to use Claude for OCR, and how different Claude models affect performance and accuracy.
  • How to buffer Claude’s output in real-time and pass it smoothly into the frontend without blocking the UI.
  • A lot about building a more thoughtful, user-centered frontend — especially how to balance visual clarity with technical feedback.

What's next for PhishFilter 🌎

  • Launch the app on the App Store and Google Play.
  • Add a learning center with interactive simulations of scam scenarios.
  • Expand phishing detection to support multilingual content and more scam formats (e.g., text messages, social DMs).
  • Partner with universities and career centers to offer PhishFilter as a tool for student safety and job scam awareness.

Built With

Share this project:

Updates