ShopStop

About The Project

Our Inspiration

We've all been there: you're watching a YouTube video—a product review, a music video, a daily vlog—and you see an item you absolutely love. What follows is a frustrating journey of pausing, taking screenshots, and desperately searching Google with vague descriptions like "blue jacket from creator's video." The path from inspiration to purchase is completely broken.

We were inspired by the seamlessness of tools like Capital One Shopping and the addictive nature of modern social media. We asked ourselves: What if we could bridge this gap? What if we could make video content instantly shoppable, and infuse it with the smart-shopping DNA of Capital One to ensure users not only find the product but also get the best deal on it? That's how ShopSpot was born.

How We Built It

ShopSpot is a modern Chrome Extension built using Manifest V3. Our architecture is split into three core components:

1. The Content Script (content.js): This is the heart of the user experience. It's responsible for injecting our "Find Products" button and the sleek, swipeable UI directly into the YouTube player. We used the HTML5 Canvas API to capture a frame from the <video> element in real-time.

2. The Background Service Worker (background.js): This is the brains of the operation. When a frame is captured, it's sent here to be processed. A core design principle was flexibility, so we didn't hardcode a single AI provider. Instead, we built a pluggable system that can send the image to Google Gemini, OpenAI GPT-4 Vision, or Anthropic Claude for analysis. Once the AI identifies products, we query the RapidAPI Real-Time Product Search to find matching items for sale.

3. Preference Learning & Capital One Integration: To make the experience personal, we implemented a simple collaborative filtering algorithm. User swipes (likes/dislikes) are stored locally and securely using the chrome.storage.local API. This trains the model on their style. Most importantly, we simulated the Capital One Shopping experience by applying representative cashback percentages to products, instantly showing users the final, smarter price they'd pay.

Challenges We Ran Into

• Real-time Performance: Capturing and processing video frames without causing the YouTube page to lag was a significant challenge. We had to optimize our Canvas API implementation to be lightweight and efficient.
• AI Vision Accuracy: Getting consistent, accurate product identification from a single, sometimes cluttered or low-resolution, video frame was tough. We learned that prompt engineering is key—we had to specifically ask the AI to identify "shoppable e-commerce products" rather than just describing the image.
• Manifest V3 Learning Curve: Building with Manifest V3's service worker model required a new way of thinking about state management and background tasks compared to traditional web development or older extension architectures.

What We Learned

This project was a deep dive into building a full-stack application entirely within a browser extension. We learned how to orchestrate a complex, multi-stage data pipeline—from visual input to AI analysis to an actionable commercial result. We also learned that a powerful backend is only as good as its UI. The Tinder-style interface was a breakthrough for us, proving that data collection (learning user preferences) can be made to feel fun and intuitive. Finally, integrating the "spirit" of Capital One Shopping taught us the immense value of embedding financial tools directly into a user's existing workflow. The real "aha!" moment for a user isn't just finding the product; it's seeing the instant savings.

Built With

• a-javascript-chrome-extension-(manifest-v3)-using-html5/css3
• canvas-api
• google-gemini-api
• rapidapi