Snap.py

Want to save money AND the planet when grocery shopping? Look no further than Snap.py — a receipt scanner/shopping list planner that helps you do just that!

Comment

Inspiration

How many times has your milk gone bad? Now think about how many times other people have had that experience. Simple occurrences like this contribute to an overarching problem of food waste, and we wanted to develop an application to help reduce that, as well as aid in other grocery-associated tasks. Our name, "Snap.py", conveys the snappy speed of our tool to aid future shopping trips as well as plays on it being a homophone to "snap pea" and formatted like a Python file name.

What it does

Snap.py has two main features: receipt scanning and shopping list creation. The receipt scanning feature accesses the webcam to see the receipt you have on hand. It will resize the window to match the dimensions of the receipt and scan that image, sending it to a LLM in order to create a list of purchased groceries, their prices, and predicted expiration dates. Shopping list creation provides a space for users to take note of what they want to buy. It will then access data from local supermarkets to find where the items are least-expensive for the user to plan their trip. In case the item is not already in the data we have, we perform web scraping on a site like Aldi's website to return a reasonable price for that item. We also have a feature to mark favorite items for ease of list creation in the future.

How we built it

We deployed a Flask server that allowed us to connect both our front-end and back-end. Our front-end used native HTML/CSS/JS. Our back-end used some web-scrapping to gather data needed for items prices. We used local storage to store the items that were entered into our list.

Challenges we ran into

We encountered challenges with accurately identifying text on receipts using the webcam. Issues with faded ink, wrinkles, and lighting hindered accuracy. Through extensive testing and refining our computer vision algorithms, we improved detection and processing, ultimately enhancing the user experience in capturing receipt data effectively. We also had difficulties with focusing on receipts using the webcam, particularly in zooming in while ignoring distracting backgrounds. The frontend team's code struggled to seamlessly connect with the backend team's code due to varying data formats and implementation, leading to delays in integration.

Accomplishments that we're proud of

The front-end team takes great pride in their design/UI elements, which combine aesthetic appeal with user-friendly functionality, creating an intuitive experience that enhances engagement and satisfaction. The back-end team is proud of their web scraping, which enable efficient and accurate data extraction, empowering users to gain valuable insights from real-world online sources effortlessly. The software development team is thrilled with their integration of cutting-edge AI and computer vision technologies, which significantly enhance the software's functionality and enable advanced features like real-time receipt recognition and text identification. Finally, we're passionate about the extensive array of features we implemented, which reflect our commitment to versatility and user satisfaction while addressing diverse needs and use cases.

What we learned

The backend software development team gained valuable insights into web scraping techniques, enhancing their skills in data extraction and improving the overall functionality of the application. Over the course of the project, we deepened our knowledge of full stack development, gaining hands-on experience with both frontend and backend technologies that enhanced our ability to build cohesive, end-to-end solutions.

What's next for Snap.py

Our next steps are to enhance the look of the list to create consistency in our website-wide design. We will further implement the expiration dates feature with increased accuracy and wider input data. Finally, we'll add a feature to tell whether the current prices shown are at a good value to give users a better sense of market trends and saving potential.

Built With

Share this project:

Updates