ScanWhich | Devpost

Inspiration

Scanwich was born out of the frustration of managing endless piles of paper receipts. We were inspired to create a tool that could automatically scan, digitize, and extract text from receipts—making expense tracking easier and more efficient for everyday use.

What it does

User’s a able to scan paper receipts and have them processed as individual item, which are appended into their personal spending tracker–giving a clear overview of your spending habits, by categorizing entries item by item.

How we built it

We built a backend server using Flask and Python, and a frontend using NextJS. The frontend handles image capturing, and calls our Python backend. Our backend receives an image and optinally an existing CSV file. The image is processed using OCR and the OCR output is then interpreted by OpenAIs text generation API. Lastly, the interpreted output is appended into a new CSV file or the existing CSV, depending on wether the user had uploaded a CSV file from the frontend.

Challenges we ran into

OCR Accuracy: Getting the OCR to work properly was challenging. We had to experiment with different preprocessing steps and fine-tune parameters, such as thresholding and dilation, to improve text extraction. Optimizing Blur: Determining the right amount of Gaussian blur was critical. Too little blur left too much noise; too much blur removed essential details. We iterated multiple times to find the optimal balance. Variable Receipt Quality: Receipts come in different conditions and lighting, requiring constant adjustments to the preprocessing pipeline. System Integration: Ensuring that all libraries and components worked together seamlessly involved extensive testing and debugging.

Accomplishments that we're proud of

Neither of us have competed in a hackathon. We made it a goal to finish our project during this hackathon, which we accomplished. Furthermore, we had the chance to succeed with implementing OCR and using API calls, despite our team's limited experience with these.

What we learned

Image Processing & OCR: We gained a deep understanding of image preprocessing techniques using OpenCV and learned how to optimize images for text extraction using Tesseract OCR. Python Integration: We enhanced our skills in integrating various Python libraries (PIL, pytesseract, OpenCV) to work together in a cohesive project. Problem Solving: Building Scanwich taught us to approach real-world problems methodically, fine-tuning both the image processing steps and OCR configuration to handle messy, real-world receipts.

What's next for ScanWhich

The application has room for future improvements which we will continue to work on; such as incorporating multiple receipts scans at once, determining when it's best to use an image enhancement over another, and supporting database storage for a more convince use experience. Some of us will use ScanWhich to track our spending habits. It will be in our hearts forever <3