-
-
A view of the inventory with automatically estimated times until expiry
-
3 ways to input items: manually, produce picture, or receipt
-
An example of a well classified pizza
-
A receipt read and accurately deciphered, ready to be imported
-
A recipe for a Fusion Fruit Salad, automatically generated based off the available produce
Inspiration
As young adults, we're navigating the new waves of independence and university life, juggling numerous responsibilities and a busy schedule. Amidst the hustle, we often struggle to keep track of everything, including our groceries. It's all too common for food to get pushed to the back of the fridge, only to be rediscovered when it's too late and has gone bad. That’s how we came up with preservia - a personal grocery smart assistant designed to help you save money, reduce food waste, and enjoy fresher meals.
What it does
Catalogue food conveniently: preservia.tech allows grocery shoppers to keep track of their purchased food, ensuring less goes to waste. Users take photos of their receipts and the app will identify the food items bought, estimate reasonable expiry timeframes, and catalogue them within a user-friendly virtual inventory. Users also have the option of directly photographing their grocery items and the app will add them to the database as well, or even manually enter items.
Inventory: The user interface offers intuitive control, allowing users to delete items from the inventory at their will once items are used. Users can also request the application to reevaluate expiry dates if they suspect any mistakes in the AI predictions.
Recipes: Additionally, users can select food items in their grocery inventory and prompt the application to suggest a recipe based on selected ingredients.
How we built it
Preservia.tech is built around leveraging Large Language Models (Cohere) as flexible databases and answer engines, here to give nuanced answers about expiration, for even the most specific food! This allows us to enter any possible food item, and the AI systems will do their best to understand and classify them. The predictive power of Preservia.tech will only expand as LLMs grow.
OpenAI’s GPT-4 was also used as a flexible system to accurately decipher cryptic and generally unstandardized receipts, a task probably impossible without such models. GPT-4 is also the engine generating recipes.
We employed Google’s MediaPipe for food item classification, and converted images to text with API Ninjas to read the receipts.
Our app is primarily built on a Python backend for computation, with Flask to handle the web app, and mySQL as a database to track items. The web pages are written in HTML with some CSS and JavaScript.
We can connect it to a smartphone through a local network to take pictures more easily.
Challenges we ran into
Working with cutting edge APIs and AI was a brand-new challenge for the entire team, so we had to navigate different types of models and documentations, overcoming integration hell to eventually arrive at a successful project. We also found prompt engineering hard, especially trying to get the most accurate results possible.
It was all of our first times working with Flask, so there was a learning curve there. Deploying our app to online services like Replit or Azure also posed a major challenge.
Accomplishments that we're proud of
Our team is especially proud of successfully integrating such a broad range of AI features we had never worked with before. From image classification to Optical Character Recognition, and leveraging LLMs in novel ways as flexible databases and parsers.
For our team members, this marked the beginning of our deep dive into the realm of APIs and AI, making the experience all the more exciting. We were impressed with our quick progress in bringing the project to life. Finally, We’re proud that our vision was realized in the app and our brand, preservia.tech, a clever play on the words — preserve [food] via technology.
What we learned
Our team learned how to use different kinds of APIs, the functionings and applications of LLMs and image models, as well as flask and mySQL principles to build future projects with easy web interfaces.
Our team was new to working with APIs and image-to-text models like MediaPipe. To integrate the image-to-text, text classification, image classification, and text interpretation features into our project, we strengthened our fundamental coding skills and learned how to weave APIs in to create a viable product.
What's next for preservia.tech
In the future, we hope to enhance our image recognition software to recognize multiple food items within a single image, and with better accuracy, surpassing the current capability of one at a time. Additionally, we’re looking into other AI LLM models that can exhibit high precision in estimating food expiry dates. We may even be able to train machine-learning models ourselves to elevate the accuracy of our backend expiry date prediction system. It’ll also be interesting to build a mobile app to make uploading content even easier, as well as accelerating the LLMs we are using.
Log in or sign up for Devpost to join the conversation.