Inspiration
There are so many foods out there that we all want to explore. Unfortunately, access to a list of ingredients might not always exist for a variety of foods; not all restaurants even identify ingredients such as peanuts and treenuts. Allertgen was built to allow everyone to stay safe while exploring the world of cuisine available!
What it does
Allertgen allows users to upload a photo of either a menu or a food item, along with a set of allergens to watch for. If a menu is uploaded, the web app reads all detected text, translates the text as needed, and saves the text representing food items. If a food item picture is uploaded, then the web app will label the food item provided using a machine learning model. Information on this/these food item(s) is then obtained from common recipes, and the list of ingredients obtained is compared to the identified allergens. It then alerts the users if they have allergies to watch out for!
How we built it
After having planned out the design flow, we sketched out our designs using Figma to build the UI using HTML/CSS. The majority of the back-end of our program was built using Python, which facilitated the use of Google Cloud for image classification (including building and training a machine learning model using Google Cloud's AutoML), and text detection/translation with Google Cloud's Vision API and Google Cloud's Translation API. We also used Google Cloud's App Engine with Flask in order to integrate the Python back-end with the HTML/CSS front-end.
Challenges we ran into
None of us had used Python or the Google Cloud platform extensively, nor had any of us used Flask at all before. This made major components of the project a challenge, as we had to learn while building the web app. Further, since we had so many different parts to be combined, we ran into multiple server/path issues when trying to integrate them all into one app.
Accomplishments that we're proud of
We're very proud of this project - we managed to design and create a mostly-functional app in 24 hours, and along the way, we all picked up significant new portions of knowledge. It was also a project that we felt could significantly enhance the quality of life for people we knew that had specific allergies.
What we learned
Working extensively in the terminal/command line as well as deploying Google Cloud applications! It was also a big learning curve to successfully integrate front-end components with back-end components for a full-stack web app.
What's next for Allertgen
More comprehensive image classification in order for machine learning model to be more accurate. As well, see if there's a possible way of making the translation results better while not significantly compromising on run-time. Overall, a great learning experience!
Log in or sign up for Devpost to join the conversation.