Smart Bites | Devpost

Logo
Menu Scanning
Preferences
Results

Inspiration

The inspiration for creating the web app "Smart Bites" stemmed from a very relatable and common challenge that many people face in their daily lives – maintaining a healthy diet while dining out. As we looked around, we noticed that there was a growing awareness of the importance of nutrition and fitness among individuals who were actively trying to make healthier choices or pursue bodybuilding goals.

However, one major obstacle stood in their way: the ambiguity of restaurant menus. Dining out often presented a dilemma, especially for those who were conscious about their calorie intake, macronutrient ratios, or specific dietary preferences. The menus at restaurants were extensive, filled with tempting options, but it was often a struggle to decipher which dishes aligned with their health and fitness goals.

This frustration led us to brainstorm a solution that could make the dining experience easier, more enjoyable, and conducive to personal health and fitness objectives. We envisioned a user-friendly web app that could bridge the gap between restaurant menus and individual dietary needs.

"Smart Bites" was born out of this vision, and it became our mission to empower people to make smarter choices when dining out. The app leverages the power of technology, particularly image recognition and personalized recommendations, to provide a seamless experience. Users could simply take a photo of the menu, and "Smart Bites" would do the rest.

The core idea behind the app was to harness the vast amount of nutritional data and culinary knowledge available to us and use it to serve individuals better. Our goal was not only to help people find dishes that aligned with their dietary preferences but also to educate them about the nutritional content of their choices. We wanted to make eating out an informed, guilt-free experience, where individuals could pursue their fitness goals without compromising on flavor.

Ultimately, "Smart Bites" was created to be a trusted companion for anyone who wanted to enjoy dining out while staying on track with their healthy lifestyle or bodybuilding journey. By addressing this common pain point, we aimed to empower individuals to make more confident and informed choices, ultimately leading to better health and well-being.

What it does

"Smart Bites" is a revolutionary web app designed to make dining out a healthier and more personalized experience. Here's an overview of what the app does:

Menu Photo Recognition: The core feature of "Smart Bites" is its ability to analyze and interpret restaurant menus. Users can simply take a photo of a menu using their smartphone's camera, and the app's powerful image recognition technology goes to work. It scans the menu to identify dishes, ingredients, and nutritional information.

Personalized Recommendations: Based on the menu analysis and the user's dietary preferences, health goals, and restrictions, "Smart Bites" generates personalized meal recommendations. These recommendations take into account factors such as calorie intake, macronutrient ratios, and specific diet plans (e.g., keto, vegetarian, vegan, low-carb).

Nutritional Insights: In addition to recommendations, the app provides detailed nutritional information for each dish on the menu. Users can easily see the calories, protein, carbohydrates, fats, and other essential nutrients for any item they're interested in. This empowers them to make informed choices that align with their nutritional needs.

Allergen Alerts: "Smart Bites" is sensitive to food allergies and intolerances. It alerts users to potential allergens present in dishes, ensuring their safety and peace of mind while dining out.

How we built it

React
Material UI
Python
Flask
OpenAI API
Google Cloud Vision API
OCR

Challenges we ran into

Image Recognition and OCR: Implementing accurate and robust image recognition and Optical Character Recognition (OCR) technology to extract text and identify menu items from photos was challenging. Ensuring that it works well across various lighting conditions, menu designs, and languages required significant testing.

Menu Data Integration: Collecting and integrating menu data from different restaurants in a standardized format can be complex. Menus come in various layouts and structures, and developing algorithms to parse and categorize menu items correctly was challenging.

Accomplishments that we're proud of

Working with Large Language Models (LLMs) and Optical Character Recognition (OCR) for the first time in the development of "Smart Bites" was a remarkable achievement. It positioned our project at the forefront of technological innovation in the realm of dining and health-conscious eating.

Our team's ability to master these cutting-edge technologies showcased our adaptability and commitment to staying ahead in the tech space. LLMs allowed us to provide personalized recommendations based on user preferences and health goals, enhancing the user experience significantly. OCR simplified menu input, enabling users to snap a photo instead of manually entering data.

This innovation not only improved user experience and data accuracy but also gave "Smart Bites" a competitive edge. We stood out as a leader in promoting healthier dining choices and catering to specific dietary needs. Positive user feedback reinforced the value of this accomplishment.

Moreover, our success with LLMs and OCR laid a solid foundation for future projects, demonstrating our capability to tackle complex tech challenges. In summary, integrating LLMs and OCR into "Smart Bites" was an achievement that transformed the dining experience, showcased our technical prowess, and positioned us as pioneers in health-conscious dining technology.

What we learned

Tech Integration Expertise: Working with Large Language Models (LLMs) and Optical Character Recognition (OCR) for the first time, we gained a deep understanding of how to integrate cutting-edge technologies into real-world applications. This experience taught us the importance of staying up-to-date with emerging tech trends.

Adaptability: We learned the significance of adaptability in the tech industry. Overcoming the learning curve associated with LLMs and OCR required flexibility, a willingness to learn, and the ability to pivot when necessary. This adaptability is a crucial skill in a rapidly evolving tech landscape.

Data Quality: Dealing with menu data and nutritional information highlighted the importance of data accuracy and consistency. We learned the challenges of maintaining high-quality data and the impact it has on user trust and satisfaction.

What's next for Smart Bites

Restaurant Reviews: Incorporate a review and rating system for restaurants, enabling users to share their dining experiences and read others' feedback. This adds a social element and builds trust within the community.

Integration with Food Delivery Services: Partner with food delivery platforms to enable users to order healthier versions of their favourite dishes directly through the app.