• 

• 

  The problem

• 

  Goals, vision

• 

  Components and Features

• 

  The BFDataset

• 

  APIs Used

• 

  AMPL

• 

  Meet The Team

SimpleShopping

Grocery shopping made convenient. SimpleShopping is an innovative platform that individualizes grocery shopping with computer vision.

What can SimpleShopping do?

SimpleShopping uses big data (10+ GB) to create neural networks and discover different recipes and nutritional info.

Inspiration

Grocery shopping has always been a "one-size fits all" experience. However, this multi-billion dollar industry has been falling behind. With new innovative technologies being employed at Amazon's Amazon Fresh and other big-tech companies step into the space. We decided to create a platform that would revolve around the end user - in this case: you!

A Novel AI Implementation

Overview

SimpleShopping uses the novel semi-supervised training method: Advanced Meta Pseudo Labels. Advanced Meta Pseudo Labels uses AI to create new efficient AI models.

Compared to previous methods, SimpleShopping is 10% more accurate than state of the art attempts.

Dataset

We used the VegFru dataset. Containing over 80000 images of exotic vegetables and fruits.

Model Architecture

A modified EffcientNet Convolutional Neural Network (CNN) model was used to make inferences.

EfficientNet is a SOTA model developed by Google in 2020. For more information, please refer to EfficientNet.

Clean GitHub

Our GitHub was cleanly created and with accurate PEP8 formatting. The code is ready for use with its easy to read API and well documented components. We pushed to our main branch frequently with well written commit descriptions to ensure we would always have a working version of our code. We ran into a few merge errors which were only able to be solved due to the well written descriptions of what we had changed.

Github Copilot: Assisting the Open Source Community

We utilized Github Copilot: a platform that enables efficient coding. Copilot helps lint and write code with a complex JPT model developed by OpenAI. It is currently in development beta, and we thank Github and OpenAI for the opportunity to test their new product. Copilot allowed us to write code faster, and implement libraries that we have never used before.

APIs Used

To create simple shopping, we utilized three core APIs: Tensorflow, FatSecret, and Streamlit. We utilized Tensorflow to create our neural network models and to process data. We employed mixed precision training, which increased performance while also reducing computational power, and we used JIT compilation for our models.

FatSecret is a powerful API used by many large - and small - companies around the world, notably including Samsung, Amazon, and Fitbit. We used the FatSecret API to access and and find nutrition info, recipes, and other important information for our application.

Lastly, we used Streamlit to deploy our novel algorithms into a user accessible manner.

Issues with API and Development

Throughout the hackathon, numerous issues arose from incompatibility between the APIs used (Tensorflow, OpenCV, ...) however, by implementing a common medium of array transfers - Numpy - we were able to resolve the issues. Furthermore, we faced numerous issues with the frontend API: Streamlit, but we were able to resolves those issues using a pseudo-HTML based formatting.

Installation

git clone https://github.com/frankchang1000/SimpleShopping.git
cd simpleshopping
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt

Use 7zip to unzip the tensorflow model file. It is located in data/simpleshopping-model.7z

To access the nutrition data, you MUST have a developer Fatsecret API key. For more information about FatSecret API, please refer to here.

Then place your OAuth1 token into a fat_secret.key file in the data folder.

{
    "consumer_key": "abunchofrandomchars",
    "consumer_secret": "abunchofrandomchars"
}

Running the GUI Application

To run the application, run the following command in the project directory:

streamlit run main.py

To finetune training or to run training:

python main_tf2.py 

Refernces

[1] Tan, M., & Le, Q. (2019, May). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR.

[2] Hou, S., Feng, Y., & Wang, Z. (2017). Vegfru: A domain-specific dataset for fine-grained visual categorization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 541-549).

[3] Pham, H., Dai, Z., Xie, Q., & Le, Q. V. (2021). Meta pseudo labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11557-11568).

License

Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License (CC BY-NC-ND 4.0)

Under the following terms:

Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

NonCommercial — You may not use the material for commercial purposes.

NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.

No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.

SimpleShopping Team 2022: Frank Chang (FrankChang1000), Thomas Chia (IdeaKing), Alton Lin (alton.d.lin), and Jason Yoo (youajason)

Built With

• html
• python
• tensorflow