GastroPay | Devpost

How to use

Inspiration

Our inspiration stems from the new canteen at the tum campus in Garching, which just two months after completion is already reaching its maximum capacity. This means long queues, which can really take away valuable time between lectures. One of the main bottlenecks of the new canteen lies in the cash registers, which are self-service and not the most reliable. Often these registers are the point where everything slows down and one can end up waiting in excess of 10 minutes, just to pay. Our solution aims to adress this issue.

What it does

Our solution to this very common problem is simple, taking self-service to the next level. Nowadays everybody is walking around with the smartphone already with them, so why not use it to handle the payment process. Our system uses computer vision to identify the products and meals and calculate the appropriate price. While taking the photo the student places his payment card, including a barcode on his tray. Our software recognizes and reads the bar code and automatically charges the associated account. Each purchase is associated with a table, enabling efficient auditing for staff. The interface is accessed by the end-consumer scanning QR-Codes on their table and provides an easy admin system for staff. In addition, our program uses know ingredients lists and allergy information associated with the user account in oder to warn the user of consuming any products that may be detrimental to their health.

How we built it

Our software is built around the Azure Custom Vision API, which we use to recognize the products and determine the position of the student card. We use our own training data to train our models. We also built an web interface and backed it with a Azure SQL database.

Challenges we ran into

Getting reliable results identifying objects using our trained machine learning system. As well as recognizing and scanning the bar code, particularly when it is oriented differently or partially in the shadow.

Accomplishments that we're proud of

Pretty reliable (for a proof of concept) recognition of some objects and, limited by time, other objects to a lesser degree. With our method we would be able to expand on this quite easily.

What we learned

We practiced using industry standard technology and expanding on those systems to develop our own product fulfilling a certain use case. We also took the opportunity to grow as a team and use each team members personal strengths.

What's next for GastroPay

Following the successful development of a proof of concept during the hackatum event, we are looking forward to develop the idea further in the future.