Inspiration
It's really annoying to encounter products that are out of stock, have a mismatching label or price or are placed in the wrong aisle. Such problems decrease customer satisfaction significantly. Also supermarkets are in constant competition of offering the lowest prices, so they're interested in budget savings wherever possible.
What it does
We try to tackle both problems by using mostly pre-existent means in supermarkets: cameras. Cameras are usually required to protect from theft or other crime, but they also create a constant stream of visual data which we can use for our purposes. What we do require are specialized labels, such as price-tags extended by a machine readable QR code.
How we built it
We have a lot of cameras in the supermarket, usually watching every shelf. Since we know the camera position and angle, we can calculate the coordinates of the area that our camera is perceiving.
Employing modern image recognition techniques, we classify everything the camera sees. We then match this classification with the information provided from the associated pricetags (QR-Codes). If we detect a mismatch, we inform employees via an app, where we show the exact position and type of the problem encountered. So the problem can be fixed as soon as possible, improving customer satisfaction while not requiring special (expensive) equipment.
Challenges we ran into
Image resolution of current security-camera feeds will likely not suffice for reading QR-Codes. This may be a problem with current supermarket equipment, but we think that this will improve drastically in the near future. We also would've loved to use the provided barcodes, but we can't save price information in them. So we had to introduce our own QR Codes.
Accomplishments that we're proud of
We developed a working proof of concept, including machine learning, a backend server and a fully fledged android application, while maintaining a healthy sleep rhythm and diet.
What we learned
Reading QR-Codes can be harder than classifying images - weren't QR-Codes invented to be easily readable?!
What's next for Kauflab
We'd love to use multiple camera feeds at once to increase resolution and positioning.
Log in or sign up for Devpost to join the conversation.