-
-
App Icon
-
Project Mascot for ProtectD
On ProtectD
Inspiration
Our team took longer than expected to come up with an idea for this project. With the stress of school, extracurriculars, and sports, it was difficult to find the time just to sit and think. But one day, while walking across his school, one of our members, Greyson, came up with an idea. Inspired by the objects around him, he realized they all had one thing in common: barcodes. In many stores, especially craft stores, the products don’t have any indication of price, use, or their environmental impact. Thus began our journey to find a solution for this problem, with ProtectD.
What It Does
ProtectD is an app that will scan the barcodes on products, since they are used for almost all products across the country. Then, after scanning the barcode, ProtectD will generate a summary of this object, which will give more information to the consumer about what they are buying.
How We Built It
ProtectD took a few parts to get working. On one hand, there was the frontend; this was implemented entirely in Swift / SwiftUI. We used navigationLinks to get between our four pages, and twostraws' CodeScanner (https://github.com/twostraws/CodeScanner) for a simple and expressive way of scanning barcodes directly through the SwiftUI. (This avoided wrapping UIKit elements, which saved some time). The frontend also has a few functions (credit to GPT-4o for writing markdownStrip and fetchPlainText) dedicated to receiving and processing information received from the backend. Speaking of the backend: the backend is a Rust program: it performant, robust and memory-safe. It communicates over HTTP (over TCP/IP), and passes information internally to an instance of Ollama, which we used as our LLM API. This runs on-device, and was wrapped in Rust code. (rusty-llama is that wrapper). As you can see, while ProtectD seemed quite simple superficially, it was most decidedly not particularly simple, with the software engineering behind it spanning multiple devices and languages. This allowed very little processing is done on-device (mobile perspective), which makes the app more accessible: one does not require a powerful frontend to use our app. Xcode predicts a processor usage of just 5% (one core) for the entire app, running on an Apple iPhone 13 mini.
Challenges We Faced
It was a big challenge just to start ProtectD, since there was a lot of decrease in motivation after we took so long to find an idea. But we were able to push through, and we were able to finish this app in the end. Another challenge was with coding, because only Greyson knew how to code (at all) beforehand, so we all had to figure out how to contribute and put in the same effort he did.
Our Accomplishments
Because it was such a big challenge, all of us are proud of finishing this app. It took a lot of effort and sleep loss to get through it, but we made it. Personally speaking, there were many more components to the app than I (Sophia) thought there would be, and I’m proud that we could figure them all out.
Lessons We Learned
We all learned a lot about time management, particularly because the lengthy process of setting an idea set us back on time. Moreover, individually, Greyson learned that Rust -- while a great language -- can be incredibly frustrating when the deadline is less than twenty-four hours away and the program isn't done yet. Michelle and Sophia learned a lot about the struggles of choosing (P3) colors based on their hexadecimal values, and Sophia learned about the issues that come with not saving the writing portion of this submission, as well as coordinating a bunch of high-school students to work on this project, which was difficult to say the least. Through this project, Michelle realized the difficulty of producing an image using AI (DallE x GPT-4o) due to its tendency to emphasize each word of the prompt rather than the big picture. With the troublesome development of the UI, Michelle and Sophia learned how difficult and intricate it can be to create a functioning system. Zephyr played a key role in brainstorming a way to introduce our app and learned valuable lessons about project completion and accountability. Despite early doubts, he recognized the importance of pushing through challenges even when initial skills were lacking. Additionally, we all learned that AI, integrated or not, is complex, and it is tedious at best to perform the functions with seeding as well as setting generation parameters being quite the endeavor.
The Future
ProtectD has some limitations; for example, as it doesn't have access to a UPC database, it isn't able to decode the codes the scanner actually scans into products. For the demo, no matter what the code is, we default to sending the large language model the product called "Apple AirPods Gen3". We’re hoping to go further with ProtectD, possibly even including a feature that can suggest other, more environmentally friendly products, but we don’t think that’s going to happen for at least another few years. We need to gain more experience and information to improve this app, and until then, we don’t think there’s going to be much editing to this app. Instead, we will grow our knowledge of coding and programming, so we can make ProtectD an app for all.
Log in or sign up for Devpost to join the conversation.