PlasticFree | Devpost

Inspiration

The climate crisis has crossed its tipping point and plastic pollution is one of the biggest contributors. Many of the plastics that we interact with contain toxic chemicals. The average American eats, drinks, and breathes in more than 70,000 microplastic particles yearly. And only 9% of plastics are successfully recycled. So we thought it was our chance to make the world better. We wanted to not just work with the consequences of our actions but to prevent them in the first place. So our app will help people to use less plastic and make their lives PlasticFree.

What it does

It helps the user identify if there is any plastic in the item that they are interested in and if there is suggests an alternative without plastic: You use a gesture to snap a pic of the plastic product We’re using CLIP to determine what an object is, and if there’s plastic in it. If there is, you’re shown several plastic-free alternatives for the product. Enter your email to have them sent to you. If there isn’t any plastic in your picture, you’re given a persistent Turtle that other Plastic Free users can find. The idea is to share places that are plastic-free with others.

How we built it

The Spectacles sends captured photos to our custom server, which runs a computer vision model for detecting plastic objects, and returns recommendations for products that are plastic-free alternatives to them.

Challenges we ran into

In non-XR game development projects, the only elements of consideration are objects(GameObjects/SceneObjects) that are manually put in the scene by developers. However, when working with AR glasses, one must also take note of the real-world environment surrounding the player. Certain features such as RayCasting - which is almost trivially easy to make use of in non-XR projects - require extra work in order to interact properly with the physical space around the user.

Additionally, game engines like Unity have a vast history and active community, allowing for common coding implementation conventions to arise. With a cutting-edge and recent engine like Lens Studio, no such conventions exist: a "best" approach does not necessarily exist yet for every feature implementation.

With regards to our "plastic detection," we experimented with three different computer vision models before settling on the best one, after considering tradeoffs between speed and accuracy

Accomplishments that we're proud of

While Lens Studio has a fully functioning asset for website interaction, it is still an experimental feature with minor bugs and optimization issues. Still, we managed to properly account for these in our development.

What we learned

We learned how to code in TypeScript, the differences between Lens Studio and game engines not designed for XR, how to realistically narrow down scope for a hackathon project, and more!

What's next for PlasticFree

If there is a store near you, the device will provide navigation, along with location details. A “My Plastic Replacements” section that tracks your non-plastic purchases. And of course, sharing.