Inspiration
More people than ever are shopping online. In the United States, over 40% of all clothing is purchased online. Shoppers love the convenience and wide array of options available online.
However, online consumers often report issues with sizing and fit, as well as products not looking how they had imagined when viewing it online. The increase in shoppers online, alongside this decrease in customer satisfaction, has resulted in rapidly increasing amounts of returned merchandise (30-40% returns, as of 2022).
Every year, roughly 40 billion dollars of apparel is returned. Additionally, the cost to manufacture and also to return the goods can result in an additional 30% increase in greenhouse gas emissions. Shoppers are unhappy, sellers are losing money, and our environment is suffering because of rampant returns and low customer satisfaction.
That’s why we built Magic Mirror, the first interactive, virtual try-on studio enabling anyone to see, from the comfort of their own home, how clothes would look like on them. We aim to increase customer satisfaction, reduce clothing returns, and protect our planet using our innovative platform blending unique voice controls and our custom image-generation pipeline.
What it does
Magic Mirror uses a custom image generation pipeline to create highly realistic and accurate generations of how users would look when wearing certain pieces of clothing. This is powered by our pipeline which uses FastSAM and YOLO for segmentation. Then, we can either use a user provided image, or optionally use OpenAI’s image gen to create unique articles of clothing, which are placed onto users using FitDIT, an open-source model.
This system is accessed via our frontend, built with React and Next.js. The interface is controlled primarily via voice, with our conversational voice agent built on LiveKit. This allows for a completely hands-off and highly interactive experience.
How we built it
Tech Stack:
- React
- Next.js
- TailwindCSS
- Python
- Pytorch
- H100 GPUs
- FastSAM
- Yolo
- LiveKit
- Flask
- FitDIT
Challenges we ran into
- Managing a conversational agent
- Image processing issues and pipeline performance
Accomplishments that we're proud of
- Worked with open-source machine learning models
- Trying Tigawok!
What's next for Magic Mirror
- We want to further enhance its capabilities by continuing to optimize the models that we’re running
- Increase ability for different clothing types and image angles
Log in or sign up for Devpost to join the conversation.