Style Sync | Devpost

Style Sync
Making a donation
Discovering new clothing
Donations Made

Inspiration

We were inspired by the custom algorithm of tinder and how it utilizes first impressions and preferences to narrow down larger catalogs to improve “shopping” experience. We also wanted to drive traffic to second-hand charity thrift shops and improve the system currently used by second-hand stores to price their items.

What it does

The app has two main parts: the explore page, where users engage in a tinder- like experience of seeing clothing options and swiping up or down. The emotions they experience when viewing a product are evaluated using Hume’s Expression Measurement API and used with their swipe decision and additional factors to calculate what products they’re more likely to want. Users can see the video footage used in the measurements and the emotions recorded as a show of transparency. The other page is the donation page, where users can add a photo to items they are donating to thrift stores. The donated pieces are occasionally cycled in the explore page to gauge interest using the same expression measurement, which helps drive customer traffic to local charity stores. Users will be able to view a donation count of the number of items they have donated, as well as a metric demonstrating the conservation that occurred as a result of repurposing used clothes. The conservation metric encourages users to repurpose clothes that are still good quality but may no longer need, reducing waste and providing cheaper alternatives for clothing.

How we built it

We employed TypeScript for the frontend and Flask for the backend. Additionally, we leveraged Gemini's multimodal API to extract characteristics of clothing from images, used Word2Vec to create a vector representation of these characteristics generated by GPT, and utilized Hume's API to analyze video streaming via a WebSocket connection, generating data related to emotions. We calculated our pricing for the donated items using a linear regression model using emotion data, prices, and other clothing data generated from advertisements placed in between other products on the Explore page.

Challenges we ran into

Our linear regression model correlating prices to emotions did not exhibit a positive correlation as expected. Additionally, we faced issues with the swiping feature in the frontend, experienced delays in calculations and retrieving images, leading to potential lag on the page. We had to manage our use of API credits carefully and conduct thorough testing. Ensuring the correct data and format were passed between the frontend and backend was also a significant concern. Finally, we grappled with the complexity of weighing different characteristics of clothes and determining the optimal algorithms for preferences but ultimately found a vector-based approach worked best.

Accomplishments that we're proud of

Innovative Algorithm: We developed a custom algorithm for personalizing clothing recommendations and narrowed down larger catalogues.

Improved Pricing: We improved the pricing system used by thrift and second-hand stores, providing a more efficient and accurate method for thrift stores to price their items.

Transparent Emotion Evaluation: Users can see the video footage used in emotion measurements, fostering transparency in how their preferences are analyzed and utilized.

Integration with Thrift Stores: Thrift stores can sell their items on the platform, increasing traffic and visibility for their most popular pieces, and driving more revenue for charities like the American Cancer Society or Salvation Army.

Linear Regression Model: Developing a pricing model using linear regression, incorporating emotion data, prices, and other clothing characteristics to determine optimal pricing for donated items.

What we learned

We learned how to use the Hume API and advanced emotion recognition and how to integrate it into an application to provide customers with value. We also learned how to use word2vec to form vector representations and make classifications of data, which allowed us to compare clothing based on string classifications like style or pattern.

What's next for Style Sync

We have a few additional features we want to add to our application. We want to improve the user experience and further encourage people to donate clothes by adding leaderboards and more "game-like" elements to make the process more exciting.

We want to increase the size of our clothing database, by scraping many more retailers and adding their catalogs to ours to provide more options to users and allow for more fine tuning of preferences. More products would also allow us to fix our regression model and come up with more accurate pricing for the donations tab.

We want to improve the integration onto platforms to reach a larger audience and we want to add an interfaces specifically for thrift stores to be able to post their items.

Additionally, we want to add log in, verification, and a user profile where users can view more precise information about the amount of money they've spent, previous purchases, trends over time, and other information.