Stylify

Inspiration

Takes a lot of time for us to get ready, especially for special occasions. We also take a lot of our time browsing through different online retailer stores to find clothes that we actually like. Or sometimes a lot of people are inspired by celebrity's outfit, but the actual outfit is expensive, so we can offer similar or identical outfit that costs significantly less.

What it does

Users can input an image/text description of a piece of clothing, and we generate possible outfit combinations around the given piece, by displaying image/answering with text. Input an image of a desired outfit, we then generate similar outfits with different price range based on the budget.

How we built it

We used the OpenAI API to get access to GPT model. Then we prompt engineered the model to turn it into an expert in the fashion industry takes certain types of inputs and generate the desired output. We tuned the model a little bit so that it returns the top 3 answers, representing 3 possible outfit combinations that user can mix and match with the given piece of clothing. We then leveraged the user input and the output to prompt the text-to-image generation.

Challenges we ran into

We originally wanted to make a website that if the user puts in an image of a piece of clothing, we would be able to generate possible outfits as well as the price and the retailers from each piece. I originally worked on Steamship multimodal agent because Steamship is beginner friendly, and I have some experience working on it. However, we later found out that Steamship is closed source, and we cannot deploy the agent to the website of our choice. Therefore, we switched to OpenAI. We initially started with the GPT-3.5 model, and we found out that this particular model tends to go off topic quite often, and we tried many ways to overcome it by refining the prompt, tuning the model... but we still had the same problem. Therefore, we decided to switch to GPT-4, a much better improvement of GPT-3.5, and it worked well. But we ran into another problem with text-to-image model, Dall-E. This model also tends to generate inaccurate images regardless of the prompt. We got it to generate the desired number of outfits, but we could not quite get the style and color right.

Accomplishments that we're proud of

We are proud that we have actually implemented it, since we thought about switching ideas and giving up a lot throughout the hackathon. We believe that there is plenty of room for improvement on this project. We are also proud of our members, because most of us had little to no experience at all in AI, ML or UI/UX, but we completed the project anyway.

What we learned

We learned how to work as a team, expanded our domain of knowledge to a lot of new areas like generative AI, web design. We also learned about time management by having smaller deadlines for different tasks to keep us on track.

What's next for Stylify

If our idea is highly evaluated, we will definitely try to continue working on it to further improve the website. We are thinking of Shark Tank pitch presentation and NittanyAI Challenge as our next steps toward bringing this project into life. Depending on the results, we will then try to determine whether to follow through with this project.

Built With

• dalle
• openai
• python