sty.ly

Syncing your style with the forecast.

Comment

Inspiration

The idea stemmed from the frustration of not knowing what to wear to account for the weather forecast, along with any unpredictable weather changes.

What it does

"sty.ly" generates outfit recommendations based on real-time weather data. The app suggests outfits suitable for the current weather conditions, taking into account factors such as temperature, humidity, and precipitation. Some additional features include being able to save your favourite outfits, a spotify player, along with a sign up page allowing users to give their gender and age range so that the ouftits we generate are specifically tailored towards you!

How we built it

We used llama3.1 to generate a custom dataset of 30k entries for outfit recommendations based on weather combined over rain, snow and other factors. We then trained llama3.1 8B on said dataset using Unsloth @ 375 steps and quantizing it to Q4_K_M. We use the trained model to get an outfit recommendation from weather data which we get from an API. We run the model locally via Ollama. We then send the outfit recommendation to a locally hosted venv instance of Stable Diffusion via A1111SDK (model: https://civitai.com/models/251478/fashion-street-photographyandfilm-v1) and then encode the generated image to Base64, sending it to our database. After the database receives it, we display it on our frontend. Flask was also used for python-based backend components.

Challenges we ran into

One of the major challenges we faced was ensuring the accuracy and relevance of the outfit recommendations. Training the llama3.1 model on our custom dataset required extensive fine-tuning to balance the different weather conditions and fashion styles. Quantizing the model to Q4_K_M while maintaining performance was another challenge, as we had to find the sweet spot between model efficiency and output quality.

Integrating Stable Diffusion into our pipeline was also tricky, particularly when optimizing for speed and image quality. Making the whole system work smoothly on local machines required overcoming various compatibility and performance issues.

Accomplishments that we're proud of

We're proud to have successfully trained llama3.1 on a custom dataset and quantized it for efficient local use. The model's ability to provide fashion-forward outfit recommendations based on real-time weather data is a significant achievement. Additionally, the integration of Stable Diffusion to generate visual representations of these outfits is something we’re particularly proud of, as it adds a visually appealing and practical element to the user experience.

The seamless interaction between our backend, model, and frontend, all hosted locally, demonstrates our team's ability to combine various technologies into a cohesive and functional product.

What we learned

Model Training & Fine-Tuning: We learned the importance of dataset quality and diversity when training models like llama3.1. Creating a custom dataset of 30k entries required careful consideration of various weather conditions and how they influence outfit choices. Fine-tuning the model on this dataset taught us how to balance specificity and generality in predictions.

Integration of Multiple Technologies: This project reinforced the importance of integration between different technologies. Connecting a locally trained model with real-time APIs, Stable Diffusion, and a Flask-based and Express.js backend required a strong grasp of each component and how they interact.

What's next for sty.ly

Enhanced Model Training: We plan to further refine our model by incorporating additional data points, such as user preferences, fashion trends, and geographic location. This will allow "sty.ly" to offer even more personalized and context-aware outfit recommendations.

User Feedback Loop: Implementing a feedback system where users can rate and refine the suggestions will help improve the model over time. By learning from user input, we can make the recommendations more accurate and aligned with individual tastes.

Expanded Weather Conditions: While our current model accounts for common weather conditions, we aim to expand its capabilities to include more specific scenarios, like extreme weather or transitional seasons, to provide even more tailored advice.

Share this project:

Updates