Inspiration
Our initial ideas for this hackathon were either to classify if something is a Labubu or not, create a Pinterest moodboard given a song name, or create something similar to the Drake vs AI incident. We ended up combining these three ideas into one and we had fun creating it.
What it does
Our project inputs a Drake song and takes the lyrics of the song. From there it will assign a Labubu that fits the emotions of the lyrics. We have seven emotions: surprised, neutral, happy, anger, sad, disgust, and fear.
How we built it
We began by exploring the Genius API documentation to understand how to retrieve song data. Once a user submits a song query, we use the Genius API to find the matching track and then use Beautiful Soup to scrape the lyrics directly from the song’s Genius page. These lyrics are then fed into a transformer-based emotion classification model—specifically, the j-hartmann/emotion-english-distilroberta-base DistilRoBERTa model—to analyze the emotional tone of the song. From the predicted emotion scores, we identify the dominant emotion and match it to a specific Labubu character image that visually represents that feeling. The entire flow is powered by a Flask backend, which serves the emotion and corresponding Labubu to a simple HTML frontend.
Challenges we ran into
We had to find a fix for segmentation faults that were occurring based on the combination of our developer tools and libraries (Torch, OS), and we eventually got through the hurdle. Another minor issue was understanding the Genius API documentation and how to implement it. Since this project was mainly focused on one artist, we still developed it so that users can search for other songs by other artists that are not limited to Drake.
Accomplishments that we're proud of
- Hooking up our model to the website
- Getting both to work properly
- Accurately assigning the emotion to a song
What we learned
Through this process, we learned that sentiment analysis can be challenging when working with songs since songs aren't just about its lyrics, but also its melody and the way it's sung.
What's next for Drakebubu
In the future we would like to expand Drakebubu to include other artists and more emotions. As of now, our project is limited to the songs that we have and emotions that can be expressed. Furthermore, we have come to an issue where our project looks only at the lyrics of the songs, which does not fully encompass how the songs are expressed. In order to improve the quality of the output, it is necessary to be able to listen to the tones, music, and melody to be able to come up with a more accurate emotion.
Log in or sign up for Devpost to join the conversation.