Songlation | Devpost

Inspiration

Our group members all had different passions and interests, but the one thing that brought us together was our love for culture and language. We realize that learning a new language can be frustrating and overwhelming, so we wanted to allow the user to pick a song they're familiar with in a different language, and have them learn it by making good memories singing along!

What it does

The user can choose the language and genre of any particular song they like, and can sing along to the lyrics of the original language alongside a line-by-line translation in English. After every two lines, the song will pause and the user has the chance to practice on their own, with the option to replay the lyric in an embedded YouTube video. After they're ready to test their skills, the user can record themselves saying/singing the lyric, hitting space after finishing the recording. The program will analyze the input and compare it to the original lyrics, providing feedback depending on the user's pronunciation. If the user does not perform well enough, the website will prompt the user to retry until they do well enough to move on. The user can continue this process for the entirety of the song, and practice their Spanish pronunciation and vocabulary as they go!

How we built it

We built a full-stack software that uses React and Flask as frameworks for JavaScript and Python for frontend and backend respectively. We also called a YouTube API in order to display the song's music video as an added visual to the karaoke experience. We called a Google Cloud speech-to-text machine learning model to convert the user's spoken words into a comparable string to the original lyrics, which would allow the program to "grade" the user's pronunciation and give them feedback.

Challenges we ran into

While each of us have our own individual strengths in backend and frontend, we struggled integrating them with each other, especially with creating APIs that would send data to and from each side. Now, however, we each feel confident in our understanding of the entirety of the project!

Accomplishments that we're proud of What we learned

The frontend is clean and is very straightforward navigation-wise, which is a great plus when it comes to UX. The backend's integration of non-conventional AI was also incredibly rewarding, as most of us had never had the chance to work one-on-one with an AI model. The integration between two different frameworks was also a very difficult task, but with such a great payoff when everything connected perfectly in the end.

What's next for Songlation

Songlation can easily add so many language-learning features that emphasize different parts of the process of learning a new language. Our current implementation focuses very heavily on pronunciation, but we would love to expand it so that users are quizzed on important vocabulary within the lyrics and can accumulate a list of relevant words that they would like to study more in the future.