Polyglotify | Devpost

English to Japanese
English to Spanish

Inspiration

When I was thinking about my journey trying to learn spanish, I was very frustrated about my motivation. I loved listening to language in other language but felt like I was never at a level to understand them. Brandyn and I both use Spotify every day. So we decided to take Spotify and modify it such that it can help with the language learning process by expanding upon their already existing lyric functionality and allowing users to select another language, displaying them side by side.

What it does

Polyglotify takes music and displays lyric and their translation real time side by side. In the minimum viable product (MVP) we had English, Spanish, Italian, Arabic, and Japanese. The interface is similar to what users are used to, both from the perspective of using any music player (our focus was on mimicking Spotify) and Karaoke.

How we built it

Once we had our idea on how to expand upon an already existing closed source project, we identified the key technologies that we'd both be comfortable using which happened to be:

React
Express (if a dedicated backend was required)
Firebase

We first implemented the authentication system and all though it wasn't 100% required, it felt necessary for future development. Once that was sorted, we looked at how we'd translate the lyrics. We explored various technologies but due to time restrictions, we opted to manually translate them for the purpose of our MVP. Once the lyrics were translated, we built the audio player using the HTML audio object and built the karaoke like interfaces, syncing up the lyrics.

From there, it was a matter of polishing up our concept and fixing any small bugs that may have arisen or were identified during deployment to Vercel.

Challenges we ran into

Both of us are across the world from each other, so a lot of the work we had to do asynchronously. Another issue we ran into was getting real-time information about either the lyrics, or the song itself. Although we found a solution to this by utilising either Googles real time language APIs or OpenAI APIs, we didn't have the time to implement or test this. This would be something we explore further down the track.

Accomplishments that we're proud of

We were immensely proud that we were able to get a viable MVP for a song that we both enjoyed. Having the lyrics side by side with a functioning audio player gives off the example of what we were trying to achieve without sacrificing too much functionality on the front end.

What's next for Polyglotify

Besides the top prize for FalconHacks, we hope to expand on the project in several ways:

Real time translation for the most popular languages (hopefully any language)
More music and the ability for users to manage their own personal libraries
Crowdsourcing lyrics and lyrics fixes for our existing translations
Creating a community of language learners and exploring ways for users to engage with each other
The ability to explore international music with varying difficulty levels assigned to them
Statistic tracking for users progress throughout their language learning journey
More user centric settings and QoL updates