LyricAI

• 

  LyricAI

Inspiration

You find a song you like, and maybe you want to sing along or do karaoke, but as it happens, the song is not available as karaoke anywhere. Many good songs aren't famous enough for people to manually write down their lyrics and make a lyric video. So, we made a program to do it!

What it does

The user just needs the mp3 file for their song, and that's it! LyricAI does its magic and spits out a lyrical version of the song with the background music to sing to! It smartly separates out the vocals and the instruments to let the user be the singer while the video provides the music.

How we built it

We built LyricAI in parts.

Step 1

We start by converting the mp3 file into two, one with just the instruments and another with just the vocals. To do this we use a TensorFlow-based library (spleeter).

Step 2

We pass on the vocals to Assembly AI using their API and call the transcript.

Step 3

Now that we have the transcript, we split it into individual words and run our video and image function on each of the words.

Step 4

We now have hundreds of mini mp4 files, each one for each word. So, we combine all of these into one mp4 file.

Step 5

Finally, we add the instrument audio that we generated some time ago to the video and render the final video. Voila!

Challenges we ran into

We ran into several challenges related to API calls, video processing and file handling. We had to fix a lot of errors related to image compatibility with the video. Since we were dealing with hundreds of files being made every few seconds, we had to do a lot of clever file management.

Accomplishments that we're proud of

We are really proud of the fact that we were able to make a full-fledged HD video just from the audio file of the song.

What we learned

We learned a lot about audio and video file management with their error handling. Also, we learned how various kinds of codes are made compatible with each other.

What's next for LyricAI

There is a lot of room to grow for LyricAI. We would like to add more complex features to our program, like music genre-specific outputs and maybe dance videos?

Built With

• assemblyai
• bing-image-downloader
• gizeh
• moviepy
• pil
• pydub
• spleeter