AudioBook

Music for Reading. Music for You.

Comment

Inspiration

While ereader software does exist, there isn't yet ereaders that can dynamically play background music that matches the contents of the story. The goal of AudioBook is to enhance users' reading experience by playing music from Spotify that matches what the genre of what they are reading. AudioBook accomplishes this by implementing a multi-classification NLP model that is trained and deployed using Microsoft Azure, Spotify's Developer API, and a stunning UI.

What it does

Using state of the art machine learning tools from Microsoft Azure, AudioBook uses a multi-classification NLP model that labels texts by genres with varying 'confidence scores'. It then finds the score with the highest confidence rating and uses Spotify's API in order to search for 50 songs within that genre. From those 50 songs, it chooses one at random and adds it to the users queue to be played later. In order to have access to this information, AudioBook incorporates an authentication process which requires users to login using their Spotify information.

How we built it

The Multi-Classification (NLP) Model was trained and deployed using Microsoft Azure. Specifically, we used the Cognitive Services and Language Studio. The UI and Frontend components were created using Chakra UI and Nextjs. We used authentication software from NextAuthjs in order to add the authentication software for the Spotify Login. We also used Spotify's API in order to make search calls and queue calls.

Challenges we ran into

On the backend side, it took awhile to understand the Microsoft Azure technology. Additionally, the data set that was used to train the model had a few bugs that needed to be removed manually. We also had trouble connecting the front and backend components together as well as using Spotify's API in order to add songs to the queue. We think that this is because Spotify requires premium membership to access all their developer tools.

Accomplishments that we're proud of

Some things that we are proud of is that we were able to train and depoly a machine learning model. As neither of us had a lot of machine learning experience that felt very rewarding. The UI for our web application is very good, and we believe that it is one of if not the strongest part of our project.

What we learned

  • How to train and deploy a NLP using Microsoft Azure
  • Create a beautiful UI using Chakra and Nextjs
  • Using Spotify's Developer API
  • NextAuth authentication software

What's next for AudioBook

We have yet to completely configure to Spotify API necessary to enable songs to be directly queued and played using our software. We hope to implement this more in the future.

Built With

Share this project:

Updates