Sentimuse

Sentimuse brings AI poetry to life with expressive narration and music that perfectly matches the mood

Comment 1

Inspiration

AI has been popping up everywhere to help make content creation easier, from writing scripts for videos to generating images and music. A lot of creators are already using these tools to speed up their work or add something new to it. Our team wanted to take that same momentum and apply it to poetry. So, we built an app that generates poems and brings them to life with voice narration and music, all in one place.

What it does

Sentimuse is an AI-powered platform that creates emotionally expressive poetry, bringing it to life with personalized audio narrations and seamlessly paired mood-matching background music, offering a unique and immersive storytelling experience. The application would provide with a seamless and immersive storytelling experience through customizable AI-generated content.

How we built it

We split the tasks by features (poem, sentiment, narration, music, and merging) for each member. The UI/UX process included wireframing, which was exclusively done through Figma with a careful selection of colors, fonts, and more for better accessibility in the application. Throughout the entire application, we used the following AI tools:

  • GroqAPI for Llama 3.1 to generate narration and select sentiment
  • ElevenLabs for narration generation
  • Jamendo for music generation
  • Supabase for export as link
  • Transloadit for merging audio

Challenges we ran into

Since we hosted through Versel, there were a lot of limitations for things like the processing power, storage for files during merging, AI tools that would be compatible, etc. These challenges often set us back a few days since the research, learning, and implementation loop would have to repeat each time. Additionally, finding tools that are free to use was incredibly difficult.

Accomplishments that we're proud of

The biggest accomplishment was completing the project in a short amount of time. Although many improvements could still be made to the application, we were able to finish the entire flow from the poem generation to the export as a link. We are also proud that we got the narration to be expressive and have the music set just right to support the narration in bringing the poem to life.

What we learned

Many team members had never worked with the AI tools we used, so it was a lot of learning through trial and error. We also did not have much experience with using Github Projects, so it was hard to keep up with constantly updating it based on the progress that we had made. Additionally, creating the front end to match the Figma was a struggle since the padding and sizing were initially different across many pages.

What's next for Sentimuse

The next step for Sentimuse is implementing image generation to have an mp4 format with live captions. This feature could complete the entire circle for content generation, allowing the application to make poems with the final product being fully ready to upload to social media.

Built With

Share this project:

Updates