TailorFrame

Rebrand without the Reshoot Audrey and Devender

Comment

Inspiration

Novel creators live on growth and building on new ideas -- not redoing the same idea repeatedly. Changing details, such as locations of an event or a product delay, it should not be time-consuming to provide an update. This can become expensive if the content was filmed at an exclusive venue or if the event details change rapidly. TailorFrame fixes words in the footage, so creators don't have to redo their content for these updates, saving creators' time, money and momentum in their campaigns -- tailoring their quality content into something that serves them no matter the change!

What it does

TailorFrame is a webapp that fixes words in a video using a simple workflow: Uploading the video, changing the transcript and downloading the updated file. This edited transcript is sent to ElevenLabs to edit a word in a voice line. In practice, creators can switch out “Monday sale” to “Friday sale”.

How we built it

Frontend: Next.js, CSS and Tailwind with a responsive desktop and mobile interface Backend: Postgres database which stores saved transcripts and uploaded videos. Docker for containerization and Vercel for deployment. AI: OpenAI API to standardize the transcripts and used ElevenLabs for the audio creation. We also utilized Cursor and Bolt for faster development.

Challenges we ran into

We experimented with a few ideas in the beginning from ideation to pure video manipulation. We both worked on different tasks and experimented with ideas to find something that was achievable within the time frame, was interesting to build and technical enough to challenge us. We decided on working with audio and videos. In the end, we decided to do only audio, as lip sync was out of scope for the duration of the hackathon. Figuring out the algorithm for audio extraction and segmentation was challenging, as we had to determine how the workflow works between ElevenLabs, OpenAI API and our app.

Accomplishments that we're proud of

We estimated the scope correctly, as TailorFrame’s main functionality was created within 24 hours. It was both fun, challenging and insightful to work with unfamiliar technologies. Also, it was very satisfying when the audio and video synced up.

What we learned

Both of us had not worked with ElevenLabs before. Additionally, looking at AI solutions in regards to audio/video was a unique use case. Both grateful for the opportunity for HackAI; it truly challenged us to create something new, fresh and interesting.

What's next for TailorFrame

We'd like to do sentence replacement, instead of only using words for more flexibility for creators to edit their videos. For this, we'd need lip syncing to edit the video, which we explored but would be expensive and too large a scope to achieve in the timeframe.

Built With

Share this project:

Updates