HashGen - TikTok TechJam 2024

Inspiration

This project was borne out of wondering how we could make our TikToks go viral. Even the best videos won’t make it far on the FYP if they do not have the necessary hashtags. This made us think, how can we automatically generate hashtags on a video to help it gain popularity?

In the ever-evolving landscape of social media, content discoverability and engagement are paramount. TikTok's dynamic platform presents a unique challenge and opportunity for content creators to enhance their visibility through effective hashtag strategies. Inspired by the need to simplify this process and leverage advanced AI capabilities, we developed HashGen. Our goal was to create a tool that empowers TikTok users to effortlessly generate relevant and trending hashtags by analysing both the audio and video components of their content.

What it does

HashGen is an AI-powered tool designed to analyse TikTok videos and generate highly relevant hashtags to boost visibility and engagement. By processing both the visual and auditory elements of a video, HashGen provides users with a curated list of hashtags that are tailored to the specific content, maximising its reach on the platform.

How we built it

HashGen was built using a combination of cutting-edge technologies and development frameworks:

Frontend: We utilized Next.js to create a responsive and user-friendly interface where users can upload their videos. The frontend is designed to provide a seamless user experience, from video upload to displaying the generated hashtags.
Backend: The backend was developed using Flask, a lightweight Python web framework. It handles video uploads, processes the video and audio data, and interfaces with the AI model to generate hashtags.
Models: We employed OpenAI's CLIP model to extract meaningful features from the video frame. We also utilised OpenAI's Whisper for audio transcription, Hugging Face for audio classification, Audd.io for song recognition.
Data Processing: We leveraged the TikTok Hashtag Analysis toolkit to gather and preprocess data for training our model. This involved scraping TikTok for videos and associated hashtags, extracting features, and training the model to recognise patterns in the data.

Challenges we ran into

We ran into some issues with the integration of the python and react frameworks, as most of us have not had much experience with linking both frontend and backend together. This is also our first foray into machine learning, so there was a very steep learning curve figuring out how to configure the pre-learned models to our hack.

Accomplishments that we're proud of

Functional Product: Successfully developing a fully functional product that meets our initial vision and provides real value to TikTok users.
AI Integration: Effectively integrating advanced AI models for both video and audio processing to generate relevant hashtags.
User Experience: Creating an intuitive and user-friendly interface that simplifies the process of enhancing video discoverability on TikTok.

What we learned

Through this project, we gained valuable insights into:

AI and Machine Learning: Deepened our understanding of using AI models for multimedia content analysis and the intricacies involved in training and deploying these models.
Full-Stack Development: Enhanced our skills in full-stack development, particularly in integrating frontend and backend technologies to create a cohesive product.
Problem-Solving: Developed our problem-solving skills by tackling various challenges related to data processing, model training, and system integration.

What's next for HashGen

Moving forward, we plan to:

Enhance the Model: Continue improving the AI model to increase the accuracy and relevance of the generated hashtags.
Expand Features: Introduce additional features such as hashtag trend analysis, personalized hashtag suggestions based on user history, and integration with other social media platforms.
User Feedback: Gather feedback from users to further refine the product and ensure it meets their needs effectively.
Scalability: Enhance the scalability and performance of HashGen to handle a larger user base and more complex video processing tasks.

HashGen represents our commitment to leveraging technology to empower content creators and enhance their social media presence. We are excited about the potential of HashGen to revolutionise the way TikTok users engage with their audience through smarter hashtag strategies.