CommentSense | Devpost

Image of Chatbot Interface

Description

Our product addresses the challenge of inspiring creativity among content creators by leveraging a generative AI-powered chatbot. This innovative solution is designed to push creative boundaries and provide valuable, real-time feedback, enabling creators to enhance their content and engage more effectively with their audience.

Problem Statement

📚Track2: Inspiring Creativity with Generative AI Global users' demands and expectations towards streaming media content are gradually increasing, with an increasing number of creators joining the streaming media industry. Develop a mobile or web application utilizing generative AI technology based on the creation and consumption scenario of streaming media, aimed at optimizing either the productivity or content quality of creators or the consumption experience of the audience. The streaming content here can include images, videos, live broadcasts, text, etc.

Key Features

Automated Content Analysis: Users input a TikTok video URL, triggering a multi-step analysis process.
Comment Scraping and Clustering: The system extracts comments from the video and uses machine learning to categorize them into meaningful groups.
Video Context Extraction: Employing the LLaVA (Large Language and Vision Assistant) model, the application derives additional context from the video content itself.
Integrated Database Storage: All analyzed data is efficiently stored in a SQLite database for quick retrieval and persistent insights.
AI-Powered Interactive Assistant: An intelligent chatbot with agentic properties provides users with tailored feedback and answers based on the analyzed video context and user queries.

Data Flow

User inputs TikTok URL in frontend
Backend receives URL and initiates scraping and analysis process
Scraped data passes through the processing pipeline
Processed data is stored in the SQLite database
User navigates to chat interface
Chat agent retrieves context from the database
User interacts with the chat agent, which provides responses based on the video context and user queries

Development tools used to build the project

Development Tools	Purpose
Next.js	Frontend
FastAPI	Backend
SQLite	Database
Docker	Containerization
Git	Version Control

APIs

TikTok API: Extracting video data and comments
Gemini API: For summarizing comments and chatbot

Assets

Video Data: Extracted from TikTok videos

Libraries

Libraries	Purpose
BeautifulSoup	Parsing Extracted Comments
SQLAlchemy	ORM for db
fastapi	Backend
Langchain	Build Chatbot
HuggingFace	For Llava model
Mantine	Frontend Aesthetics
React	Tabler Icons, Hot Toast
NextJS	Image and Routers
Typescript	For next.js