Sentimentor | Devpost

Inspiration

Our app addresses a common challenge in online lectures: the difficulty for professors to gauge student engagement. By leveraging the capabilities of online learning, we provide tools for instant feedback, empowering professors to assess and enhance student engagement effectively.

What it does

The web app streamlines the setup of lecture rooms for professors, eliminating the need for tedious logins. Students can seamlessly join these rooms using a unique join code, reminiscent of platforms like Quizlet Live. Upon entry, students activate their webcams, enabling the program to conduct real-time sentiment analysis. This analysis assesses whether students are confused or content with the lecture content.

Additionally, students have the option to provide feedback through sentiment buttons located conveniently at the bottom of the screen. Meanwhile, the program autonomously evaluates student drowsiness, providing invaluable insights for the professor's awareness.

The culmination of sentiment and drowsiness analyses is presented succinctly on the professor's dashboard in the form of intuitive statistics. Armed with this comprehensive feedback, professors can dynamically adjust their teaching approach to optimize student engagement and comprehension.

How we built it

Backend: PyTorch Tensorflow

Utilizing Python, we crafted two powerful machine learning models leveraging PyTorch and TensorFlow libraries. For sentiment analysis, we drew inspiration from the Residual Masking Network, integrating its methodologies to discern student emotions effectively. Our adaptation of driver drowsiness detection, sourced from the GitHub repository, https://github.com/satyamgeek/driver_drowsiness_system_CNN, provided a solid foundation for evaluating student alertness.

To orchestrate server operations seamlessly, we employed Node.js, ensuring efficient handling of data processing and communication.

Frontend: TypeScript React.js Tailwind.css

In crafting the user interface, we opted for a robust stack comprising TypeScript, React.js, and Tailwind.css. TypeScript's static typing facilitated code maintenance and reliability, while React.js empowered dynamic UI rendering for a fluid user experience. Tailwind.css streamlined the styling process, enabling rapid prototyping and aesthetic refinement. Collectively, these technologies synergized to deliver an intuitive and visually appealing frontend interface.

Challenges we ran into

We encountered several challenges throughout the development process:

Originally, we aimed to incorporate advanced media-sharing features like screensharing and muting/unmuting capabilities directly within the web app. However, integrating web sockets proved more complex than anticipated, leading us to revise the scope of the app to serve as a supplementary tool alongside platforms like Zoom or Google Meet.

Implementing the functionality to send periodic screenshots posed a significant hurdle. Managing multiple values and transmitting them as binary files presented technical difficulties, particularly in integrating the frontend with the backend. Initially using Flask, we encountered numerous issues, prompting a shift to FastAPI. Additionally, we refactored our code to leverage asynchronous programming to improve performance.

Time constraints posed another challenge, limiting our ability to enhance the frontend's visual appeal as desired. Despite our intentions to create a more impressive user interface, time limitations necessitated prioritization of core functionality over aesthetic refinement.

Accomplishments that we're proud of

We take pride in several key accomplishments achieved during the project:

Successful Implementation of Machine Learning Models: Despite the complexities involved, we were able to integrate two machine learning models seamlessly into our app. Maintaining performance efficiency while ensuring respectable accuracy levels speaks to the dedication and proficiency of our team.

Robust and User-Friendly UI Design: Our commitment to delivering an intuitive user experience is reflected in our clean and straightforward user interface. By prioritizing simplicity and clarity, we have created an interface that enhances usability and facilitates seamless interaction for both professors and students alike. What we learned Throughout the project, our team acquired valuable knowledge and skills:

Mastery of FastAPI Integration: Through hands-on experience, we deepened our understanding of FastAPI and its seamless integration with frontend technologies such as HTML, CSS, and JavaScript. This expertise enabled us to develop a robust backend infrastructure that effectively supports our application's functionality.

Advanced TensorFlow and PyTorch Utilization: In adapting machine learning models to our product, we delved deeper into the TensorFlow and PyTorch libraries. This process provided us with invaluable insights into optimizing model performance and integrating them seamlessly into our application framework.

What's next for Sentimentor

Implementing our app as a dedicated Zoom or Google Meet extension is an exciting prospect that aligns with our vision of enhancing online learning experiences. By seamlessly integrating our functionality into these popular platforms, we can provide a more comprehensive solution for professors and students alike.