Our inspiration was primarily from understanding the scene in education and wanting to make a change in virtual lessons in specific where students tend not to pay attention due to lessons which lack human touch.
We have 2 components to our entire project. Firstly, we made use of a Iris Tracking model to observe attention of students when they turn their webcam on during lessons and came up with a list of % attention of students throughout the lecture. Using this, we mapped with portions of lessons that are hot spots where students are not paying attention. Using generative AI (Berri-AI and ChatGPT), we mapped a transcript of what is being said during lesson during a specific timestamp to student's % attention during that period. We offer specific and personalized responses from these generative AI models on how teachers can improve their lessons to increase attentiveness of students in each of these concepts taught during the lecture. Test scores may also be uploaded for teachers to visualize which portion of exams students are not doing well in and teachers can recap these sections in class. Finally, we offer automated report generation to provide statistical feedback on how students have fared after every lesson. The best part is that teachers are able to ask any questions regarding their lecture and with the help of our customized ChatGPT/Berri-AI, we are able to answer these questions.
Thus, quite literally 'A Cortana For Education'.
Our whole idea takes the concept of learning to the next level where virtual teaching becomes less of a pain for teachers as they are able to monitor their students directly. We have even come up with surveys and interviews with our teachers to come up with this final product. If necessary, the entire survey and interviews will be published.
Our next step would be to roll out such a product in real life as well through the use of thermal PTZ camera's in an actual face to face education setting. More informative statistics can also be provided and we are currently working to integrate a sentiment analysis and gesture detection model along with our eye tracking attention models to achieve maximum accuracy in attention detection.
Note: Our project's UI is in the form of a telegram chatbot which can easily be converted to a webpage UI. Essentially what is more important in our project is the AI and mechanisms involved in computing attention and the generative AI component, all of which are rather advanced features.
Log in or sign up for Devpost to join the conversation.