Cognetech | Devpost

Thumbnail
Landing Page

Inspiration

Child Psychologists often spend hours, days, or even weeks, timestamping and coding behavioural indicators in hours of footage with their patients to make their diagnoses and assessments. This process of manually coding their footage is unnecessarily time-consuming and tedious.

What it does

Our solution to this inefficiency is an AI-powered video indexing tool that utilizes video classification and semantic understanding to generate timestamps and behaviour annotations throughout the video for them to browse behavioural indicators in seconds, rather than days. Cognetech is a tool to empower and assist child psychologists, not replace them. It helps them conduct their clinical assessments without the hassle of having to review hours of footage themselves. It will not make a determination for them, but provide them with the means to make one faster and more efficiently.

Identity!

Every child possesses a unique behavioral "fingerprint" that defines their individual identity, yet these nuances are often buried within hours of clinical footage. Cognetech serves as a high-definition lens, surfacing the subtle cues and idiosyncratic patterns that make each patient distinct. By isolating these specific behavioral markers, the tool empowers psychologists to build a diagnostic profile that is as singular as the child themselves. This personalized approach ensures that assessments are based on the full complexity of a child’s identity rather than generalized observations.

How we built it

We’re using TwelveLabs’ video indexing and understanding engines, Marengo and Pegasus, to generate timestamps and annotations of behavioural indicators that the patient exhibits. We've constructed a FastAPI backend hosted with Railway that queries the TwelveLabs Analyze API for timestamped behavioural markers from uploaded clinical footage without storing any specific and sensitive information. The frontend is uses ReactJS and ThreeJS hosted on Vercel to deliver for an interface that healthcare professionals feel compelled to use.

Challenges we ran into

Creating a novel idea for a project was our biggest challenge. We struggled to find a situation in which video indexing generates a lasting, meaningful impact. From the time hacking began to the next morning at 10AM, we were still in the idea brainstorming stage. At which point we found an angle for video indexing to genuinely help healthcare professionals in the child psychology field. Once we had our idea, we immediately began building our application with ease and determination, knowing that this project would help these professionals in a valuable way.

Accomplishments that we're proud of

We’re proud of the progress we made in the relatively short timeframe we had to develop within. Our idea came to us later than expected, but development accelerated quickly because of our shared passion for our project idea. We’re also very proud of our UI and UX, as we wanted to play around with three.js to bring the presentation of our project forward. We wanted healthcare professionals to feel compelled to use our application, having been presented with a stunning and intuitive interface that reliably saves them time.

What we learned

Through this project, we learned how to innovate under intense time constraints while still building something meaningful. Since we were using TwelveLabs’ API, something we hadn’t worked with before, we had to quickly learn new technical concepts and apply them across multiple parts of our build. The biggest takeaway was learning how to stay focused, make fast decisions, and still deliver a complete solution within the scope of a hackathon.

What's next for Cognetech

Expand the scope of the tool to become a desktop application that serves as child psychologists’ own file system for storing the notes generated from their past sessions. Another area for improvement is to expand the scope of this project from a child psychology perspective to a broader patient demographic.