Neo=Alert

Inspiration

Before Rebecca John's talk, we didn't recognize the graveness of healing NICU patients in under resourced regions/areas. Moved by her personal experience as a Neonatal nurse practitioner working at SickKids, we were inspired to innovate, develop, and train our own AI model to prevent further premature baby deaths.

What it does

In the finished product, the software would be hooked up to a NICU unit and would have ECG and respiratory data streamed directly to it. In our demonstration, this is not present for reliability. By running our custom AI (CNN/RNN-LSTM) prediction/classification model constantly on this data, we can help predict possible cases of bradycardia well in advance, giving doctors time to respond and save NICU infant's lives. This data is sent to a custom platform where the data can be shown on the hospital's account with early warnings, including a visualization of a batch of the ECG and respiratory data.

How we built it

AI Part: We first identified a dataset to use for the model training. Our dataset of choice was PICS, a ECG and respiratory dataset. We transformed it into a Huggingface datasets library dataset and uploaded it to the site. Upon upload, we realized that the ratio of classes was 99.4% to 0.6%, which was far too low to train the model correctly. We then implemented a data augmentation system using statistical techniques to generate new data which can be used to train the model effectively. We defined the model ground-up using Pytorch and initialized weights at random, and did 2 epochs on the 208k length dataset. The model was over 98% accurate on evaluations (the original dataset), which was our go-ahead to integrate it into the platform. We used one Nvidia A100 (80gb vram) to train the model as quickly as possible.

Frontend Part: Using nextjs, we made a fully functional dashboard to visualize ECG and respiratory data, along with a warning system to let doctors know whether a patient will be going into bradycardia. Our data is displayed in a 15 second window, where the doctor can utilize a sliding window through the full set of data to look at historical patterns throughout the patient's data.

Challenges we ran into

AI Part: We first ran into a data label inbalance in the original dataset. This was solved using statistical techniques to augment data, which allowed us to end with a 50-50 ratio of bradycardia to not-bradycardia. The next major issue was a model size issue. Since we use a CNN/LSTM hybrid model, we needed to ensure that the CNN could extract the correct features from the input sequence and the LSTM could extract the long-term dependencies to accurately predict whether the patient has bradycardia or not. After a long deliberation process, we decided on a model with roughly 136M parameters. We were able to do this loading and training the model with BF16 weights instead of traditional FP32 ones, slashing the compute needed in half.

Frontend Part: We ran into a major issue with linking the model and the frontend systems. We tried using a port system to transfer the predictions over, but ultimately we had to come up with a supplementary method for demonstration using gradients, although the model is still fully functional and extremely accurate.

Backend Part: We ran into issues parsing and ingesting test data from the very large (multi-gigabyte arrow files). We ultimately ended up parsing one file, and putting the first 2 minutes, or 37.5k rows of data in a postgres table using a script. We also ran into many issues with random bugs in Next.JS and getting the data to render properly on the web.

Accomplishments that we're proud of

We successfully made an AI model from scratch with an extremely high accuracy rate (~98% accuracy). We also created a fully functioning sliding window so that doctors can view all data (respiratory and ECG rates) at once.

What we learned

We learned how to use the engineering design process and efficient combined it with machine learning to make a functional product. Participating in this hackathon also increased our productivity rate and time management skills.

What's next for Neo=Alert

With more resources, we will be able to create futuristic glasses that have properly integrated systems and finalized multi-patient functionalities. The glasses will feature augmented reality and alert any medical emergencies to doctors and nurses' assigned NICU patients. These glasses will be very helpful in third world countries since the only hardware needed for doctors are augmented reality glasses. This means that even if hospital facilities are under developed, these trained healthcare professionals will still be able to effectively and accurately treat their patients.

Built With

• accelerate
• ai
• huggingface-datasets
• nextjs
• postgresql
• python
• pytorch
• supabase
• typescript