Inspiration
We wanted to create a device that would help to communicate with victims of natural calamities in real-time. The prime motivation is to increase the efficiency and effectiveness of first responders.
What it does
Our Solution is an AI-enabled image classification product. Based on any image, it will identify it as burnt or wound or normal. Thereafter Hermes starts getting more information about the state of a victim by having a conversation with them and offers the most suitable solution as soon as possible. It uses state of the art NLP and deep learning cloud-based algorithms(IBM) to detect various types and region of wound and stores all the conversation for further analysis.
This is a tool that would be put on some sort of automated scout like a drone in the aftermath of a natural disaster. This drone would then use the Hermes system in order to asses the severity and type of the injuries of people it encounters and then relay that data to the first responders. Giving the first responders information about the victims of natural disasters in an affected area will allow them to be prepared for what injuries they will have to treat as well as prioritize victims who need immediate medical attention. This prioritization and location of victims in a triage like system will allow the first responders to operate in an efficient and organized fashion even in the most chaotic of events.
How we built it
We have used Flask to make our application. The front end is based on React and Node.js. Firstly, we used to upload certain wounded picture. Then, we have used IBM Visual Recognition (Machine learning API) to analyze each image and identify the type of effects on a victim of affected areas. Further, we used the IBM Watson Assistant (NLP API) to have a conversation with the victim to understand their situation and immediate requirements. The complete conversation is stored in Mongo Atlas Cluster which will be used for future analysis. The complete architecture was microservices-driven and all the rest endpoints have been tested used Postman tool.
Challenges we ran into
Our team was new to API's and Front-end and our project involved a lot of IBM cloud API calls which were limited to lite mode. It was difficult to interface the backend and frontend, in the beginning, we struggled a lot in getting desired results, understand the platform and creating a model.
Accomplishments that we're proud of
We are so proud of the whole project itself. We were able to fit all the pieces together and create a working MVP. Given the diversity in our team, we enjoyed a lot.
What we learned
We learned how technology can help in making a powerful impact in today's world. Most of us were first-time hackers and we learned to keep working hard! We even understood the importance, a good mentor can bring to the success of any project.
What's next for Hermes AI
There are lot of improvements planned for Hermes:
- Convert the conversational AI agent to drone agent for an immediate solution
- Make a Prediction for natural calamities.
- In-depth analysis of conversation between the stakeholders to be prepared and help efficiently.
Log in or sign up for Devpost to join the conversation.