CatchACab | Devpost

output data

The mobility sector is at the beginning of the biggest transition in its history. In a likely scenario 10 to 15 years from now people will be using autonomous vehicles to get from point A to point B. One of the challenges the developers have to face when designing the car of the future will be how the user can communicate with the car. Especially in the case of elderly people who are often reluctant to adapt to new technologies a transition has to be as seamless as possible. Also a lot of use cases are imaginable in which the user has no access to other electronic devices. How would one hail a cab without these? Our solution: we are using image recognition to detect hailing signals which are identical to the way people have been hailing a cab for a long time. We identified several features such as the position of the hand or the position of the person to the road which indicate an interest of the pedestrian to be picked up by the cab. We implemented a service for each feature which returns a probability of it happening. This gives us a weighted feature vector which then is summed up to the final result. As there won't be any natural human judgement in a cab without a human driver, the system needs to be able to identify threats such as drunk people. Therefore, there is a blacklist of items that decrease the chance of a potential passenger being picked up. Another major problem without the access to communication technology we experience pretty long waiting times as arriving cabs are up to chance. For this reason, we envision all future autonomous cars to be equipped with our system and have the ability to automatically notify nearby cabs. In cases of uncertainty our vehicles will first slow down and analyse if the person takes an interest in the cab by analyzing his behavior such as the direction head is facing or the way he reacts emotionally.

Challenge1: App-free Car Hailing (AID) Not everyone has a smartphone at any given point, even in a digitized world like ours. That's why people should be able to hail an autonomous cab with just their hands - like in the good old days. For that, the car estimates the pose of every person it sees and detects certain blacklist items like alcohol or weapons that should not be allowed in the car.

Challenge2: Smart Vision Application (Intel) We used the Intel OpenVino for our perception algorithms. They enabled us to quickly develop an awesome application that runs efficiently on CPUs..

object detector (people, bottle...)
wireframe of people ; generates wireframe data -> got hand node of wireframe and coded a rectangle around that node (snippet follows hand as well) -> cut image to that -> save image 1/10 -> object detector loads image 1/10 ; generates obj data i.e. drunk person
facial measures happiness and eye contact to car ; generates data
finger count: takes cropped & positioned image (entails hand) and counts the fingers ; generates data
fusion takes all data and generates the best person & amount of people to ride in a cab

mix of C++ and python