Lucarne

Lucarne is a visual translation platform that identifies objects in a photo and gives their translations.

Comment

Inspiration

Driven by an interest in all things augmented reality and a desire make to communication among all people easier, our team strove to create a platform that does both. Lucarne helps us break down barriers and allow us to enjoy the beauty in our world together.

What it does

Lucarne first prompts the user to take a picture with their mobile device, and select a language for translation. This image is then processed, and the names of the most predominate objects are identified. These names are then translated into the selected language and displayed to the user as a caption for the photo.

How we built it

The driving force behind Lucarne is a combination of two powerful Microsoft Cognitive Services API's, Computer Vision and Translator. The Computer Vision API "offers image “tagging” based on more than 2000 recognizable objects, living beings and actions." These tags are returned along with other data that must be parsed through, adding to the difficulty of the project. Once isolated, these tags are then sent to the Translation API. To complete the translation, a security token is required. This token expires after ten minutes, and is taken, along with the phrase to be translated, to the translation API. The phrase is, as the name suggests, is then translated, and returned in the specified language.

Challenges we ran into

The greatest challenges that ran into were related to time constraints and inexperience with javascript. Getting Microsoft's Translator API to work was very challenging because there were no examples of the GET and POST requests online. Thus, we had to toy around with POST requests until the web server responded. Additionally, CV2 has some difficulties with processing accents and more symbolic languages.

Accomplishments that we're proud of

We are a team of mostly first time Hackers! Two freshmen and a old senior come together and made something work!

What we learned

How to make a static website. How to use Microsoft API, OpenCV.

What's next for Lucarne

Integrating the program with a fully functioning website, as well as improving the user interface of the website. We would also like to add support for live video streaming, with real-time translation.

Built With

Share this project:

Updates