VeRFluent

Inspiration

We were thinking about how when studying abroad, it is easier/faster to learn the language due to its immersive nature and inability to communicate in your preferred language. With the idea of learning a languages and VR, we thought we could emulate this studying abroad experience without the costly prices of airline tickets and hotel fees as well as having a helping hand rather than completely being unable to communicate/understand.

What it does

VeRFluent puts the user into situations with objectives to complete which teaches the user phrases, vocabulary, and culture with associated language. For our demo, we decided to focus on Chinese since most of our developers knew Chinese. With the time constraint, we created a short scenario of one eating at a Chinese person's home and the etiquette associated with it.

How we built it

We used mostly Unity as well as objects and creations from SketchFab and Mixamo. Whatever objects we could not find or download on these sites, we made on Blender(dumplings). Making use of our resources, we had some help from the mentors at the Hackathon as well as Slack.

Challenges we ran into

• figuring out Unity and Blender for the first time
• how to get speech to text to work
• how to animate a character fluidly (as possible)
• ## Accomplishments that we're proud of A few accomplishments that we are proud of are:
  • including a speech to text feature
  • creating a original object (dumplings) through blender
  • got the animations (granny) to work

What we learned

We learned how to use Unity and the different features it has, along with getting familiar with HTC Vive Pro.

What's next for VeRFluent

Firstly, we would like to have many more scenarios such as going to the store, eating at a restaurant, going to a festival, etc. We would also like to add many more languages such as Korean, Japanese, Spanish, French, etc. Another idea that we would we could add to our program is a multiplayer feature. We believe learning a new language with another person allows faster development and understanding of the language. It would put two people learning the same language into a scenario, such as going to the grocery store. And they would work together to complete objectives, for example: 请找香蕉 (please find the bananas) or 请在收银台支付150元 (please pay 150 yuans at the cash register). It would implement a call feature so they can freely speak to one another as well as help each other on vocabulary or phrases that may stump the other.

Built With

• blender
• unity