Talk2Code | Devpost

Inspiration

This idea came to us while thinking of a way where everybody will feel included in the programming community. People with disabilities, arthritis and RSI makes it difficult for people to program doing it the traditional way with mouse and keyboard. Our software allows voice as a medium to allow users to code their ideas into a reality and allowing a new way to learn and enjoy coding.

What it does

It uses NLU using DialogFlow to communicate with Google Assistant which turns speech into a coding line. Talk2Code is a platform that enables everybody to feel included in the programming community. Currently, the traditional process of programming with mouse and keyboard is very difficult for people with disabilities, arthritis and RSI. This software allows voice as a medium to allow users to code their ideas into a reality and allowing a new way to learn and enjoy coding. The platform leverages Google's Natural Language Understanding(NLU) using DialogFlow, communicates with Google Assistant, and utilizes a Node.js server to parse the speech that is processed through Dialogflow into code.

How we built it

We used used dialog flow for the front end. Once dialog flow figures out what type of code is it and the parameters associated with the code. It send an api request our node server where each line is code line is processed (if required) and concatenated.

Challenges we ran into

Creating the compiler to turn the human English language into javascript was harder than it seems.

Accomplishments that we're proud of

Being able to create a simple program using your voice. The program is generated from natural language understanding

Sample program

users says: make x equal 100 returns: x = 100

users says: make planet equal mars returns: initialize planet = Mars

users says: if var1 greater than var2 returns: if (var1 > var2)

What we learned

-how to use node.js -using a nlp processing engine, Dialog Flow

What's next for Talk2Code

Due to time constraints, we had to had to limit the functionality for this hack. We would like to continue to integrate the items listed in challenges. Afterwards, we would like to improve the nlp engine by creating a bigger dataset to dialogbox. One software Architectural design we could have improved is have seamless processing and generating code as you are talking.

Our goal for for Talk2Code would be to become a new opensource platform where programmers all around the world can come together and help contribute to this codebase.