GrassRoot

A use of the new T-Mobile Your Number Anywhere API to collect and document canvasing data for remote areas. Uses AI tools in order to process and record information.

Comment

Inspiration

When the T-Mobile Your Number Anywhere (YNA) API was proposed to our group we immediately thought “data collection”. After brainstorming ideas of how we could use the features of this API, we decided to focus on connecting companies to customers with minimal internet access. This could be used for COVID data collection, political polls, or services that customers could call into. We quickly realized the real power of this API is automation of phone calls, and not just simple robo-calls. We could incorporate any other 3rd party APIs and/or custom code to make a really powerful system. For example, LLMs could work in conjunction with transcription AI to interpret spoken word and label responses based on sentiment. The possibilities are endless, and although we had to focus on one idea for this hackathon, there are so many areas to explore in the future.

Purpose

It is essential for political figures and advertisers to have a cost effective way of reaching and recording information from their constituents or intended audience. Many politicians hire part time employees or volunteers to call people in an area to collect interest and inform them of voting schedules, or to garner support. Smaller and local politicians don't operate with large budgets, and this can either become very expensive or rely heavily on volunteers. However, this overly tedious job can be done simply and effectively through automation. Our product has the goal of informing and collecting data on a large amount of constituents in a specified area. The audio data from these calls are transcribed and processed by machine learning tools, providing companies and politicians a better understanding of their constituents needs, enabling them to alleviate key issues in the community. The politician or advertiser benefits from the labor reduction and better information. The constituent benefits from making sure their concerns are heard and interpreted, not just as yes/no questions, but in their own words. Lastly, this service is sustainable because companies will pay to use it, and there is no other system like this currently on the market.

Creation

Our creation entails of a few parts. Firstly, we worked on code that could view and edit a collaborative spreadsheet. We figured the collaborative element was important, since that way the users could see information enter the spreadsheet as it was collected, and share it with any other employees instantly. Once we get that working we moved on to the AI transcription. Using the Google Speech-To-Text API, we transcribed an audio file into text, and piped it into the spreadsheet element. With the Google TTS API, we incorporated text-to-speech. This is important because we want the system to be completely automated, so we need the program to ask the customers the question of interest (i.e. "What are your opinions about X candidate?"). From there, we saved the response to an audio file, which is then transcribed upon the end of the call. Finally, we created a basic user interface for the user to enter the question of interest, and a list of phone numbers to call. This provided a basic skeleton app that had all the functionality we needed, and demonstrated the power and flexibility of GrassRoot.

Challenges

In the process of making our app, we ran into quite a few challenges. The first of course was the 24 hour time limit, meaning that we could only work with the skills we came in with. One of our members was proficient in JavaScript, which is the main language we used for this project. Another member had various experiences in website building, and was able to whip up a simple web interface for the app. The main challenge we faced was working with the T-Mobile YNA API. We were all new to the API of course, but the API was also new itself. The documentation was difficult to interpret, and the demo code given to us was unable to run which made it pretty challenging. However, we pivoted our efforts and demoed our app using a laptop microphone, allowing us to circumvent any issues with the API. Given more time with this app, and time to debug the demo code, we are confident we could incorporate the YNA API and have a fully functional demo.

Accomplishments

In the end we were very happy with the progress we made in just 24 hours, with a working demo and an app that we believe could easily be turned into a real and profitable service. We combined the abilities of four APIs and could add more for extra functionality with fairly little development, due to the modular design of the app. We all gained very valuable skills with APIs and creating apps in JavaScript. Most importantly, we got experience working in a team to brainstorm, design, and implement an entire project from start to finish. Walking away from this hackathon, we've gained invaluable expertise that can and will be used in our careers going forward.

Future Improvements

There are many improvements we have in mind for GrassRoot. Most notably, an LLM such as OpenAI's Chat-GPT could receive the transcribed responses and perform literally any operation on them. For example, personal opinions about a political candidate could be analyzed to determine how likely that person is to vote for a candidate. Or, a patient's verbal description of their availability over the next year could be used to schedule an appointment. This could also be used to port other apps to GrassRoot for accessibility. Sometimes a call is easier to interact with than a website, especially with a slow internet connection or an incompatible browser. This technology will prove its worth in the lower stakes information collection based setting. However, with the strengths of the AI language model and rapid ability to record language in real time over the phone many other avenues to help customers open. In cases of danger where a caller is not able to speak direct words, the model will be able to discern their situation from their tone of voice and speech patterns.

Built With

Share this project:

Updates