-
my original idea written in paper Saturday morning
Inspiration
Free software (open source) has had a large impact in the development of modern technology. The culture of free software has become extremely relevant in our community. My inspiration for this project was to contribute to the free software community. My goal was to create an open source tool for developers to use and expand on.
Protobuf is a powerful tool used by Google's cloud services to store and serialized data. It is fast and efficient for the computer but not so much for the developer. Protobuf's API is untraditionally, non-object-orientated. It is easy to use for small application but for larger applications it requires extensive hardcoding and complicated algorithms to search through nested message types.
What it does
proto-query behaves like a middle man between a software application and Google's Protocol Buffer (protobuf). proto-query's API allows developers to retrieve messages from serialized data with Protobuf's API using SQL like queries. This allows the developers to spend more time in their software applications and less time deserializing data.
How we built it
First, I started by creating a small sample of serialized data using Protobuf's API and dummy data from the internet. After running a series of tests I figured that the best way of extracting serialized data was by wrapping Protobuf messages and searching for other messages with in them. Then, I began working on a class with the capabilities of accepting queries.
Challenges we ran into
I chose to use SQL like commands because of their popularity and common knowledge amongst developers. This was a challenge because protobuf nests data (like JSON) rather than creating a table like storage system like SQL-Databases. SQL was not created with the intentions of searching through nested data.
Another challenge was finding child messages within a message. Say that I want to select the number from person where the name equals Joe. I begin by wrapping the root message which in this case would be address_book. This is difficult because protobuf messages do not allow us to call for a sub/child message. There is only the option to use a boolean function message.HasField(_field_)
Accomplishments that we're proud of
One accomplishment I am very proud of is starting this project. I have been wanting to participate in another hackathon for a very long time. This is my first hackathon through MLH and it has been a great experience. This project has helped me get my mind off of work and school. It reminds of the reason why I decided to choose Computer Science as a major in the first place.
Another accomplishment that I am proud of is that I was able to expand my knowledge. Read What we learned.
What we learned
I was able to identify the nature of nested data. Nested data behaves a lot like trees, therefore allowing a simple recursive implementation when searching through nested data (like JSON). Also, being able to learn more about the nature of how SQL queries are processed and possibly used in SQL applications.
What's next for proto-query
I think proto-query can have two paths:
- proto-query can be just another hackathon project submission that will be later forget about and sit in a dusty old repository on GitHub or-
- I can decide to finish proto-query. That means finishing the API, testing with different queries, adding more SQL like functionalities and finally implementing it into my own projects.
Serialization can have many applications from sending a human to Mars to sending a text message to a friend. This is an idea that can be used my many developers across the world. proto-query will become much relevant wether it is this project or someone else's -heck! maybe there already is a proto-query that I don't know about. That being said... here is my CUhackit 2021 submission.
Log in or sign up for Devpost to join the conversation.