Greenboard Audio

A music livestreaming platform for musicians that solves the lag problem and audio storage tool

Comment 3

Inspiration

During the coronavirus pandemic, schools were shut down and music programs were struggling because an essential part is being able to play music together, rather than individually. While online communication apps such as Microsoft Teams, Google Meet, and Zoom were employed, one cannot use these to play music because of the "lag" problem: because the calls have a delay, the music will not sync up over a call.

I, Varun Singh, devised a solution which I call Recursive Live Streaming. Recursive Live Streaming allows musicians - including rock n' roll fans! - to play live music from miles away.

Theory

Currently, lag is not accounted for in a live stream. In our solution, we came up with a way to account for a specific lag time. We stagger the starting time of each musician by the specific lag time. For example, we can arbitrarily set the lag time between what is being recorded by a camera and what is being streamed to 10 seconds. We instruct Musician 1 to start playing and streaming. Musician 2 watches Musician 1's live stream and starts playing when they perceive Musician 1 has started (i.e. 10 seconds later). In the meantime, Musician 1 has already played the first 10 seconds of their song while Musician 2 is just getting started. Musician 2 streams them playing as well as the stream they are watching. Musician 2's stream has them synced up with Musician 1 perfectly. Musician 3 watches Musician 2's stream and starts playing when he perceives that the other 2 have started playing. In reality, Musician 3 starts playing when Musician 1 reached the 20-second mark and Musician 2 reaches the 10-second mark. This pattern continues through the ensemble. We both verified this solution before and during the project. It works with up to 5 people using YouTube Live. Unfortunately, because communication apps are made to minimize delay, they do not work for Recursive Live Streaming. Therefore, Recursive Live Streaming requires every performer to download an encoding software (such as OBS), use headphones or ear buds with a mic, create a YouTube channel, and learn how to use YouTube Live.

What it does

Greenboard Audio Mobile is an app that makes Recursive Live Streaming easy and accessible by abstracting out all of the configurations to the code, using the YouTube Streaming API. You can also do more simple functions like record, upload and organize audio files. Currently, Flytrap Mobile lets you browse, but it will eventually be full-featured with editing, collaborating, uploading, and recording.

How we built it

We used Flutter for the front-end and Django for the back-end. The Python back-end acts as an API between the front-end and the SQLite database. It is in charge of creating performance rooms, placing the performers in order and assigning each performer their correct streaming link. The back-end was also responsible for creating and maintaining each stream by interacting with the YouTube Live Streaming API.

Challenges we ran into

The first major roadblock was the fact that YouTube Stream API does not provide adequate documentation for sending audio or video data. We spend most of the time researching this API, and we found that this application needed to be registered and approved by Google to obtain an API key + an OAuth 2.0 key. For this reason, we could not fully implement the back-end and have not yet published it.

Accomplishments that we're proud of

We're proud of just the two of us building this app, which could be revolutionary for the music industry! Figuring out the usage of a complicated API with no previous knowledge or experience in a short amount of time was really hard and we are happy we did it. My teammate created an exceptional user interface which will entice many musicians and artists of all sorts to our application.

What we learned

Krish: I learned how to use the YouTube Live Streaming API. I also learned how to create efficient and clean server-side code using Django while implementing and maintaining complex streaming structures.

Varun: I created the front end with Flutter. I hadn't used Flutter in a while and learned about some new widgets that existed in Flutter and acclimated myself with the Flutter environment again.

What's next for Greenboard Audio Mobile

We intend to polish it up by adding many new features. Once we finish implementing the back-end so that it interacts with the YouTube Live Streaming API, we will deploy the back-end and release it into beta-testing and eventually production. We also want to check out a tool like https://ffmpeg.org/ that encodes the stream in order to make the live streaming feature possible.

We also want to allow users to upload normal audio files so that their streams and audio files can be all in one place with Flytrap.

Built With

Share this project:

Updates