-
This is the visual representation of music that we created. It can be viewed live at noteroll.com.
Inspiration
A combination of a past in music and a present interest in artificial intelligence and machine learning brought us to want to explore genetic algorithms to generate music. What's cooler than mixing together our two biggest passions?
What it does
Midi files are abstracted and flattened into a form in which a powerful, light-weight machine learning algorithm known as "augmented n-gram modeling" can analyze and compare multiple songs. The ability to compare songs allows us to use this model as a test of fitness for a genetic algorithm, which evolves slowly towards being a "legitimate" song of similar style to the training data (from where we got the model).
Just as important as creating the song, the presentation of data can allow new music students to learn more effectively. There is no more accessible platform than a simple web browser on the internet, and so we have chosen a warm, friendly design to display online that can read and play midi files, including ones generated by the A.I.
How we built it
This project was multi-faceted and has four main components: Hardware: 20 odroids are networked together to allow for parallel processing to expedite the machine learning process. This took time to make disk images for each mini-computer, as well as a lot of configuring just to physically connect all the wires. On top of this, special scripts were written to distribute computational processes efficiently amongst all the odroids.
Data cleansing: Open source, publicly available midi files were chosen due to their discrete nature. Symbolic music is much easier to manipulate than wave forms, but there is still a lot of data to parse. So, after much brainstorming, we came up with an algorithm to flatten and parse the midi files until they were in a very manageable state.
Machine learning: A two-step process, the machine learning aspect first developed a test of fitness using an augmented n-gram, and then used this test of fitness for a genetic algorithm to compare against. Tricky calculations allow for the genetic algorithm to mutate, evolve, and eventually converge to a song.
Visualization and application: Once a song has been generated, it needs to be shared or learned. A web interface was developed that can read midi files, show their musical representation, and actually play the music as well. A piano was chosen as the instrument due to its efficacy for teaching music to people fo all ages. The keys on the piano light up in anticipation of upcoming notes, and quickly flash exactly in time with the music, providing an intuitive user-experience. This interactive representation of music could potentially be more effective than traditional sheet music for those who are more visually inclined.
Challenges we ran into
With so many moving parts and interlaced layers, it was very challenging to get everything to "wire together" properly. Disk images took time to set up. Data collection and cleansing eats time as well. And one of the downfalls of coding in multiple languages among multiple people was getting these different languages to interface cleanly and quickly with one another.
Algorithm design was also a big hurdle, since before code can come to screen, a clear idea of the goal and path to it is needed.
Accomplishments that we're proud of
Out of nowhere, a bunch of code was written that worked. Data was clean. Analyses were made. The server rack that holds the odroids is well-designed and is both visually-appealing and highly functional.
What we learned
How to work as a team and get code to work amongst several people. A lot of technical tricks regarding specific programming languages. Cluster set up. Machine learning algorithm design (n-gram, genetic algorithm). What it really means to work nearly nonstop and under pressure. That this is definitely the right line of work for us.
What's next for ScoreSight
We plan to continue our work with this algorithm and approach to music analysis. If music can be represented in more efficient and better ways, more people will be able to learn and study music. In addition to this, as the quality of music generation increases, more and more interesting combinations of new songs can be written by A.I., and this can be shared by friends to others in a future social networking platform.
Log in or sign up for Devpost to join the conversation.