AbleGen | Devpost

Inspiration

Music creation stands at a historic inflection point as generative AI rapidly transforms what's possible. Despite new tools starting to emerge, their functionality and editability are still largely restrained, with limited integration into professional production workflows.

AbleGen was inspired by the lack of built-in generative capabilities in current production environments such as Ableton Live. We wanted to create an easy way for producers to inject generative musical ideas into their projects.

Our vision is a future where anyone can go from musical inspiration to production with ease. Anyone can create professional music using natural language, or examples, or humming/tapping. We are the bridge between them so anyone can become a producer.

What it does

AbleGen is an inspirational generative audio tool powered by AI, natively integrated to any production environment (e.g like Ableton Live) with the goal of opening up music creation to all by removing technical barriers. We enable producers to use generative music as part of their production process (natively integrated AI-generated music) into their process.

How we built it

A generative music plugin that creates original melodies, harmonies, beats and musical ideas using AI technology. Natively integrated into the Ableton Live digital audio workstation. Generates wav files within Ableton that can be directly used in the music production process based on natural language input MVP is built on MusicGen, but future iterations could connect to any natural language music LM on the backend

Challenges we ran into

Max for Live is a difficult setup to work in, we had to learn how to use Max and create the tool within Ableton Live. We also needed to all ramp up on Ableton Live to understand where it would fall within the workflow. We ran into multiple challenges with bandwidth and loading for MusicGen. We also ran into a bug setting up MusicGen on a HuggingFace space that took significant time to work through. Finally, MusicGen is particularly hard to work with, often writing music in ways that are not ideal for production (e.g. starting off the audio on off-beats, ignoring any commands about key signatures or chords, creating Mp4 files instead of Mp3 files, etc). The MusicGen is made as a toy/proof of concept right now and is not optimized for production workflows.

Accomplishments that we're proud of

We’re proud that we figured out the tooling for Live and got a working product.

What we learned

There is a current dichotomy in the current production environment. 1) Issues with MusicGen: Requires technical skills and knowledge to use, not user-friendly for producers Extremely limited editability - MusicGen provides no options for users to directly edit, refine, modify, or fine-tune the generated music. Black box output - There is uncertainty around what types of musical outputs might be produced without heavy prompting. Limited integration - MusicGen hasn’t been integrated into production workflows. The generated music requires exporting for use in production Small dataset - MusicGen was trained on a dataset that is smaller than what Meta can access. This constrains the diversity and quality of output. 2) Issues with Ableton Steep learning curve - The workflow and interface has a reputation for not being the most intuitive, especially for beginners. There is a learning curve to become fluent. No built-in generative capabilities - Unlike some DAWs, Ableton lacks tools for procedural/algorithmic music and sound generation. Limited built-in sounds/instruments - It lacks the huge virtual instrument libraries that come with other DAWs like Logic Pro or FL Studio. Users often need to buy third-party VSTs. Closed ecosystem - Project files use a proprietary format that doesn't translate well to other DAWs. Collaboration can be limited.

What's next for AbleGen

Phase I: A natively-integrated generative audio plugin for Ableton [this hackathon] Phase II: Generative audio with MiDi conversion that allow users to directly edit generated music Phase III: AI-Native DAW and tooling for all users to easily be able to make music, regardless of ability