What it does
Magenta is a research project, released as open source, that looks at how machine learning can serve creative work, with most of its featured projects centred on music. Its newest item is Magenta RealTime 2, described as an open model and real-time synthesis engine for building and playing AI musical instruments on a laptop. The site features several projects. Magenta Studio is a set of music plugins for Ableton Live that can also be downloaded as standalone apps. DDSP-VST is a DAW plugin that turns the sound of one instrument into another live and can be played with MIDI. The Infinite Crate wraps the Lyria RealTime API in a DAW plugin and is open sourced under the Apache 2.0 license. Older work includes Onsets and Frames for transcribing piano, MusicVAE and NSynth. Developers can use the Python library, which is built on TensorFlow and covers preparing music and image data, training models and generating new content, or Magenta.js, which runs pre-trained models in the browser through TensorFlow.js. The site also links to demos, a blog, research, talks and a community page.
Capabilities
- Magenta RealTime 2, an open model for real-time music on a laptop
- Magenta Studio plugins for Ableton Live, also available as standalone apps
- DDSP-VST plugin that morphs one instrument into another in real time
- The Infinite Crate plugin, which uses the Lyria RealTime API and is open sourced under Apache 2.0
- Python library for preparing data, training models and generating content
- Magenta.js for running pre-trained models in the browser
When to use it
- Building or playing AI-driven instruments and real-time music tools
- Working with machine-learning music models inside Ableton Live or a DAW
- Prototyping creative machine learning in Python or in the browser
