Making music with AI
Disclaimer: please note that this quickstart does not deal with the question of whether you SHOULD be using generative AI in your processes. These quickstarts are part of our own sessions in which we make this an integral part of the discussion and process.
AI & Music
Artificial Intelligence (AI) has been playing a significant role in the way music is created, distributed,
and listened to for several years. Consider, for example, Spotify's 'you might also like' recommendations.
These apps usually utilize AI algorithms. Furthermore, tools have existed for some time that can isolate
the 'stems' (instruments and vocals) within a song.
Relatively new are generative AI tools trained to create something new. Think of ChatGPT for text,
Stable Diffusion for images, Runway for video, and Suno for music. 'Creating something new' is relative
here: the software is usually particularly good at creating new variations based on all the material it
has been trained on previously.
Whether all this is morally and legally permissible and desirable is a discussion in itself. However, for
musicians, the developments also offer interesting opportunities for new forms of musical creative
processes and music education. Consider experimenting with merging styles or creating a cover of your
own song with a different voice.
Video made by Sander Huiberts that showcases various tools: https://nextcloud.hku.nl/s/WwQGndyw7LTtrKt
What role can AI play in musical creative processes?
You'll find that professional creators almost always choose to collaborate with AI for only part of the
creative process (except for a few AI enthusiasts). Within this, there are many choices. A few
examples:
- De Staat wrote, sang, and played the song for 'Running backwards into the future' themselves but
created the 'dreamy' music video in collaboration with Stable Diffusion Deforum.
- The AI song 'Heart on my sleeve' used voices from Drake and The Weeknd, but the lyrics and vocal
input were done by TikTokker Ghostwriter977 himself.
- SETUP and Pure Ellende experimented in 2021 with an AI-generated ‘smartlap’ (folk song), but the
lyrics were only half generated by GPT-2 (alternating one line by the AI and one by a human).
In recent months, we also see tools appearing that generate tracks without needing a text, voice, or
sample as input. Stable Audio does this for (electronic) music without (understandable) vocals. Suno
gives you the choice to write or generate lyrics and then combines vocals and instruments in a style of
your choice. It still sounds a bit crackly, but such a song might inspire a new melody, a piece of text, or
a combination of music styles.
Want to experiment yourself?
- Suno.ai. Generate songtext and music including vocals. (Temporary) free account needed.
- StableAudio.com. Generate music without vocals. Free account required.
- Vocalremover.org. Separate vocals and instruments from existing audio files.
- Voicify.ai. Voice changer. Paid plan only.