Data, Data, Data

How to build cool things, legally!

One of the first questions we always get asked when we say we are building a generative AI music tool is, ‘So, where are you stealing your data from?’… Although it seems like common sense in this day in age, people still don’t understand the importance of squeaky-clean data. Along with the rise in major lawsuits between the likes of Suno, Udio & co, and all the major record labels, this topic has become a really fun one for all those concerned with the music industry and artist rights. The way we see it is very clear: if you create it, it’s yours! But, this is clearly not the way many other folks in the sector think. 👀

When we decided it was time to wrap up our market research and start getting our hands dirty, the first thing we decided to tackle was DATA. We knew this would be one of the biggest battles in our journey, but we also recognized the fight as an opportunity to differentiate ourselves from other tools. If we could actually take the time to curate the perfect dataset, a dataset that only we have, we could train models that no one else in the world could replicate. So that is exactly what we did. We decided our first model should be a kick drum generation tool, aptly titled ‘Just 4 Kicks’, and we got straight to work.

Our first few pieces of training data - live recordings trimmed in Bitwig.

The following weeks consisted of long nights glued to drum machines and computer screens. I started with all the hardware I and my buddies own. I would connect a machine to my interface, set a very simple 4x4 kick loop, and start recording. By fiddling with different modulations, effects, and settings, I was able to crank out a solid 1,000 kick drums. Going directly from gear to DAW, I trimmed up each sample, then applied another series of effects and transformations on the sounds to quadruple my data set. Already at that moment, I realized I was sitting on our treasure chest.

I then performed the same tasks with the drum plugins I own which allow for secondary data use. Again I created, trimmed, and transformed the samples to end up with another 2,000 kick drums. But, my trusted friend/machine learning wizard, Henning, reminded me that we needed much more than 6,000 data points to get the ball rolling. So, drudgingly, I took to the internet. For the last push of kick drum scavenging, I dug through all the open-source sound libraries that exist in a quest for high-quality, copyright-free sounds. I must have listened to around 15,000 kick drums at this point until I called it quits, took what I found into my DAW, and began the trimming and transformation process.

All-in-all, this whole experience shaved off some days, but I have 0 regrets. Now that we have used this dataset of over 9,000 kicks to train our first end-to-end ML workflows, I am reminded of the importance of data and why so many people take shortcuts. That being said, if you are serious about what you are building and want to build something that is not only truly useful, but also fair, ethical, and legal, treating your data with love and respect should always be step number 1.

Rant over. Thanks again for listening to these words and giving me an outlet that’s cheaper than my therapist :)

As always — feel free to get in touch!

- Max (max@just-noise.com)