Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM
How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM
I discovered how to train a custom diffusion model for kick drums using just an old Linux desktop with 6GB VRAM. By optimizing the workflow and leveraging efficient techniques, you can generate unique audio samples without needing expensive hardware or cloud services. This guide walks you through the process, proving that powerful generative AI is accessible to everyone.
You don't need a data center to create the next generation of AI; your old laptop is enough.
- larme
People who are interested in this application should check synplant[0]. It has a ML technology called "Genopatch" which gives you 2 functionality:
1. you can try to describe a sound with some tags and it will try to generate a sound to capture the feeling of these tags
2.you can feed it with a sound sample and it will try to re-synthesize the sound with its synth engine. Though the end result will usually be just a "re-imagined" version of your input sample.
My guess is the underlying model is not a "deep" model. The main benefit is that the end result is not a wave file, but a list of generated parameters that can be synthesized by the synthplant engine. And now it comes the interesting part: you can tweak these parameters to finetune the generated sound. These parameters have actual meanings (FM ratio, reverb etc.)
- thangalin
Slightly off-topic. Now that 1920s jazz music is falling into public domain, has anyone tried to reinvigorate the music using AI and generative adversarial approaches? Pre-1940s music didn't have high-fidelity sound, so the strong bass lines weren't captured. In theory, we could "downgrade" modern recordings to sound like 1920s recordings, then use adversarial techniques to train the machine on how to restore the antique recordings. Anyone know of any work being done in this area?
- BrandNewRetro
I have three (and pray I do not come up with any more) needs for AI audio apps.
One, a ubiquitous restoration model. Find degraded copies of music in the wild, old YouTube's, transcodes, vinyl rips, bad masters, half destroyed tapes... Pair them with modern pristine lossless encodes of the same music, train. Then use that model on music we don't have pristine copies of.
The second is similar but more specific. There are so many stems floating around from popular music. My idea is to compare individual stems against the results from MVS/Spleeter(same song, same instrument). This would surely stand a chance of pushing that tech forward, so we can treat the FFT artefact heavy sound of new efforts.
Thirdly, from a creative point of view, I wanna do the equivalent of image to image on my tracks... But I actually want it to hallucinate in the manner of the early deep dream images, I want to be able to play with that space..
I can knock out musak to spek in minutes already, gen music is just reducing low effort to nearly no effort, preventing people with needs from networking with creators.... Uhh.. but I think that's a very general issue with Gen AI away from the corporate/entrepreneurial dev space
- mk_stjames
This is cool, I had some questions because I was trying to understand this more - not the diffusion model training part, because that I've actually seen before- but the actual way you bundled this into a web application and the choices made...
Did you publish the actual trained model anywhere? I see how in the code there is python for how individual samples can be generated, but the model checkpoint pulldown comes from a directory that... I don't see.
I then went through the code of how this runs on the web and- I'm not a web dev guy- so I'm pretty confused at all the bits bolted together to make this into a web app. It seems like there is a WASM bit that is compiled from a typical C++ audio plugin that is doing the stuff like conv reverb and limiting and distortion in the web app - all that is oldschool, non-generative AI, DSP being applied to the samples. Then the samples are just... a few default generated samples, to start- where are they pulled from, physically? And you have a login requirement to spool up the actual generative AI part to generate new samples to run into the DSP (because that needs a GPU on the backend to do, so, a login to help rate limit this)
How big is the actual generative model? Did you ever think about building the generation engine into the WASM bundle, using maybe WebGPU in the WASM to accelerate in a platform agnostic way, so that the entire app would run offline in someone's browser window?
I'm having fun just playing with the kick progra […]
- robotswantdata
Confused. Why not just make the kick drum from a sine? Seconds