Add capability to separate instruments (sax,trumpet,piano) in addition to voice in Audacity music separation capability

[chuckamata]

Adding the ability to separate selected instrument(s) into a separate track in addition to vocals would be a great new feature. Suggest Sax, trumpet, and piano as a good starting point. Ultimately, it would be nice to be able to select from a range of instruments including guitar, bass, drums, string instruments, piano, organ, strings, etc. with a total of 8 selectable separations. This would provide a unique capability that is not available anywhere else on the market currently.

[Brodvd]

There's LALAL.IA doing something similar online, it’s not free but if you add a tool like this, audacity becomes much more flexible than the limitations that it has now. Unfortunately you have to find the right service to connect to openVINO.

[RyanMetcalfeInt8]

It's a great idea! If there is an open source implementation for this kind of feature, then this would be great-- our team (developers / maintainers of this project) aren't really training any new models, we leverage existing models / feature implementations to work inside of Audacity.

Anyway, if anyone has any leads, let us know.

[Brodvd]

I don’t know great models or projects, looking around most of the open source models for musical separation are similar to what has audacity, have plus only the piano, but I found Audio Sep on GitHub, if it was executable seems a nice model.
https://github.com/Audio-AGI/AudioSep

[Brodvd]

@RyanMetcalfeInt8 I tried Audio Sep on Huggin Face, there are numerous bugs rather strange, however should work with 32KHz audio and with outstanding results. I’m going to give you an example (the video is the audio produce by Audio Sep):

Produce.mp4

[TEOL2023]

Plenty of 16-Bit video game players has checkbox based independant channel listening now so I think it is possible to program that sort of thing into a model for spitting real music properly most definitely.

Maybe even turning up the max of how many splits you can make within the rules of the existing model from 4 - 64 might be able to get us a better split from what we have as the splitter model is already programmed to work all it requires is someone to change values of how many splits we can make to 64 max and maybe the same model will learn how to split up to 64 instruments on the model we have.

[MarcoRavich]

To my knowledge the most popular is @sigsep's Open-Unmix (check out SiSEC 2018 results), but there are many exiting new approaces too (@merlresearch's Hyperbolic Audio Source Separation is just an example).

My resources collection is always there:

HyMPS project \ AUDIO \ AI-Based \ Separating

Hope that helps !

[Brodvd]

Also this probably can be a good choise: https://github.com/mainak0907/Wave-UNet---Audio-Source-Separation-

[MarcoRavich]

Added to HyMPS !