Looking for co-maintainers/developers + some yapping

[tesselode]

I'm not an audio expert! I've made an audio library that does some unique things and works well for my use cases, but there's areas where I think things could be better:

Performance

I have no idea if Kira's performance is good or not. On my i5-13600K, I can play somewhere between 500-1000 sounds at once before I get audio crackles (depending on the playback speed of the sounds). Is that good? Is that bad? It seems low to me, but I have no idea if it actually is.

I tried to benchmark OpenAL via love2d's audio module (from what I understand, love.audio is just a basic wrapper around OpenAL), but that didn't tell me much because OpenAL was capped at 128 sounds.

Kira might perform better if it used buffers more. Currently, all audio is processed one frame (stereo sample) at a time. From my understanding, that's pretty terrible for performance, since the CPU has to load every resource once per sample instead of once per, say, 500 samples. And it does seem like loading data is a bottleneck for Kira, since even slight changes in the size of the StaticSoundData struct noticeably impacts the benchmarks.

I did previously attempt to change Kira to use buffers, which you can see on the buffers branch, but I didn't go all the way into making everything use buffers. The performance improvements were...moderate - I could play about 15% more sounds without stuttering. That's cool, but I'm not sure it's worth it for the downsides of making everything buffered:

• It'll be more complicated to make sure modulation and clocks are sample-accurate
• I don't know how the heck to do such a sweeping refactor in a digestible way
  • And I don't want to rewrite Kira because that's even more work, and I'll lose a lot of knowledge built up in the codebase
• A lot of tests will have to be rewritten

That last point is the most discouraging for me - I really don't enjoy writing these tests. Which brings me to the next topic:

Testing

The unit tests for static and streaming sounds are pretty gnarly. They work on an individual frame level, and I'm not sure that's a good thing. They're very sensitive to tiny details that aren't really what I'm trying to test for. I was trying to refactor streaming sounds recently and I ended up with something taking one frame longer to happen, which caused a test to fail. Maybe it's good that the tests are that picky about timing? I'm not sure.

There's no testing for the effects at all. I don't know the right way to test that a low-pass filter does low-pass filtering. Maybe with some kind of audio snapshot test framework? Is that a thing in Rust? That would be cool.

In conclusion: the tests are a pain to change, and that discourages me from even attempting to change Kira to use buffers more.

Code quality

There's a lot of same behaviors done different ways between static and streaming sounds. Maybe that's OK, because they are different things. I've tried to make reusable utilities that are shared between the two kinds of sounds, but there's still some code duplication.

I did try to alleviate this by making a SoundWrapper type that handles behaviors common to static and streaming sounds, like pausing/resuming, volume changes, panning changes, etc. However, this made the developer-facing API a lot more complicated, and it negatively impacted performance for some reason, so I scrapped the idea.

Me having opinions

I don't add features unless I understand the use case, but I think I might be too closed-minded sometimes. Take a look at the issues and PRs to see how often I drag my feet on features that I don't personally care about at the time. I think as a maintainer, it's good to be discerning about what features you add to a library, but there's only so much perspective on real-world use cases one person can have. I could use another person who I trust to make good decisions to help with Kira.

Energy

I'm suspecting I don't have as much motivation to work on Kira as I did a year ago. Every day I work on Kira is a day I'm not finishing up the playtesting build of my game to get into the hands of players. As of the time of writing, I'm considering switching my game to use FMOD. But then I'm not dogfooding my own library.

I don't want to give up on the library! I really want there to be a good open-source game audio library with nice modulation and timing features for dynamic music and such! But at the time of writing, I'm scared that Kira isn't "good" and that I don't have the expertise or energy to make Kira "good", whatever that means.

In conclusion: please help?

• I don't know what the Heck I'm doing
• There's very intelligent and knowledgeable people in the Rust community
• I'd like some of them to help me with Kira
• I don't want to be the only person working on Kira forever

[miloder]

Regarding the use of a buffer, in my understanding, buffer size is about making a trade-off between latency and "scheduling guarantee". Basically, how sure can you be that within the duration of a frame (say 128 samples at 48kHz - which is about 2.67ms), the CPU will get around to scheduling and completing your frame-wise processing. If your effects chain is short and not very complex, you could definitely get around to processing each sample in time (edge case of frame size == 1 sample). However, if you were to pile on more stuff (dynamic EQs, multi-band compression, fancy frequency-domain effects, etc.), then you will eventually run out of time to process between one frame and the next (and cause drop-outs). What's more, some processes inherently require a non-unit frame size to function in a meaningful way (e.g. DFT). I guess you can also just insert the required buffers between processes. In any case, in my audio-developer opinion, handling different buffer sizes is crucial not necessarily for performance but for reliability. Hopefully this is helpful, I'm only a neophyte as a rustacean so I probably won't be of much help on the architecture side, but I love to see audio projects in the Rust ecosystem. Don't get discouraged, you're doing valuable work!

[orottier]

Hey @tesselode, I really appreciate your honest story here. I'm the maintainer of the web-audio-api crate which has some similarity to Kira. I don't have time to step into co-maintaining Kira but you might appreciate my ramblings here:

First of all, writing an audio library and gaining some popularity is no small feat. This actually makes you an intelligent and knowledgeable person in the Rust audio community.

The crate I'm maintaining is based on a well defined specification for web browsers, and it has a giant 3rd party test suite already, so I can't really relate to your hassles with testing and deciding to add features or not. You could get some inpiration from the test suite to see how browser vendors build these tests. Spoiler: it's sample accurate testing, with almost no margin of error. https://github.com/web-platform-tests/wpt/tree/master/webaudio/the-audio-api (it's written in JS, but they are useful for us because we provide JS bindings to our rust lib)

The Web Audio API is based on block processing of 128 frames at the time. This is good for performance, but it means implementing sample accurate scheduling is quite tricky. Our implementation can play around 4k sounds concurrently, but this is the fast path. When you change the playback rate or use sub-sample scheduling, we need to resort to more expensive code. 4k concurrent sounds is somewhat on par with the big browser vendors with their mature C++ implementations. You can find more info about the benchmarking here

One option you could consider -- but it will be a lot of work -- is to rework Kira as a game-oriented, quality of life, layer on top of another intermediate level audio engine. Right now you have built everything on top of cpal, which is very low level. You could build kira on top of rodio or web-audio-api which have convenient intermediate-level building blocks. Then the 'hard' DSP parts are no longer part of kira which should leave time and energy for new features. Of course, you'd have to evaluate if these intermediate libraries have all the building blocks you need. I’d be happy to help assess that for web-audio-api if you're interested. For example, your 'tweens' could be mapped to our 'audioparams'

[NeuralModder]

If we're talking about going the route of using another intermediate audio engine, then something smaller like awedio may also be of interest.

[Roms1383]

My own comment here won't weight much since I'm not experimented with audio at all, I literally learn as I use kira.

The only point that I want to stress is that I've tested a lot of options out there (including FMOD) and kira is literally the best one I could find (in terms of available features, and ease of use) so don't give up on it, you did an amazing job.

Having co-maintainer(s) would probably be beneficial since it would allow kira to iterate faster, and alleviate some of your burden. I'd love to step as one, but sadly my knowledge of audio is insufficient to be of any use.

I'd suggest to give some visibility to this discussion in This week in Rust, that would help reaching out to more potential assistance.

[olekspickle]

I am not audio professional, but I am a musician and do have some experience with Rust and a desire to get to a lower level audio for some time now.
I'd be glad to help out (starting with something easy of course hehe, maybe restructuring or smth)

[codewing]

They there @tesselode and everyone else!
I'm currently building an app to play audio for my pen and paper adventures and kira feels really nice to use as the audio manager in the background. I've also tried some others but it's really good!

So far the only thing that I would have wished for was a get_volume for my tracks on which I used the set_volume on that returns the value I have set before.
but not having that wasn't a dealbreaker because as you said in #92 I just tracked it myself.

maybe you want to set-up a messaging group on discord or somewhere else so people can chat and share their stuff and you see what people build with your lib :)

Thanks a lot so far!