Custom pooling AsyncMethodBuilder

[NinoFloris]

I'm trying to write a custom pooling builder as the framework one does not fit the usage pattern. The method I'd like to improve the readability of has already been optimized with a fairly large pool of custom completions, static continuations a bit of careful async void for the unhappy paths to avoid as much the tedium that is a handwritten async method. It's not fun, it looks terrible and you need to be a TPL expert to do it right.

As the language feature for the per method AsyncMethodBuilder was explicitly created to allow extensibility and custom pooling behaviors it seemed like a good fit to try to use here.

However how exactly would someone write a builder that doesn't regress in terms of allocations compared to what the framework provides? Given all the IAsyncStateMachineBox infra is internal? Wasn't it a goal to allow non-runtime authors to write their own?

As it stands today I can't seem to create an alternative pooling implementation that is competitive without being able to use these pieces.

Thinking out loud, might static abstract methods have a place on a new parameterized pooling method builder? Users could then provide their own rent/return logic without the runtime having to expose IAsyncStateMachineBox and friends?

[ghost]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[NinoFloris]

@stephentoub I would very much appreciate your input on this issue.

[stephentoub]

Wasn't it a goal to allow non-runtime authors to write their own?

Yes, and they can. That doesn't mean it's a requirement that external code can do it as efficiency as code in corelib; no matter what APIs are exposed, there's invariably opportunity for code with access to all the internals to do it better. And the internals in question here are implementation details that grew organically out of optimizing it, and that have changed non-trivially over time; I'd be extremely hesitant to expose any of that... doing so would likely make some further improvements much more challenging if not impossible. There are also correctness/reliability concerns involved, error checking that's esched because the internals trust themselves, etc., and we'd likely need to make some things more expensive to expose things publicly.

might static abstract methods have a place on a new parameterized pooling method builder?

We spent a good deal of time exploring options around such parameterization in the .NET 6 timeframe, including with whether such static abstract methods would fit well, and we didn't come up with anything that worked well. If you're motivated to design and prototype something in corelib, so that we get an end-to-end picture of what a solution would be as well as its implications on the rest of the system, you're obviously welcome to do so. I don't personally plan on investing in that right now, though, and I'm not aware of anyone else who will be for the foreseeable future, either.

[NinoFloris]

I absolutely agree that IAsyncStateMachineBox and other TPL internals should not find their way to the public api surface. I've seen the evolution you and others have made there and locking that agility down by making all of it public api is the last think anybody should want.

If you're motivated to design and prototype something in corelib

If I would it will most likely be something that relies on static abstract methods handling the rent and return logic. It would then be up to implementers to figure out the best way to acquire a pool. As I understand from the github issues around at the time (in the csharplang? repo) figuring out a way to share state, ideally instance state, was a big factor in the design impasse.

In any hypothetical design I would need to leave this up to the implementers to handle, any improvement here would need compiler support to pass in the current instance reference or other such state. There are some - not terrible but neither very satisfactory - workarounds that can be used like thread statics to setup instance state before the actual async body + builder is called into. I could imagine scenarios where this could still be worth the additional time it would take to resolve a thread static just to remove those allocations.

Moving back to the main issue. I want to reiterate that in its current state any custom implementation trades one alloc for the one you're trying to remove and on top of doing so you add overhead to store and retrieve pooled instances. It's effectively never worth it. Seeing as the intended use of this feature is more than just being 'pubternal' compiler <-> runtime glue and given that authors predominantly opt for pooling to improve performance aspects of their code having this particular limitation exist at all means the feature is not fit for function in that broader context.

I understand very well this is a niche feature so I won't speculate on its popularity or any increase thereof after fixing this. Doing so however seems like a necessary and worthwhile effort to know anything more at all. I hope I can count on your support to get something through if I do bring a design forward!

[stephentoub]

the feature is not fit for function in that broader context

Most of the use cases were not about performance.

I hope I can count on your support to get something through if I do bring a design forward!

I can promise to look at it. I can't promise more than that.

[NinoFloris]

Most of the use cases were not about performance.

I'm legitimately curious, what use cases are they?

I can promise to look at it. I can't promise more than that.

Of course, that's all I ask :)

[stephentoub]

what use cases are they?

Several involved wrapping the existing builders to layer on additional functionality. For example, wanting to be able to instrument / log when various aspects of the async method lifecycle happened. Another wanted to be able to ensure completion happened by queueing to a specific scheduler. Another involved still wanting to use ValueTask but ensuring that all scheduling was routed through a specific set of types. Etc.