Skip to content

Get/Return pooled connections #3404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from

Conversation

mdaigle
Copy link
Contributor

@mdaigle mdaigle commented Jun 9, 2025

Description

This PR adds Get and Return functionality to the new ChannelDbConnectionPool class.
A channel and corresponding channel reader and writer are added to underpin these operations.
An array is added to hold references to all connections managed by the pool.
Logic is included to respect max pool size in all cases.

Not included: pool warm up (including respecting min pool size), pool pruning, transactions, tracing/logs. These will come in follow-up PRs.

Also includes extensive unit test coverage for the implemented functionality. Integration testing is reserved for a later state when more of the pool functionality is available.

Issues

#3356

Testing

Tests focus on the Get and Return flows with special focus on the following:

  • async behavior
  • max pool size respected
  • connections successfully reused
  • requests successfully queued and queue order respected
  • stress testing with many parallel operations

@mdaigle mdaigle force-pushed the dev/mdaigle/get-return-pooled-connections branch from 9d95355 to 73a379b Compare June 9, 2025 20:45
@@ -97,6 +97,8 @@ internal DbConnectionInternal(ConnectionState state, bool hidePassword, bool all

#region Properties

internal DateTime CreateTime => _createTime;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exposes creation time for load balancing and max connection lifetime enforcement.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why to make getter and setter internal ?

It looks like this class DbConnectionInternal is managing its creation time, and this value isn’t exposed to derived classes.

If there’s any chance that this responsibility might shift in the future, or if derived classes may need to update the creation time, I recommend adding a protected method (e.g., UpdateCreationTime(DateTime newTime)) rather than exposing a protected setter or property.

My reasoning is:

Updating creation time should be an explicit, intentional action, not just a simple property assignment. An explicit method makes this clear in the API and signals that it’s a special operation as most of the logic is still in the base.

Let me know your thoughts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Other classes shouldn't be updating this property. This syntax will only expose a getter for the property: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/properties#expression-body-definitions

@mdaigle mdaigle changed the title Dev/mdaigle/get return pooled connections Get/Return pooled connections Jun 9, 2025
@mdaigle mdaigle marked this pull request as ready for review June 9, 2025 23:25
@mdaigle mdaigle requested a review from a team as a code owner June 9, 2025 23:25
@mdaigle
Copy link
Contributor Author

mdaigle commented Jun 10, 2025

@roji tagging you here if you have any time to review. Thanks!

Copy link
Contributor

@benrr101 benrr101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, it looks good. A bunch of things I'd like addressed, but you know I'll accept valid arguments against fixing them :)

Copy link
Contributor

@paulmedynski paulmedynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the ChannelDbConnectionPool changes. I will look at the tests once we have converged on an implementation. It might be a good idea to host another meeting to walk through the commentary here.

for (var numConnections = _numConnections; numConnections < MaxPoolSize; numConnections = _numConnections)
{
// Note that we purposefully don't use SpinWait for this: https://github.com/dotnet/coreclr/pull/21437
if (Interlocked.CompareExchange(ref _numConnections, numConnections + 1, numConnections) != numConnections)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is complicated enough to warrant an English explanation of what's happening.


return Task.FromResult<DbConnectionInternal?>(newConnection);
}
catch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catch only what is documented to be thrown.

/// </summary>
private static int _instanceCount;

private readonly int _instanceID = Interlocked.Increment(ref _instanceCount);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interlocked.Increment(ref _instanceCount);

does it matter if it becomes negative ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it starts at zero and only ever goes up

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counters are naturally unsigned, so can we use uint or ulong instead? I'd be proud if those ever wrapped :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should this be _instanceId ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no Interlocked.Increment() overload for uint. It doesn't matter if it goes negative. It's only used for identification of unique pools in trace messages.

@@ -20,52 +25,136 @@ namespace Microsoft.Data.SqlClient.ConnectionPool
/// </summary>
internal sealed class ChannelDbConnectionPool : IDbConnectionPool

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelDbConnectionPool

it might be good to add some documentation on how this class works and how this is different than the WaitHandleDbConnectionPool.

Copy link
Contributor

@benrr101 benrr101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, needs another round... But the other first round was pretty good!

/// </summary>
internal DateTime CreateTime
{
get;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: these can/should be on the same line

/// </summary>
private static int _instanceCount;

private readonly int _instanceID = Interlocked.Increment(ref _instanceCount);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it starts at zero and only ever goes up

/// <inheritdoc />
public DbConnectionPoolIdentity Identity
{
get;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I rescind my Nit designation - let's get these on the same line

}

/*
* This is ugly, but async anti-patterns above and below us in the stack necessitate a fresh task to be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think your indentation got messed up here

}

/// <inheritdoc/>
internal Task<DbConnectionInternal?> OpenNewInternalConnection(DbConnection? owningConnection, DbConnectionOptions userOptions, TimeSpan timeout, bool async, CancellationToken cancellationToken)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please chomp this biggie of a line :)

@@ -389,7 +500,7 @@ private async Task<DbConnectionInternal> GetInternalConnection(DbConnection owni
try
{
// Continue looping around until we create/retrieve a connection or the timeout expires.
while (true)
while (true && !finalToken.IsCancellationRequested)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.... true is completely redundant now, lol

I'd rather have a while(true) than while(!finalToken.IsCancellationRequested). At least I know intuitively that while(true) implies we'll break out somewhere inside. The latter implies we run this code until timeout... And the combination is just, well redundant 😂

It would be nice if there was a way to have the condition of the while be while(connection is null && finalToken.IsCancellationRequested). Which I think is what we were working towards by suggesting a do/while loop - basically make at least one attempt to get a connection, and loop around if we haven't. I'd suggest then making it like:

do {
  finalToken.ThrowIfCancelled();

  // do stuff to set connection
} while(connection is null);

return connection;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the latest refactor should help with this. coming soon!

private readonly ChannelWriter<DbConnectionInternal?> _idleConnectionWriter;

// Counts the total number of open connections tracked by the pool.
private volatile int _numConnections;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will this be different than the size of _connections array?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This counts the number of slots in the array that have actually been reserved. It starts at 0 and increments each time a connection is added to the pool for tracking. It's decremented every time a connection is destroyed and removed from the pool. The only time it will equal the size of the _connections array is when we're at max pool size.

// We enforce Max Pool Size, so no need to create a bounded channel (which is less efficient)
// On the consuming side, we have the multiplexing write loop but also non-multiplexing Rents
// On the producing side, we have connections being released back into the pool (both multiplexing and not)
var idleChannel = Channel.CreateUnbounded<DbConnectionInternal?>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the efficiency argument but have we compared the difference in performance for our specific use case implementation for Bounded vs Unbounded Channel. Bounded channel can offload the part of us worrying about Max/Min poolsize, flooding and starvation in exchange for some throughput loss.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With a bounded channel alone, we won't respect max pool size. The channel only stores connections that are "idle" (sitting unused in the pool). Connections that are in active use are not represented in the channel. So, if we used the bounded size of the channel to enforce max pool size, we wouldn't take into account any connections that are in active use.

/// Tracks all connections currently managed by this pool, whether idle or busy.
/// Only updated rarely - when physical connections are opened/closed - but is read in perf-sensitive contexts.
/// </summary>
private readonly DbConnectionInternal?[] _connections;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in essence _numConnections is doing the reservation part for us. Does the array itself provide any thing that Concurrent Bag/Dictionary/Queue will not provide. SemaphoreSlim does look seem to be the lightest, thread-safe solution here.

Copy link
Contributor

@paulmedynski paulmedynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new round of comments, questions, and suggestions :)

/// </summary>
private static int _instanceCount;

private readonly int _instanceID = Interlocked.Increment(ref _instanceCount);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counters are naturally unsigned, so can we use uint or ulong instead? I'd be proud if those ever wrapped :)

PoolGroupOptions = connectionPoolGroup.PoolGroupOptions;
ProviderInfo = connectionPoolProviderInfo;
Identity = identity;
AuthenticationContexts = new();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's our take on initializing fields/properties that don't depend on a constructor body? Do we prefer to do it in each constructor, or do it on the field/property declaration? I prefer the latter. We've got a mix in this class so far (see _instanceID line 40).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really have a preference here. Maybe a slight preference towards doing it in the constructor for instance variables, doing it outside the constructor (where possible) for statics. But I think I have more of a preference for not having a preference :)

/// </summary>
private static int _instanceCount;

private readonly int _instanceID = Interlocked.Increment(ref _instanceCount);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should this be _instanceId ?

@@ -6,7 +6,6 @@ namespace Microsoft.Data.SqlClient.ConnectionPool
{
internal enum DbConnectionPoolState
{
Initializing,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was only used inside the constructor while fields were being initialized. You can't get a reference to a pool until the constructor completes, so we don't need this state.

@@ -81,7 +83,7 @@ internal interface IDbConnectionPool
/// <summary>
/// The current state of the connection pool.
/// </summary>
DbConnectionPoolState State { get; set; }
DbConnectionPoolState State { get; }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

State should only be modifiable by the pool itself.

@@ -76,6 +77,7 @@
<dependency id="System.Configuration.ConfigurationManager" version="9.0.4" exclude="Compile" />
<dependency id="System.Security.Cryptography.Pkcs" version="9.0.4" />
<dependency id="System.Text.Json" version="9.0.5" />
<dependency id="System.Threading.Channels" version="8.0.0" />
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The System.Threading.Channels namespace isn't included in the netstandard API, but the package provided System.Threading.Channels targets netstandard2.0

async: true,
timeout
).ConfigureAwait(false);
taskCompletionSource.SetResult(connection);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason to not use TrySetResult and TrySetException to avoid unnecessary exceptions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea, done!

// We're potentially on a new thread, so we need to properly set the ambient transaction.
// We rely on the caller to capture the ambient transaction in the TaskCompletionSource's AsyncState
// so that we can access it here. Read: area for improvement.
ADP.SetCurrentTransaction(taskCompletionSource.Task.AsyncState as Transaction);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the current source code of WaitHandleDbConnectionPool, it seems like everything between ADP.SetCurrentTransaction and taskCompletionSource.SetResult is synchronous, unlike here, where GetInternalConnection can go async. I wonder whether it's to make the default TransactionScope work (you have to pass a parameter to support async) with OpenAsync, although it's kinda weird to do so anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is an important point. I haven't included transaction support in the scope of work for this PR and included this line prematurely. This line is incorrect for the reason you stated. When we go down the async path, we may end up creating/retrieving a connection in a continuation and that continuation would have no reference to the ambient transaction.

This does make me wonder whether we need any additional hygiene measures to make sure we reset ambient transaction state on threads. We wouldn't want a managed thread to hold onto the ambient transaction state and incorrectly enlist other connections.

… connection disposal when opening new connection.
Copy link
Contributor

@paulmedynski paulmedynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're getting closer!

/// <returns>Returns true if the connection is live and unexpired, otherwise returns false.</returns>
private bool CheckConnection(DbConnectionInternal connection)
{
if (connection == null)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactored this a bit so that it makes sense based on the nullability constraints and to remove the side effects from this method.

get;
private set;
}
internal DateTime CreateTime { get; init; }
Copy link
Contributor

@paulmedynski paulmedynski Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

init means "Must be assigned during construction", but doesn't restrict that assignment to a constructor body. A caller could do this:

DbConnectionInternal conn = new(...) { CreateTime = DateTime.UnixEpoch; }

The initializer body runs after the constructor body, but still during construction. This isn't what we want.

I think all you need is:

internal DateTime CreateTime { get; }

That will make CrateTime immutable - you will be forced to assign it in all constructor bodies, and then prohibited from assigning to it anywhere else.

I suspect this applies to all of your use of init in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I didn't realize the init properties could be set in that way.

{
/// <summary>
/// A thread-safe array that allows reservations.
/// A reservation *must* be made before adding a connection to the collection.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an array or a collection? Let's use the most precise term consistently.

/// A thread-safe array that allows reservations.
/// A reservation *must* be made before adding a connection to the collection.
/// Exceptions *must* be handled by the caller when trying to add connections
/// and the caller *must* release the reservation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you trying to explain that callers must release a reservation if adding a connection via that reservation fails?

I'll have to see how this works. Leaving a breadcrumb for myself...

/// try {
/// var connection = OpenConnection();
/// slots.Add(connection);
/// catch (InvalidOperationException ex)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing }.

}

if (!CheckConnection(connection))
// Calling PrePush prevents the object from being reclaimed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment - does this belong inside ValidatOwnershipAndSetPoolingState() ?

// We were able to get a connection, but the task was cancelled out from under us.
// This can happen if the caller's CancellationToken is cancelled while we're waiting for a connection.
// Check the success to avoid an unnecessary exception.
this.ReturnInternalConnection(connection, owningObject);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this. here?

// attempts on the idle connection channel.
if (newConnection is not null)
{
RemoveConnection(newConnection, trackedInSlots: false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems fragile to separate the reservation and un-reservation across methods.

Could we make a reservation a first-class object and use disposal to have it un-reserve itself? And then have a Reservation.Keep() method that you call if you want to keep it (this would just ignore disposal). In fact, maybe only ConnectionSlots.Add() is allowed to call Reservation.Keep(), then all the paperwork is done for you by IDisposable and ConnectionSlots.

There is usually a single "keep" path, and multiple "don't keep" paths, so its safer to let the compiler worry about the "don't keep" ones for you.

I prefer this failsafe-by-default approach when acquiring resources.

You could go one step further, and only allow Add()inga connection to its reserved slot via a method on Reservation. Then you avoid the problem where you can call Add() on ConnectionSlots without having made a reservation. And once it's impossible to Add without a Reservation, you can just always decrement _reservations when you Remove() a connection that you found in the slots.

Essentially, you're using the lifetime of a Reservation object to:
a) guarantee you can Add(), and
b) guarentee someone else can't add if there are no free slots.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I'll take a look at this and see what I can do. A failsafe API definitely sounds nicer, I just want to make sure we don't force extra allocations if we can avoid it. It's probably fine as long as we only allocate when opening and closing connections and not every time we get/return.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it wouldn't be a big allocation anyway

if (!found)
{
return;
if (trackedInSlots) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is this parameter worth it? Now I can make a mistake and pass trackedInSlots = false when the connection is indeed in a slot.

In the false case, isn't that an exceptional condition anyway (i.e. we couldn't do some part of the "make a new connection" flow)? Does it really matter if we spin through our slots just to find out it wasn't there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I agree. Like you mentioned, it's on an exception path anyway, so not as critical.

{
cancellationToken.ThrowIfCancellationRequested();
Debug.Assert(finalToken.IsCancellationRequested);
// We didn't find an idle connection, try to open a new one.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These ??= comments are a bit misleading. At line 466 we don't know whether or not we got an idle connection. We only know that if we hit the await part. Maybe this would be clearer as If we didn't find ..., then try to ....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants