Why are some of our grants giving CryptographicException ?

[aaronclawrence]

Sometimes, when merely listing grants for a user (GetAllUserGrantsAsync) we get an exception logged "System.Security.Cryptography.CryptographicException: The payload was invalid. ..."

We are currently on 6.2.5, and I'm aware of DuendeSoftware/products#1479 which was fixed in 6.3.8.

But my question is: what can we do about these failing decryptions on some existing grant? There is no information in the message to know WHICH grant is failing. Is there something that will identify them, or purge them?

For context, we upgraded from static signing key to dynamic keys in Sept 2023, but most of those grants are long gone. We are seeing it with current automated users that have grants only in 2025.

[maartenba]

@aaronclawrence this could be caused by misconfigured ASP.NET Core Data Protection.

Can you check the docs at https://docs.duendesoftware.com/identityserver/deployment#aspnet-core-data-protection, and verify a couple of things?

• Data protection keys are shared for all nodes of your deployment (if you have multiple servers)
• An application name is set (we'd recommend to set it to be the same as the default used when not configured, which is the application content root - this also has to match on all nodes)
• If keys are separate on different nodes, you can copy the key material to a shared location like blob storage or the database, and use that on all nodes

You could also try to decrypt the entries in the grants table with some custom code, and see which ones are failing.

[aaronclawrence]

I guess I can write some code to try decoding them, but I was hoping there was some way built into IS. Perhaps the error messages could include information like an ID to track down which grant could not be decrpyted.

[aaronclawrence]

I think I can do it by date now that I know when it happened (.net 8 deployment)

[maartenba]

We don't have anything out of the box here (have discussed a small tool at some point but it lives in our heads and not in code I'm afraid).

The change in .NET 6 was that the application name defaults to the content root, and the trailing slash was added/removed to that default.

If you still have the ability to run and deploy the old version in the same way it was done back then, you could check the application name e.g. in Startup:

// ...

var app = builder.Build();

// ...
var discriminator = app.Services.GetRequiredService<IOptions<DataProtectionOptions>>() .Value.ApplicationDiscriminator;

// discriminator is now the (auto-generated?) application name

Using that, you could perhaps unprotect the grants from that era and even build up data to migrate. Note you can also use this code in the .NET 8 version to get the current application name, and then set it explicitly going forward :-)

Now, since you know the date this happened (.NET 8 upgrade), perhaps another consideration could be to drop the records created before that date and expect users to renew their consents?

[aaronclawrence]

Hi Maarten, yes, I can just delete the records before then. I guess they are not working anyway, right?
But before I go and do that I would like to be sure that this is the problem and that I don't delete working consents/tokens.

With the existing error logging in Identity Server it seems impossible to figure out what grants are invalid. Some small enhancements to the log messages would be helpful, to include something that identifies the grant row it is trying to unprotect.

[maartenba]

That logging is not there currently. Would you mind chiming in on https://github.com/orgs/DuendeSoftware/discussions/152 ?