Early warning when getting throttled #1812
Unanswered
AndersRask
asked this question in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
We are using Invoke-PnPSiteTemplate to apply templates to a lot of site collections preparing them for a migration.
We use client id for authentication, but even so with quite few threads we experience throttling.
Throttling was confirmed through premier support, because what we saw was... nothing... just no replies or anything. All cmdlets against the tenant that utilized the CSOM endpoints just stopped responding from one moment to the next.
I know of the implementation of retry logic when getting throttled at HTTP level that @jansenbe and @KoenZomers mention here
pnp/pnpframework#478
But my question is, if there is a way using the cmdlets to detect that you are getting throttled or about to get throttled?
We did not receive any warnings, the connection just stopped working. My guess is that the throttling did not get resolved within the 10 retries, or maybe that the provisioning logic spawns enough threads to get the client id "banned" for a period of time (note that an intraactive connection still worked fine, but when connecting with client id even cmdlets like Get-PnPWeb would just "hang" until the ban was released on the endpoints for the client id.
It would be helpfull if we either standard, or using an environment variable or parameter could get warnings that we were getting throttled. I also seem to remember from earlier support tickets that a header information is sent with each response when you are about to get throttled (not Retry-After). This header will count up until you reach throttling level. I know that some of the migration tools that use CSOM read this header and scale up/down based on this number.
If this is still the case, it would make sense to implement that this header was part of an "early warning" system, so that you would get warnings that you were about to get throttled and should slow down (or even that the framework itself slowed down) so that you would not end up with a completely blocked endpoint.
Any thoughts on this Bert or others? :)
kind regards
Anders Rask
Beta Was this translation helpful? Give feedback.
All reactions