PnP Powershell and throttling

[Tran-Kien]

We are using PnP Powershell and the Provisionning engine to enforce governance and control on the SharePoint and Teams sites that get created across the organization.

SharePoint sites, Teams and Groups are created and configured by Azure Automation PowerShell Runbooks (with the help of PnP.PowerShell vers 1.10.0 cmdlets). The runbooks are executed with a client app/certificate which has proper permissions on the tenant.

However, we are experiencing some very sporadic and random errors all over the place. Errors that would get resolved by simply retrying the runbook. Sometimes it would run fine, and sometimes it would give all sorts of errors and fail.

I've been thinking could throttling be causing us all these problems? It's especially troublesome when we try to provision multiple sites at the time. The concurrency had been manually reduced to 1 site at a time but it's still sluggish sometimes.

We would get random Access Denied errors, "Object reference not set to an instance of an object" errors, or "Cannot find an overload for Add and the argument count: 1" when doing Invoke-PnPQuery or Invoke-PnPSiteTemplate.

Anyone has some insights, ideas or tips that they can share?

Thanks

[AndersRask]

With no code and real errors it's hard to figure out exactly what is failing.
However, having done a ton of runbooks, a thing to remember is that they were initially made as PowerShell workflows, so they have retry logic built in. Hence it is important that they are idempotent, as they could restart half way during execution.

Related to this, as this is something that can trigger a restart, always make sure that code that returns something is either assigned to variables or piped to out-null (especially when calling PnP methods on objects such as Add() and similar

Release connections when used.

Use proper error handling and pipe to relevant streams (verbose, warning, error, output). Setting up diagnostics will enable sending logs to log analytics and Azure monitor where alerts can be configured

Even though throttling should be handled in PnP core I still have response http error checks and retry logic for dropped connections

Hope this helps
Regards Anders

[Tran-Kien]

Thanks for your reply.

Yes I'm aware of what you mentioned in regards to assigning return values to variables and I've made sure of that.

Here's a piece of code that was hanging on Access Denied error the whole day and when it was retried later in the afternoon it executed without issue.

$connectioncs = Connect-PnPOnline -ClientId $appId -Url $siteUrl -Tenant $tenantId -Thumbprint $appCert.Thumbprint -ReturnConnection
$ctx = $Site.Context
$ctx.Load($Site)
$Site.Title = $frenchTitle
$Site.Update()
$ctx.PendingRequest.RequestExecutor.WebRequest.Headers["Accept-Language"] = "fr-FR"
$ipq = Invoke-PnPQuery -Connection $connectioncs

The complete stack trace:
Message : Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
Stacktrace : at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream)
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()
at Microsoft.SharePoint.Client.ClientRequest.d__53.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientRequest.d__39.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientRuntimeContext.d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientContext.d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.SharePoint.Client.ClientContextExtensions.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientContextExtensions.ExecuteQueryRetry(ClientRuntimeContext
clientContext, Int32 retryCount, String userAgent)
at System.Management.Automation.CommandProcessor.ProcessRecord()
ScriptLineNumber : 12
InvocationInfo : System.Management.Automation.InvocationInfo
Exception : Microsoft.SharePoint.Client.ServerUnauthorizedAccessException: Access is denied. (Exception from
HRESULT: 0x80070005 (E_ACCESSDENIED))
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream)
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()
at Microsoft.SharePoint.Client.ClientRequest.d__53.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientRequest.d__39.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientRuntimeContext.d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientContext.d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.SharePoint.Client.ClientContextExtensions.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.SharePoint.Client.ClientContextExtensions.ExecuteQueryRetry(ClientRuntimeContext
clientContext, Int32 retryCount, String userAgent)
at System.Management.Automation.CommandProcessor.ProcessRecord()