fix(datastore): change OutgoingMutationQueue use TaskQueue for state transitions

[5d]

Issue #

• Crash in OutgoingMutationQueue #3325
• DataStore Crash in OutgoingMutationQueue #3467
• Crash in OutgoingMutationQueue.doStart(api:mutationEventPublisher:reconciliationQueue:) #3649

The crash reports indicate that a SEGV_ACCERR occurred due to an attempted dereference of a garbage pointer. In the implementation of the OutgoingMutationQueue, the mutationDispatchQueue was targeted to the concurrent global queue, with state transition actions being dispatched to this queue. This setup means that starting and stopping could trigger two state transite operations running concurrently. Specifically, the starting action needs to complete some I/O work, while the stopping action is cleaning up the operation queue, potentially causing a data race.

Description

In this PR, we modified mutationDispatchQueue to use the TaskQueue implementation, ensuring that all state transition actions run serially. Also, we set the mutation event operationQueue to have the same QOS as the global queue and removed the underlying queue.

General Checklist

• Added new tests to cover change, if needed
• Build succeeds with all target using Swift Package Manager
• All unit tests pass
• All integration tests pass
• Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
• Documentation update for the change if required
• PR title conforms to conventional commit style
• New or updated tests include Given When Then inline code documentation and are named accordingly testThing_condition_expectation()
• If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[lawmicha]

i think the default is .default so no need to set this

[5d]

According to the doc, the default value for the OperationQueue is NSOperationQualityOfServiceBackground. I set it to the .default because DispatchQueue.global() uses a default quality of service (qos = .default).

For queues you create yourself, the default value is NSOperationQualityOfServiceBackground. For the queue returned by the main method, the default value is NSOperationQualityOfServiceUserInteractive and cannot be changed.

[lawmicha]

ah, got it, thanks

[codecov]

Codecov Report

Attention: Patch coverage is 60.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 68.46%. Comparing base (b6a0400) to head (6baf419).
Report is 4 commits behind head on main.

Files	Patch %	Lines
.../OutgoingMutationQueue/OutgoingMutationQueue.swift	60.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3720      +/-   ##
==========================================
+ Coverage   68.38%   68.46%   +0.07%     
==========================================
  Files        1089     1089              
  Lines       37651    37651              
==========================================
+ Hits        25748    25777      +29     
+ Misses      11903    11874      -29     

Flag	Coverage Δ
API_plugin_unit_test	68.97% <ø> (+0.21%)	⬆️
AWSPluginsCore	65.81% <ø> (ø)
Amplify	48.15% <ø> (ø)
Analytics_plugin_unit_test	84.52% <ø> (ø)
Auth_plugin_unit_test	79.52% <ø> (+0.16%)	⬆️
DataStore_plugin_unit_test	81.39% <60.00%> (+0.07%)	⬆️
Geo_plugin_unit_test	72.00% <ø> (ø)
Logging_plugin_unit_test	62.87% <ø> (ø)
Predictions_plugin_unit_test	37.32% <ø> (ø)
PushNotifications_plugin_unit_test	86.21% <ø> (ø)
Storage_plugin_unit_test	76.75% <ø> (ø)
unit_tests	68.46% <60.00%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.