SQLite Transaction is very slow causing system CPU spike #13957
Description
Before opening, please confirm:
- I have searched for duplicate or closed issues and discussions.
- I have read the guide for submitting bug reports.
- I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
JavaScript Framework
React Native
Amplify APIs
GraphQL API, DataStore, Storage
Amplify Version
v5
Amplify Categories
No response
Backend
Amplify CLI
Environment information
# Put output below this line
System:
OS: macOS 14.2
CPU: (8) arm64 Apple M1
Memory: 125.53 MB / 16.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 14.21.3 - ~/.nvm/versions/node/v14.21.3/bin/node
npm: 6.14.18 - ~/.nvm/versions/node/v14.21.3/bin/npm
Watchman: 2024.09.23.00 - /opt/homebrew/bin/watchman
Browsers:
Chrome: 130.0.6723.70
Safari: 17.2
npmGlobalPackages:
appcenter-cli: 3.0.1
corepack: 0.15.1
dotenv-cli: 7.4.2
firebase-tools: 13.7.3
npm: 6.14.18
Describe the bug
Most of my clients are using Android 8 devices with 4 GB of RAM. My schema includes 8 models: 5 of these models each contain approximately 100 records, while the remaining three have 30k+ records each, potentially reaching up to 80k records. I’m using SQLite as the storage adapter for DataStore. During a FullSync, there’s a delay between modelSynced and syncQueriesReady of around 8 to 10 minutes, depending on the data retrieved for each model. I'm not noticing any delay in the actual data synchronization process. However, there is a noticeable time gap between the last data synchronization and the "modelSynced" & "syncQueriesReady" event, which seems to be substantial.
Upon observing the process using Amplify.Logger.LOG_LEVEL = "DEBUG", it appears that records are being written to the SQLite database one at a time. This approach is overloading the CPU, causing the system to become unresponsive and, in some cases, to reboot.
"aws-amplify": "5.3.6",
"aws-amplify-react-native": "7.0.2",
Expected behavior
The "ModelSynced" events should be triggered immediately upon completion of the synchronization process. Additionally, the synchronized records should be inserted into SQLite without any delay. Also, System CPU spike shouldn't happen when this process is happening.
Reproduction steps
- Happens only on Datastore FullSync
Code Snippet
// Put your code below this line.Log output
// Put your logs below this line
aws-exports.js
aws_appsync_authenticationType: 'API_LAMBA'
Manual configuration
No response
Additional configuration
No response
Mobile Device
No response
Mobile Operating System
No response
Mobile Browser
No response
Mobile Browser Version
No response
Additional information and screenshots
No response