-
Notifications
You must be signed in to change notification settings - Fork 166
Snapshot creation parallelization #2818
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Git commit description, explain the changes you made here Leave above in PR description, copy the below into a comment Tracking
Standard development
CI Testing Labels
Documentation checklist
|
9ac798c
to
d56f5b1
Compare
d56f5b1
to
1e54139
Compare
also, I read that std::hardware_concurrency can return 0 on some machines and that it makes sense to have "back-up default". This is example from the c++ concurrency in action:
Maybe would be good that we change this at all places in the code and provide a utility function for obtaining the number of threads since std::thread::hardware_concurrency is only a hint https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency |
Was explained that the approach was different than I initially thought. All corner cases I could think up were resolved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
New flags:
--storage-parallel-snapshot-creation <- switch between single and multi threaded creation
--storage-snapshot-thread-count <- define number of threads to use (default to all)