-
Notifications
You must be signed in to change notification settings - Fork 952
Fix module context object re-usage in scripting engines #2358
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: unstable
Are you sure you want to change the base?
Conversation
This commit refactors the scripting engine to support multiple cached module contexts per engine, rather than relying on a single cached `ValkeyModuleCtx` object. Previously, having only one cached context object caused data races over the state stored in the context object, because it's possible that a script that is running for a long time to yield and the server event loop may call the `scriptingEngineCallGetMemoryInfo` function to get the scripting engine memory information, which re-uses the same cached context object. Another possible data-race is caused by the asynchronous scripts flush, which calls the `scriptingEngineCallFreeFunction` function in an background thread, and also re-uses the cached context object. To address this, a cache array of module contexts was introduced in the scripting engine structure, with each slot dedicated to a specific use case—such as script execution, memory info queries, or function freeing. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #2358 +/- ##
============================================
- Coverage 71.41% 71.34% -0.08%
============================================
Files 123 123
Lines 67132 67163 +31
============================================
- Hits 47942 47915 -27
- Misses 19190 19248 +58
🚀 New features to boost your workflow:
|
This theoretically should be backported, but we probably won't since the impact doesn't seem that high. This is not required for RC1 but should be merged before 9.0. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I may not be familiar with the details, but overall LGTM.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
@enjoy-binbin thanks for the review! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't originally approve it because I was confused about the code call path. It's odd to me that we call engineSetupModuleCtx(engine, NULL);
even on the non-module call path. I sort of understand the fix, but it sort of feels like there might be a more structural issue.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
We setup a module context for all callback functions as a conservative approach because the callback implementation might use the context object for some unforeseen reason. |
This commit refactors the scripting engine to support multiple cached module contexts per engine, rather than relying on a single cached
ValkeyModuleCtx
object.Previously, having only one cached context object caused data races over the state stored in the context object, because it's possible that a script that is running for a long time to yield and the server event loop may call the
scriptingEngineCallGetMemoryInfo
function to get the scripting engine memory information, which re-uses the same cached context object. Another possible data-race is caused by the asynchronous scripts flush, which calls thescriptingEngineCallFreeFunction
function in an background thread, and also re-uses the cached context object.To address this, a cache array of module contexts was introduced in the scripting engine structure, with each slot dedicated to a specific use case—such as script execution, memory info queries, or function freeing.