Make write_env_usage atomic
#2661
Conversation
|
How about also filtering it down either when we load it or when we write it back to only keep the most recent usage of each manifest path? Otherwise I think that as this file gets large this will start to be problematically slow. If you're just appending (which isn't atomic, as we've learned), then it makes sense to keep all the old entries, but since we're not doing that any more, let's clean up as we go. |
IanButterworth
force-pushed
the
ib/atomic_usage_file_write
branch
from
abe8c0f to
ca5e6b8
Compare
|
I think that's implemented now. It tidied up my usage file nicely, and sorted it. Btw, I have a feeling the code could be reduced a bit |
IanButterworth
force-pushed
the
ib/atomic_usage_file_write
branch
from
ca5e6b8 to
4246e3d
Compare
|
Just want to add that this is really nicely done and should make Pkg much more useable on shared file systems. |
This reverts commit 9f74abe.
This reverts commit ebac45f.
This reverts commit 9f74abe.
release-1.7 Revert "Make `write_env_usage` atomic (#2661)"
|
welp |
|
I'm pretty sure that all that was required to fix this was adding https://github.com/JuliaLang/julia/blob/c5f348726cebbe55e169d4d62225c2b1e587f497/base/file.jl#L328 |
|
That'd be nice and simple! Before that I'll try to add a multi-process test to Pkg master that reproduces the problem |
Nah. See JuliaLang/julia#42253 (comment). |
This reverts commit ebac45f.
This reverts commit ebac45f.
Fixes #2633
As per @StefanKarpinski's suggestion in #2633 (comment)