-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Optimize file copies by using NIO methods #27459
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
ce89326 to
6591ac3
Compare
| throw translateNioToIoException(from.asFragment(), e); | ||
| } | ||
| return; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also replace ByteStreams.copy(in, out) with in.transferTo(out) below? It can still do in-kernel I/O when both sides are files, and falls back to essentially equivalent code otherwise.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did that and realized that the copy fallback in moveFile can also be optimized in this way.
FileSystemUtils.copyFile via Files.copy if possible There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refactors exception handling in the VFS implementation to centralize and standardize error message formatting across different FileSystem implementations. The key changes introduce a new translateNioToIoException utility method that maps NIO exceptions to consistent java.io exceptions with Unix-style error messages.
- Centralized exception translation logic in
FileSystem.translateNioToIoException() - Refactored
getIoFile()andgetNioPath()methods to returnnullinstead of throwingUnsupportedOperationException - Optimized file copy operations to use NIO fast paths when available
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| FileSystem.java | Added centralized translateNioToIoException() method and error message constants; changed getIoFile()/getNioPath() to return null instead of throwing exceptions |
| JavaIoFileSystem.java | Simplified exception handling in createSymbolicLink(), readSymbolicLink(), and renameTo() to use the new translation method; removed duplicate error constants |
| WindowsFileSystem.java | Updated createSymbolicLink() to use centralized exception translation |
| PathTransformingDelegateFileSystem.java | Added @Nullable annotations to getIoFile() and getNioPath() methods |
| FileSystemUtils.java | Added fast path for file copying using NIO APIs; replaced custom buffer copying with transferTo(); removed copyLargeBuffer() utility method |
| AbstractFileSystem.java | Added null-check wrapper for getIoFile() call; removed duplicate ERR_PERMISSION_DENIED constant |
Comments suppressed due to low confidence (1)
src/main/java/com/google/devtools/build/lib/vfs/AbstractFileSystem.java:99
- The
getNioPath(path)method now returns@Nullableas per the changes, but this code doesn't handle the null case. IfgetNioPath()returns null, this will throw aNullPointerExceptionwhen passed toFiles.newByteChannel(). A null check should be added to handle file systems that don't support NIO paths.
return Files.newByteChannel(getNioPath(path), READ_WRITE_BYTE_CHANNEL_OPEN_OPTIONS);
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
`Files.copy` uses various optimizations such as kernel buffers (`sendfile` on Unix) or copy-on-write (`clonefile` on macOS, `copy_file_range` on Linux with a supported file system).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also amend the testGetNioPath_* tests in FileSystemTest? They no longer need to account for the UnsupportedOperationException case.
|
Importing this one myself. |
Files.copyuses various optimizations such as kernel buffers (sendfileon Unix) or copy-on-write (clonefileon macOS,copy_file_rangeon Linux with a supported file system).InputStream.transferTocan use kernel buffers when both input streams areFileInputStreams.