Skip to content

Dmabuf IPC support for amdgpu plugin (v2) #2613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: wip/amdgpu
Choose a base branch
from
Open

Dmabuf IPC support for amdgpu plugin (v2) #2613

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4d8f217
restorer: Skip non-regular VMAs
fdavid-amd Feb 12, 2025
bfeb5f2
files-ext: Allow plugin files to retry
fdavid-amd Feb 12, 2025
1768541
plugin/amdgpu: Add amdgpu drm header
fdavid-amd Feb 19, 2025
7bcda56
plugin/amdgpu: Add drm header
fdavid-amd May 15, 2025
a665500
plugin/amdgpu: Add handling for amdgpu drm buffer objects
fdavid-amd Feb 12, 2025
f5a9936
plugin: Add DUMP_DEVICES_LATE callback
fdavid-amd May 15, 2025
2097325
plugin/amdgpu: Support for checkpoint of dmabuf fds
fdavid-amd Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions criu/files-ext.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,11 @@ static int open_fd(struct file_desc *d, int *new_fd)
{
struct ext_file_info *xfi;
int fd;
bool retry_needed;

xfi = container_of(d, struct ext_file_info, d);

fd = run_plugins(RESTORE_EXT_FILE, xfi->xfe->id);
fd = run_plugins(RESTORE_EXT_FILE, xfi->xfe->id, &retry_needed);
if (fd < 0) {
pr_err("Unable to restore %#x\n", xfi->xfe->id);
return -1;
Expand All @@ -57,8 +58,11 @@ static int open_fd(struct file_desc *d, int *new_fd)
if (restore_fown(fd, xfi->xfe->fown))
return -1;

*new_fd = fd;
return 0;
if (!retry_needed)
*new_fd = fd;
else
*new_fd = -1;
return retry_needed;
}

static struct file_desc_ops ext_desc_ops = {
Expand Down
5 changes: 4 additions & 1 deletion criu/include/criu-plugin.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ enum {

CR_PLUGIN_HOOK__POST_FORKING = 12,

CR_PLUGIN_HOOK__RESTORE_INIT = 13,
Copy link
Member

@rst0git rst0git Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to update plugins/amdgpu/README.md with information about the new plugin callbacks.

Copy link
Contributor Author

@fdavid-amd fdavid-amd Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That README doesn't seem to contain information about plugin callbacks; I think the appropriate place for such documentation might be the callback functions themselves or criu-plugin.h
In any case I hope it is clear what RESTORE_INIT does

Copy link
Member

@rst0git rst0git Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

criu/plugins/amdgpu/README.md

Line 203 in a000d89

*This new plugin is enabled by the new hook `__UPDATE_VMA_MAP` in our RFC patch

criu/plugins/amdgpu/README.md

Line 225 in a000d89

*This new plugin is enabled by the new hook `__RESUME_DEVICES_LATE` in our RFC

criu/plugins/amdgpu/README.md

Line 246 in a000d89

*This optimization technique is enabled by the `__POST_FORKING` hook.*

Copy link
Member

@rst0git rst0git Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this documentation is to help new users to understand how CRIU and the AMD GPU Plugin work. Most users don't read the comments in header files like criu-plugin.h.


CR_PLUGIN_HOOK__MAX
};

Expand All @@ -70,7 +72,7 @@ enum {
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__DUMP_UNIX_SK, int fd, int id);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__RESTORE_UNIX_SK, int id);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__DUMP_EXT_FILE, int fd, int id);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__RESTORE_EXT_FILE, int id);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__RESTORE_EXT_FILE, int id, bool *retry_needed);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__DUMP_EXT_MOUNT, char *mountpoint, int id);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__RESTORE_EXT_MOUNT, int id, char *mountpoint, char *old_root, int *is_file);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__DUMP_EXT_LINK, int index, int type, char *kind);
Expand All @@ -81,6 +83,7 @@ DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__RESUME_DEVICES_LATE, int pid);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__PAUSE_DEVICES, int pid);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__CHECKPOINT_DEVICES, int pid);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__POST_FORKING, void);
DECLARE_PLUGIN_HOOK_ARGS(CR_PLUGIN_HOOK__RESTORE_INIT, void);

enum {
CR_PLUGIN_STAGE__DUMP,
Expand Down
4 changes: 4 additions & 0 deletions criu/pie/restorer.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1961,6 +1961,10 @@ __visible long __export_restore_task(struct task_restore_args *args)

for (m = 0; m < sizeof(vma_entry->madv) * 8; m++) {
if (vma_entry->madv & (1ul << m)) {

if (!(vma_entry_is(vma_entry, VMA_AREA_REGULAR)))
continue;

ret = sys_madvise(vma_entry->start, vma_entry_len(vma_entry), m);
if (ret) {
pr_err("madvise(%" PRIx64 ", %" PRIu64 ", %ld) "
Expand Down
5 changes: 5 additions & 0 deletions criu/plugin.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ static cr_plugin_desc_t *cr_gen_plugin_desc(void *h, char *path)
__assign_hook(PAUSE_DEVICES, "cr_plugin_pause_devices");
__assign_hook(CHECKPOINT_DEVICES, "cr_plugin_checkpoint_devices");
__assign_hook(POST_FORKING, "cr_plugin_post_forking");
__assign_hook(RESTORE_INIT, "cr_plugin_restore_init");

#undef __assign_hook

Expand Down Expand Up @@ -260,6 +261,10 @@ int cr_plugin_init(int stage)
if (stage == CR_PLUGIN_STAGE__RESTORE && check_inventory_plugins())
goto err;

if (stage == CR_PLUGIN_STAGE__RESTORE && run_plugins(RESTORE_INIT))
goto err;


exit_code = 0;
err:
closedir(d);
Expand Down
2 changes: 1 addition & 1 deletion criu/servicefd.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,4 +313,4 @@ int clone_service_fd(struct pstree_item *me)
ret = 0;

return ret;
}
}
Loading