Skip to content

Checkpointing fails for a process with "uprobes" VMA #1961

@mironov-stripe

Description

@mironov-stripe

Description
Hey folks,

Whilst trying out CRIU today I noticed that docker container checkpointing fails for the (containerized) processes with [uprobes] VMA's. I was able to "mitigate" the issue by making this area known to CRIU and skipping the madvise() bits on restore. I found this piece of code mentioning uprobes though it's commented-out and the original commit doesn't say much about it.

Steps to reproduce the issue:

  1. A machine with CONFIG_UPROBE_EVENTS=y (and/or BPF) enabled and some global (e.g. libc) uprobes attached (e.g. through https://osquery.io/ auditing)
  2. docker checkpoint create CONTAINER_ID CHECKPOINT_NAME

Describe the results you received:

Error response from daemon: Cannot checkpoint container CHECKPOINT_NAME: runc did not terminate successfully: criu failed: type NOTIFY errno 0 path= /run/containerd/io.containerd.runtime.v2.task/moby/<SHA>/criu-dump.log: unknown

Describe the results you expected:
Successful checkpoint.

CRIU logs and information:
From criu-dump.log:

(00.026560) Collecting mappings (pid: 123)
....
(00.028257) Error (criu/proc_parse.c:703): Unsupported mapping found 00007fffffffe000-00007ffffffff000
(00.028266) Error (criu/cr-dump.c:1524): Collect mappings (pid: 123) failed with -1

Looking at /proc/123/smaps:

7fffffffe000-7ffffffff000 --xp 00000000 00:00 0                          [uprobes]
Size:                  4 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                   4 kB
Pss:                   4 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         4 kB
Private_Dirty:         0 kB
Referenced:            4 kB
Anonymous:             0 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
FilePmdMapped:         0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
THPeligible:    0
ProtectionKey:         0
VmFlags: ex me io dc de sd
Output of `criu --version`:

# criu --version
Version: 3.17.1

Output of `criu check --all`:

# criu check --all
Looks good.

Additional environment details:
Linux kernel: 5.15.0-1015-aws / Ubuntu 20.04

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions