CDH: Update secure mount for BlockDevice #1052

Xynnn007 · 2025-07-14T10:28:36Z

This patch mainly reconstructs the block device module of secure mount. It includes the following contents

Rust-ify luks2 related operations
Provided several typical temporary/persistent volume application scenarios
Optimized the code structure
Optimized the protobuf definition

Please check the specific commit message

Xynnn007 · 2025-07-15T07:31:44Z

The underlying libcryptsetup-rs leverages FFI to a dynamic library of cryptsetup than a static lib. Thus it cannot be built by musl. So the way to resolve error would be make CDH dynamic linked by gnu than musl. This would also require us

install some extra runtime dependencies on the kata rootfs side

Xynnn007 · 2025-07-15T07:41:11Z

cc @ChengyuZhu6

fitzthum

Nice PR. I made a few comments.

confidential-data-hub/hub/protos/api.proto

confidential-data-hub/hub/src/storage/drivers/luks2.rs

fitzthum · 2025-07-15T19:28:59Z

confidential-data-hub/hub/src/storage/volume_type/blockdevice/mod.rs

+    /// If set to `true`, there will perform a filesystem format before mounting, if
+    /// any `filesystem_type` is specified.
+    /// If set to `false`, the block device will be mounted as is.
+    pub ephemeral: Option<String>,


I wonder if we should call this something more literal, like format_filesystem. Whether it is ephemeral storage actually depends on a few different things, such as if we format the fs, how we handle the encryption key (see above option), and how the storage is actually provided by the host. So it might be more clear to name this field more specifically. I don't have strong feelings about this though.

Also not sure this needs to be an option. We can probably just use a bool with a default value.

oh. I have lost some more notes upon this. If ephemeral is true, the original device will also be encrypted which might cause data erasing and overwritting.

Yes, with ephemeral mounts there will be a new random key. Any existing data will not be readable.

I am just saying that we have more than one option that relate to ephemeral storage. This one and the encryption key parameter above. Thus, maybe this should be called something more specific rather than ephemeral.

Let me re-design this with a more clear parameter.

confidential-data-hub/hub/src/storage/volume_type/blockdevice/mod.rs

fitzthum · 2025-07-15T19:41:55Z

confidential-data-hub/hub/src/storage/volume_type/blockdevice/mod.rs

+                }
+            })?;
+
+            self.symbol_files.push(mount_point.to_string());


I only see us push one location to this Vec. Do we need a vector? Isn't the link name always going to be the mount point?

Also, I would change the name of this field to be symlink_names, since symbol file tends to mean something else more generally.

Yes, for future extension that potentially more symlink files would be generated when doing secure mount.

fitzthum · 2025-07-15T19:49:49Z

confidential-data-hub/docs/use-cases/secure-mount-with-block-device.md

+
+Ephemeral storage is a storage that is not persistent.
+
+1. Mount a block device file, encrypt it using `luks2` with an ephemeral key, and mount the plaintext device to `/mnt/dev-path` as a file.


You might want to make it clear that these are two different options not two steps.

Oh. This is actually a case, neither two steps or two options.

Fixed the document due to new API design. ptal

confidential-data-hub/docs/use-cases/secure-mount-with-block-device.md

Xynnn007 · 2025-07-16T07:50:50Z

btw, the most important issue is the musl build. #1052 (comment)

wdyt?

mythi

this needs re-thinking:

Binary file added BIN +20 MB confidential-data-hub/hub/test_files/luks2-disk

some minor suggestions. will take another tomorrow

.github/workflows/cdh_basic.yml

attestation-agent/deps/crypto/src/rand.rs

fitzthum · 2025-07-17T15:45:41Z

@Xynnn007 I think the musl question might be one for the coco slack or the community meeting.

Xynnn007 · 2025-07-18T05:18:37Z

@fitzthum I change the API with more explicit work mode (named action). This would help to understand the blackbox much better without being annoyed by the complex Options. ptal

Xynnn007 · 2025-07-18T06:09:19Z

Binary file added BIN +20 MB confidential-data-hub/hub/test_files/luks2-disk

This is for unit test to mount an encrypted volume. I tried 16 MB~ but luks2 does not support that small, 20MB is enough.

An option is to mark it as git-lfs?

mythi · 2025-07-18T06:33:40Z

Binary file added BIN +20 MB confidential-data-hub/hub/test_files/luks2-disk

This is for unit test to mount an encrypted volume. I tried 16 MB~ but luks2 does not support that small, 20MB is enough.

An option is to mark it as git-lfs?

I was thinking the option where the test generates it runtime. Something like file create + nix fallocate.

Xynnn007 · 2025-07-18T07:29:27Z

I was thinking the option where the test generates it runtime. Something like file create + nix fallocate.

let me take a try. Combining two tests.

Updated: Done.

mkulke

LGTM!

I wonder whether the implementation will cover the requirements from the ephemeral space feature in peerpods (afaik the old bash script was just copied 1:1 to the peerpod repo) cc @bpradipt

.github/workflows/cdh_basic.yml

confidential-data-hub/hub/src/storage/drivers/luks2.rs

Xynnn007 · 2025-08-04T06:57:39Z

I wonder whether the implementation will cover the requirements from the ephemeral space feature in peerpods

Do you have any designs/issue/PR for this?

ChengyuZhu6

LGTM

The encrytion format we are using is luks2 than luks. This patch fixes this typo. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

mount_path field is specified in the request, thus we do not need it in the response body for duplicaiton. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

random_key helps to generate a random data with specified length. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

we use a rust-c-binding crate libcryptsetup-rs to implement luks2 operations. The related code can be reused so a separate mod is added. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

The SecureMount object would carry some context information when doing mount, like the directories created. It is useful when doing clean jobs. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

filesystem module provides operations for fs. Now it only supports ext4. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

This patch refactors BlockDevice module, including the following: 1. use luks2 rust module to do luks2 related operations. 2. adds a new option named `filesystem_type` to specify what filesystem is used if we need a directory mount 3. adds a new option named `ephemeral` for cases where an ephemeral storage is needed. 4. also some unit tests Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

after the change of block device code, we now support new modes of block device. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Similar to RESOURCE_PROVIDER, we define STORAGE to control features of secure mounts. Now only luks2 is supported. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

libcryptsetup-dev is required to build luks2 feature of CDH. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Now libcryptsetup does not support musl build for it cannot be linked statically, thus we use gnu to build cdh in CI pipeline. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Now libcryptsetup does not support musl build for it cannot be linked statically, thus we use gnu to build cdh in the test. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

bpradipt · 2025-08-20T04:57:27Z

I wonder whether the implementation will cover the requirements from the ephemeral space feature in peerpods

Do you have any designs/issue/PR for this?

@Xynnn007 this is the tracker issue - confidential-containers/cloud-api-adaptor#2486

fitzthum

LGTM

seems like we are generally ok with ditching musl here, but need to work out the details

Xynnn007 · 2025-08-22T01:14:41Z

let me put a PR for installing dependencies to the rootfs too and merge both at the same time

Xynnn007 requested a review from a team as a code owner July 14, 2025 10:28

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from fa35e8b to 61533ef Compare July 15, 2025 02:51

fitzthum reviewed Jul 15, 2025

View reviewed changes

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from e64b1f0 to 4907ce0 Compare July 16, 2025 07:49

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch 2 times, most recently from 5cd1934 to 279e244 Compare July 17, 2025 15:22

mythi reviewed Jul 17, 2025

View reviewed changes

.github/workflows/cdh_basic.yml Show resolved Hide resolved

attestation-agent/deps/crypto/src/rand.rs Outdated Show resolved Hide resolved

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from 279e244 to d96130a Compare July 18, 2025 05:17

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch 2 times, most recently from f15859b to 714c815 Compare July 18, 2025 06:03

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from 714c815 to 59f534e Compare July 18, 2025 06:58

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch 2 times, most recently from bdb7c69 to bc792fb Compare July 18, 2025 07:43

Xynnn007 mentioned this pull request Jul 28, 2025

CDH | storage: consider implementing luks-encrypt-storage in Rust #670

Open

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch 2 times, most recently from 3189f24 to ff316a0 Compare August 1, 2025 08:34

mkulke approved these changes Aug 1, 2025

View reviewed changes

.github/workflows/cdh_basic.yml Outdated Show resolved Hide resolved

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from ff316a0 to 3d2fca9 Compare August 4, 2025 02:47

bpradipt reviewed Aug 4, 2025

View reviewed changes

confidential-data-hub/hub/src/storage/drivers/luks2.rs Show resolved Hide resolved

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from 3d2fca9 to 0fb4b57 Compare August 4, 2025 06:47

ChengyuZhu6 approved these changes Aug 19, 2025

View reviewed changes

Xynnn007 added 16 commits August 19, 2025 11:53

cdh: rename luks to luks2

83d16f2

The encrytion format we are using is luks2 than luks. This patch fixes this typo. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

CDH: remove duplicated field of secure mount proto

7aa02db

mount_path field is specified in the request, thus we do not need it in the response body for duplicaiton. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

crypto: add rand module

4a50ee8

random_key helps to generate a random data with specified length. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

cdh: add rustify luks2 implementation

f16bb74

we use a rust-c-binding crate libcryptsetup-rs to implement luks2 operations. The related code can be reused so a separate mod is added. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

cdh: add luks2 unit tests

742c13f

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

cdh: change SecureMount trait to mut

3bd5051

The SecureMount object would carry some context information when doing mount, like the directories created. It is useful when doing clean jobs. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

cdh: fix typo

5607c51

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

cdh: add filesystem module

49cd04d

filesystem module provides operations for fs. Now it only supports ext4. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

CDH: format Cargo.toml

79250d9

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

CDH: update secure mount docs

0a8f63e

after the change of block device code, we now support new modes of block device. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

CDH: print error chain for secure mount

b2d0a5c

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

CDH: update readme and makefile for secure storage

7fc3200

Similar to RESOURCE_PROVIDER, we define STORAGE to control features of secure mounts. Now only luks2 is supported. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

ci: install libcryptsetup-dev for CDH pipeline

243c289

libcryptsetup-dev is required to build luks2 feature of CDH. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

ci: use gnu to build cdh

1f9235e

Now libcryptsetup does not support musl build for it cannot be linked statically, thus we use gnu to build cdh in CI pipeline. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

image-rs: fix CDH build in integration test

30f656a

Now libcryptsetup does not support musl build for it cannot be linked statically, thus we use gnu to build cdh in the test. Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>

Xynnn007 force-pushed the update-cdh-rbd-encrypted branch from 0fb4b57 to 30f656a Compare August 19, 2025 03:53

fitzthum approved these changes Aug 21, 2025

View reviewed changes


		Ephemeral storage is a storage that is not persistent.

		1. Mount a block device file, encrypt it using `luks2` with an ephemeral key, and mount the plaintext device to `/mnt/dev-path` as a file.

CDH: Update secure mount for BlockDevice #1052

Are you sure you want to change the base?

CDH: Update secure mount for BlockDevice #1052

Uh oh!

Conversation

Xynnn007 commented Jul 14, 2025

Uh oh!

Xynnn007 commented Jul 15, 2025

Uh oh!

Xynnn007 commented Jul 15, 2025

Uh oh!

fitzthum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Xynnn007 Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Xynnn007 commented Jul 16, 2025

Uh oh!

mythi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fitzthum commented Jul 17, 2025

Uh oh!

Xynnn007 commented Jul 18, 2025

Uh oh!

Xynnn007 commented Jul 18, 2025

Uh oh!

mythi commented Jul 18, 2025

Uh oh!

Xynnn007 commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkulke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Xynnn007 commented Aug 4, 2025

Uh oh!

ChengyuZhu6 left a comment

Choose a reason for hiding this comment

Uh oh!

bpradipt commented Aug 20, 2025

Uh oh!

fitzthum left a comment

Choose a reason for hiding this comment

Uh oh!

Xynnn007 commented Aug 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Xynnn007 Jul 18, 2025 •

edited

Loading

Xynnn007 commented Jul 18, 2025 •

edited

Loading