Skip to content

virtio: add virtiofs #86768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

fkokosinski
Copy link
Member

@fkokosinski fkokosinski commented Mar 7, 2025

This PR:

  • adds virtiofs functions and required fuse structures
  • implements zephyr filesystem operations for virtiofs
  • adds virtiofs sample that presents use of this filesystem

Note that this draft PR uses the VIRTIO PCI driver implemented in #83892, and is meant to showcase how the VIRTIO PCI driver can be used with Zephyr.

@cfriedt
Copy link
Member

cfriedt commented Apr 7, 2025

@fkokosinski -

Looks very good so far.

Is it possible to separate the virtiofs and fuse contributions into separate commits? I think they can be used independently - e.g. network based FUSE.

I was initially searching for the separation between host and device roles under drivers/virtio, but now it's a bit more clear that drivers/virtio is mainly for common components and host and device drivers will live under their respective subsystems.

Maybe slightly different naming would make it easier to differentiate which is the host driver and which is the device between e.g. with virtiofs.c and virtiofs_zvfs.c, maybe virtiofs_host.c and virtiofs_device.c. Naming things is hard though, so it's maybe good to discuss. Additionally, different subsystems have different naming conventions for things, but it would be nice to have some cross-subsystem naming.

Given that it might be hard to identify all of the different virtio drivers that will eventually be scattered throughout Zephyr, and which might do the host side vs device side, I would also like to see a new section in the documentation.

  • maybe place things under doc/hardware/virtualization
  • good high-level topics
    • Overview
    • Concepts (an SVG diagram would go a long way here)
    • Common VirtIO libraries
    • Host-side VirtIO drivers
    • Device-side VirtIO drivers
    • VirtIO Samples

Happy to help move this along any way I can.

@fkokosinski fkokosinski force-pushed the 66707-virtiofs branch 2 times, most recently from 05ddf1c to 59c7537 Compare April 23, 2025 10:47
@fkokosinski
Copy link
Member Author

Hey @cfriedt, thanks for your comment!

I believe all of your remarks have been addressed in the recent push (here and in the VIRTIO PCI API PR).

@jmichalski-ant jmichalski-ant force-pushed the 66707-virtiofs branch 2 times, most recently from 9049399 to 1a0b654 Compare June 3, 2025 09:21
@fkokosinski fkokosinski marked this pull request as ready for review June 3, 2025 10:27
@fkokosinski fkokosinski changed the title [DRAFT] virtio: add virtiofs virtio: add virtiofs Jun 3, 2025
@de-nordic de-nordic requested a review from aescolar June 3, 2025 11:37
/*
* Copyright (c) 2025 Antmicro <www.antmicro.com>
*
* SPDX-License-Identifier: Apache-2.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original header seems to be BSD licensed,
https://github.com/torvalds/linux/blob/master/include/uapi/linux/fuse.h
shouldn't this reflect the original copyright and license?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We moved the code that was based on thi sheader to a separate file (fuse_abi.h), with the original BSD license

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be possible to install this header with a kernel headers package of some kind?

Why are we even duplicating it inside of the Zephyr repo?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, there are some issues with including externally supplied headers:

  • Layout of FUSE structures is not stable and was changing in the past (e.g. struct sizes were changed regularly and in some early commits even struct names and field names were sometimes altered) and presumably will also change in the future. This may cause all sorts of problems if a subset of this header used by Zephyr is modified. The virtiofsd itself doesn't use this header, instead it uses its own definitions (https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/fuse.rs) and as of now its already a few versions behind the fuse.h from Linux
  • Zephyr allows building on platforms different than Linux and this header won't be present on those. For example, up until recently (QEMU 8.0), virtiofsd was supporting macOS hosts (in QEMU 8.0 implementation was changed to rust based one and non-linux support was dropped), so including header from Linux would artificially limit this use case.

#include <stdint.h>
#include <zephyr/kernel.h>

/* adapted from Linux: include/uapi/linux/fuse.h */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this require some license/copyrights from the original file? @kartben

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We moved the code that was based on thi sheader to a separate file (fuse_abi.h), with the original BSD license


struct fuse_out_header {
uint32_t len;
int32_t error;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spaces

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

#ifdef CONFIG_VIRTIOFS_VIRTIOFSD_UNLINK_QUIRK
struct fuse_entry_out lookup_ret;
/*
* Even if unlink doesn't take nodeid as a param it still fails with EIO if the node
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-EIO?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've reworked this function a bit, but yes - it was -EIO; changed to that.

@fkokosinski
Copy link
Member Author

Licenses make ci red

@de-nordic right. I've tagged this PR with the TSC label so that it is brought to the TSC's attention, which I believe is the required next step for this PR.

In the meantime, could you please revisit this PR and let me know if there's anything else that still needs to be addressed here?

@nashif
Copy link
Member

nashif commented Jun 30, 2025

@de-nordic right. I've tagged this PR with the TSC label so that it is brought to the TSC's attention, which I believe is the required next step for this PR.

is this about bsd licensed files?

@fkokosinski
Copy link
Member Author

@de-nordic right. I've tagged this PR with the TSC label so that it is brought to the TSC's attention, which I believe is the required next step for this PR.

is this about bsd licensed files?

Yes, it's about this single BSD-licensed header file, which specifies the FUSE ABI: https://github.com/zephyrproject-rtos/zephyr/blob/b6cb351e5418d635341b72606e45c73c4a3aa851/subsys/fs/fuse_client/fuse_abi.h.

@nashif
Copy link
Member

nashif commented Jul 1, 2025

Yes, it's about this single BSD-licensed header file, which specifies the FUSE ABI: b6cb351/subsys/fs/fuse_client/fuse_abi.h.

isnt this targeting native simulator only?

@fkokosinski
Copy link
Member Author

Yes, it's about this single BSD-licensed header file, which specifies the FUSE ABI: b6cb351/subsys/fs/fuse_client/fuse_abi.h.

isnt this targeting native simulator only?

We’re using this with QEMU. We’re showcasing this in the subsys/fs/virtiofs sample this PR is adding: https://github.com/zephyrproject-rtos/zephyr/tree/b6cb351e5418d635341b72606e45c73c4a3aa851/samples/subsys/fs/virtiofs

@nashif
Copy link
Member

nashif commented Jul 1, 2025

We’re using this with QEMU. We’re showcasing this in the subsys/fs/virtiofs sample this PR is adding: b6cb351/samples/subsys/fs/virtiofs

ok, thanks.

@@ -0,0 +1,269 @@
/* SPDX-License-Identifier: BSD-2-Clause */
Copy link
Member

@nashif nashif Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

originally in Linux this file is licensed:

SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 

looks like you are picking BSD-2 here, the question if this is something we can actually do, because here you are basically changing the license and dropping GPL and only the copyright holder should be able to do that. Need to check if we should keep the original license as is and add some clarification.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example, mbed-tls is SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later and we are fine with that, we do not modify the SPDX and do not remove GPL-2.0....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I've changed the fuse_abi.h license so that the original identifier is preserved.

@fkokosinski fkokosinski dismissed stale reviews from cfriedt, kgugala, and aescolar via 3807ec3 July 2, 2025 12:48
aescolar
aescolar previously approved these changes Jul 2, 2025
@fkokosinski
Copy link
Member Author

The following changes were made:

  • doc/LICENSING.rst now references the subsys/fs/fuse_client/fuse_abi.h.h file
  • subsys/fs/fuse_client/fuse_abi.h now directly specifies under which license it is used while preserving the original SPDX license identifier

cfriedt
cfriedt previously approved these changes Jul 3, 2025
This commit adds fuse structures, requests and functions to fill them

Signed-off-by: Jakub Michalski <jmichalski@antmicro.com>
This commit adds virtiofs functions implementing fuse operations required
to enable zephyr filesystem support

Signed-off-by: Jakub Michalski <jmichalski@antmicro.com>
This commit implements zephyr filesystem operations for virtiofs

Signed-off-by: Jakub Michalski <jmichalski@antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
This commit adds virtiofs sample that presents use of this filesystem

Signed-off-by: Jakub Michalski <jmichalski@antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
This commit adds sections to the virtio docs with the currently
supported transfer methods, drivers and samples

Signed-off-by: Jakub Michalski <jmichalski@antmicro.com>
@fkokosinski
Copy link
Member Author

Rebased due to conflicts after #92682 was merged.

Copy link

sonarqubecloud bot commented Jul 7, 2025

@nashif nashif moved this from Todo to Done in TSC Attention Needed Jul 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

7 participants