Networking with QEMU for mac

[lorepieri8]

Is there a mac version of Networking with QEMU?
https://docs.zephyrproject.org/latest/guides/networking/qemu_setup.html#networking-with-qemu

[carlescufi]

You can just install QEMU with brew and it should work

[lorepieri8]

The problem is that

git clone https://github.com/zephyrproject-rtos/net-tools
cd net-tools
make

returns

 lorenzo@MacBook-Air-di-Lorenzo ~/gitHubRepo/net-tools $ make
cc    -c -o tunslip6.o tunslip6.c
cc -o tunslip6   tunslip6.c
cc    -c -o echo-client.o echo-client.c
echo-client.c:27:10: fatal error: 'linux/sockios.h' file not found
#include <linux/sockios.h>
         ^~~~~~~~~~~~~~~~~
1 error generated.
make: *** [echo-client.o] Error 1

[jukkar]

The tunslip6 program works only for Linux. Currently there is no plan to support Mac. Patches are welcome if you want to contribute Mac support.

[d3zd3z]

Have we considered trying to find if there is a NIC that Qemu is able to emulate, that has a driver for Zephyr? In that configuration, we'd had a lot more options, and the networking would be managed by Qemu, say through tun/tap or equivalent mechanisms on Mac.

[jukkar]

For x86, the Qemu supports Intel e1000 network card and we have a driver for that in Zephyr. For ARM based Qemu, there is for example stellaris ethernet driver support in Zephyr.
I do not have a mac, so cannot really comment how much changes would be needed in host side setup script. If you want to investigate this, the net-tools project has net-setup.sh script that is used to setup the host side of the tun device. The relevant Qemu configuration in Zephyr side is done by cmake/app/boilerplate.cmake script (search for zeth string in that file)

[beriberikix]

FYI, there's an early patch for QEMU that uses the modern native network emulation features in macOS.

[beriberikix]

I've been testing the patch from @codyd51 and it seems to correctly create network interfaces 🎉 ! I'm at the point where someone more experienced with Zephyr might be needed. My particular test is with qemu_cortex_m3 (since x86 doesn't work on macOS) and after I build with an echo test, here's what I'm seeing:

jonathanberi:zephyr 🚀 $ sudo /Users/jonathanberi/downloads/qemu-system-arm -cpu cortex-m3 -machine lm3s6965evb -nographic -vga none -netdev vmnet-macos,id=vmnet,mode=bridged -device rtl8139,netdev=vmnet -pidfile qemu.pid -chardev stdio,id=con,mux=on -serial chardev:con -mon chardev=con,mode=readline -kernel /Users/jonathanberi/zephyrproject/zephyr/build/zephyr/zephyr.elf
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: Started vmnet interface with configuration:
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: MTU:              1500
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: Max packet size:  1514
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: MAC:              ba:8c:3a:8a:fd:ee
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: UUID:             4B84C3D0-BDBE-405C-888E-BD04740BE69F
qemu-system-arm: -device rtl8139,netdev=vmnet: No 'PCI' bus found for device 'rtl8139'
jonathanberi:zephyr 🚀 $ sudo /Users/jonathanberi/downloads/qemu-system-arm -cpu cortex-m3 -machine lm3s6965evb -nographic -vga none -netdev vmnet-macos,id=vmnet,mode=bridged -device stellaris,netdev=vmnet -pidfile qemu.pid -chardev stdio,id=con,mux=on -serial chardev:con -mon chardev=con,mode=readline -kernel /Users/jonathanberi/zephyrproject/zephyr/build/zephyr/zephyr.elf
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: Started vmnet interface with configuration:
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: MTU:              1500
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: Max packet size:  1514
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: MAC:              6a:31:4e:7a:8d:8e
qemu-system-arm: -netdev vmnet-macos,id=vmnet,mode=bridged: info: UUID:             DCBC030E-4687-452C-8880-5443E75216A4
qemu-system-arm: -device stellaris,netdev=vmnet: 'stellaris' is not a valid device model name

Anyone willing and able to work on this with me? 🙏

[AfflatusX]

Hey @beriberikix - I'm trying to get Ethernet bridging working for my m3 QEMU emulator as well (on macOS) and found your post. I'm curious if you've made any further progress on this?

[beriberikix]

I have not, unfortunately. Hoping to connect with the maintainer @codyd51 and try to get it working soon.

[AfflatusX]

Ah okay. I think the old tap approach does not need the actual ethernet device to be emulated. rtl8139 is something QEMU can emulate, but I guess it is not compatible with the machine selected? (checking the code indicates the error is not vmnet/eth related)

stellaris does not seem to be supported by QEMU - hence the error message.

The key here seems to be somehow letting eth drivers such as eth_smsc911x or stellaris to connect to that vmnet interface and treat it as eth interface. But I could be totally wrong. In any case, if someone with better knowledge in this domain can offer some guidance I'm happy to help as well!

[AfflatusX]

@beriberikix - you can give this command line a try: qemu-system-arm -nographic -vga none -nic vmnet-macos,model=lan9118,mode=bridged ...

essentially this uses vmnet-macos backend, and lan9118 as frontend. I'm using mps2-an385 board + SMSC911X ethernet config. The vm starts without any error with this.

However, I'm still confused around how would my host macOS send packets to the VM? Is there an IP address I can just send TCP packets to?

[codyd51]

Hi everyone!

As mentioned by @beriberikix above, I'm the author of the linked QEMU patch that adds support for interfacing over a network with a macOS host via Apple’s vmnet framework. I thought I'd chime in here to answer a couple questions that've come up, and to publicly document a few details that I think are important. Please feel free to ping me if there's anything I've missed or that would be useful from my end!

1. Tun/Tap on macOS

QEMU provides (at least) two mechanisms to pass packets between a host and an emulated guest: "SLIRP" networking, and "Tun/Tap" networking.

"SLIRP" networking works internally by shipping a full TCP/IP stack within QEMU. I don't know very much about this and so won't spend too long trying to explain it, but a major limitation is that since it operates at the TCP/IP layer, it's technically incapable of sending or receiving packets being transported at a lower level or over different protocols (such as an Ethernet frame carrying ARP, for example).

"Tun/Tap" networking is the canonical way to gain access to packets at the Ethernet frame layer within QEMU, which is the lowest level relevant for these purposes. It works via a two-sided mechanism: on the host side, a special ‘device file’ is created, referred to as the tap device. When the host writes a packet to this file, it will show up as a received packet within QEMU. When a QEMU guest sends a packet via its emulated NIC, the host operating system’s implementation of the tap file will inject the packet into its native network stack. Magic!

Unfortunately, it took me months to uncover that this tun/tap mechanism as used within QEMU is incompatible with macOS, even though this is the standard approach for using a QEMU guest on a Linux host. The details aren’t particularly relevant, but the main issue is that QEMU uses GLib to manage its event loop, and GLib uses the poll() syscall to monitor the application’s open files (including the special tap device file) for changes. In this case, a change would indicate a packet being received that was sent from the host operating system. As noted in the “BUGS” subsection in the macOS documentation of poll(),

The poll() system call currently does not support devices.

Thus, QEMU never would have been capable of detecting the receipt of a packet on a macOS host when using QEMU’s tun/tap networking support, and wouldn’t be able to do so without radical changes to QEMU’s event loop. Additionally, the de-facto way to create the special tap device file on macOS is by using the tuntaposx kernel extension, which has been out-of-maintenance since 2015. This isn’t technically a reason for it not to work, but is worth noting.

Thankfully, Apple has provided the vmnet framework intended for use by emulators like QEMU to allow passing packets to and from the host, with various options to configure the network topology that’s produced. This framework is what I added support for to QEMU in the patch, and fulfilled my use case (implementing a network stack into my hobby OS).

2. Networking within QEMU

@AfflatusX asked a couple great questions both within and outside of this GitHub thread, and to answer them I’d like to give a bit more detail on how networking within QEMU works:
QEMU has two notions that play a part in its network story: there is a “network backend”, and a “network device”.

The “network backend” is essentially the strategy that gets used to communicate packets to and from the host: examples are the SLIRP stack, a tun/tap device, or the vmnet framework. This “network backend” strategy is what you choose when you invoke QEMU with the “-netdev vmnet-macos” option. When doing so, you’ll typically also set a name to refer to the network backend, such as in the invocation -netdev vmnet-macos,id=my_network_backend. This name is important, because it’s how we’ll show QEMU how to connect the backend to the emulated NIC (the “network device”) we’ll set up.

The “network device” is the emulated NIC that the guest operating system can send packets out from and receive packets with. When the emulated NIC is instructed to send out a packet, QEMU will find the network backend the NIC is connected to and ask it to send a packet to the host. When the network backend receives a packet from the host, it’ll direct the packet to the emulated NIC it’s connected to, and the NIC will pretend it’s received a packet over the wire. This network device is created via the -device=rtl8139 part of the QEMU invocation, and you connect it to to the network backend you’ve selected by assigning it the name you’ve defined for the network backend earlier: -device=rtl8139,netdev=my_network_backend. The actual emulated NIC can be of any model supported by QEMU, and it looks as though in this GitHub issue you’ve narrowed down a NIC that’s supported on the platform you’re targeting. This is good, and the vmnet network backend should work fine independently of any NIC that it gets connected to.

The question I’d like to get to is the one of how the host can send a packet to the guest, and the right answer depends on your exact use case. I’ll explain one way to do it, and point to the other ways. The vmnet framework offers several options for configuring the exact setup of the network that gets created, and I’ve added support for each of these options within the QEMU patch. That said, I’m not too knowledgable on networking in general, so I’ll try to stick to what I’m comfortable explaining. You can run help to see the various options available for configuring the vmnet network backend.

When the vmnet backend starts, it’ll communicate with the vmnet framework to establish a new network connection. Once this succeeds, macOS will provide either an IP address or a MAC address that the guest can consider as belonging to it. This will be printed out, as you’ve seen, once the interface is established after QEMU starts. Getting macOS to provide you with an IP address is probably the most straightforward approach, but it’s one that I only added support for in QEMU and am not knowledgable on myself. For my use case, since I was writing a network stack, it made more sense to address with a MAC address directly.

When this is the case, and the guest has a MAC address that uniquely identifies it, macOS will automatically send packets to the vmnet interface when a packet arrives with that MAC address set as the destination within the Ethernet frame header. If the vmnet interface is running in the “bridged” mode, the guest will further receive all packets that the host receives, and can optionally filter those packets based on the NIC’s configuration or guest operating system code.

It’s then the guest operating system’s responsibility to either choose an IP address for itself and to announce it to the rest of the network by sending an ARP announcement packet over its NIC to the rest of the network, or to request an IP address from the network router over the DHCP protocol. Once an IP address has been established, the guest can send and receive packets over IP, and we’re pretty much back into familiar territory.

As I explained to @beriberikix (and I am quoting an email I sent earlier today), the delay in getting it merged lies squarely on me. I received feedback from the QEMU maintainers that I've yet to integrate. Unfortunately, new personal-life developments have made it difficult to find the headspace to apply some of the suggested changes, and I know that makes things inconvenient for everyone. I’m sorry about that.

I hope this helps, and I’m eager to explain anything else that’s unclear or that comes up! Thanks very much for the ping and for the good discussion, and I’m hoping everyone is able to get things working =)