Trap guestOS to run SDL-oriented applications #551

ChinYikMing · 2025-01-31T23:31:07Z

To trap the guest OS SDL applications, virtual memory translation should be handled when accessing the bidirectional event queues in syscall_sdl.c. Additionally, when using the guest OS, the SDL application might be launched and terminated multiple times. To enhance the user experience, it is heuristic to properly handle three main types of exits:
1, SDL application built-in exit: The source code of the SDL application should be modified to call the syscall_exit(syscall number: 93) somewhere when executing cleanup routine before exit().
2. Ctrl-c (SIGINT) exit: Detect the ctrl-c keycode.
3. SDL window Quit event: The source code of the SDL application should be modified to call the syscall_exit(syscall number: 93) when receiving SDL window Quit event.

The main reason for handling these three types of exits is that SDL2_Mixer may malfunction if not properly initialized and destroyed, as it runs on the host side not on the guest side. Additionally, when terminating an SDL application in the guest OS, the previous SDL window should be closed.

Moreover, the default audio backend of SDL2_Mixer(downloadable from brew or Linux distro pkg managers), FluidSynth, occasionally generates the warning: "fluidsynth: warning: Ringbuffer full, try increasing synth.polyphony!" This issue causes the audio to hang, leading to an unstable playback experience. Thus, dropping FluidSynth in favor of the Timidity backend, which provides a stable audio playback experience.

Known issue: Calling SDL_DestroyWindow() on macOS does not close the previous SDL window.

Close: #510

Note that the following steps are conducted on Ubuntu 22.04 x86-64 machine

Build SDL2_Mixer from source

Download https://github.com/libsdl-org/SDL_mixer and checkout to commit de7b2f4d2a7e78db363cb26a66df4da1f9524e25
Disable FluidSynth

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -110,7 +110,7 @@ endif()
 
 option(SDL2MIXER_MIDI "Enable MIDI music" ON)
 
-cmake_dependent_option(SDL2MIXER_MIDI_FLUIDSYNTH "Support FluidSynth MIDI output" ON "SDL2MIXER_MIDI;NOT SDL2MIXER_VENDORED" OFF)
+cmake_dependent_option(SDL2MIXER_MIDI_FLUIDSYNTH "Support FluidSynth MIDI output" OFF "SDL2MIXER_MIDI;NOT SDL2MIXER_VENDORED" OFF)
 cmake_dependent_option(SDL2MIXER_MIDI_FLUIDSYNTH_SHARED "Dynamically load libfluidsynth" "${SDL2MIXER_DEPS_SHARED}" SDL2MIXER_MIDI_FLUIDSYNTH OFF)
 
 if(WIN32 OR APPLE OR HAIKU)

Build (note that this will override the pre-installed SDL2_Mixer)
Might need to install some dependency library(see the error logs if it shows)

$ mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_INCLUDEDIR=include && make -j$(nproc) && sudo make install

Download timidity's GUS patches which from the README of SDL2_Mixer.
Extract timidity/* into /etc/ directory.

Testing procedure

Unzip Image.zip and rootfs.cpio.zip to desired path.
Image.zip
rootfs.cpio.zip
Build rv32emu

$ make INITRD_SIZE=64 ENABLE_SYSTEM=1 ENABLE_SDL=1 -j$(nproc)

Run rv32emu with provided Image and rootfs.cpio

$ build/rv32emu -k <kernel_img_path> -i <rootfs_img_path>

Run SDL-oriented application in guestOS (doom-riscv or quake or smolnes)

# doom-riscv

Expectation

Three SDL-oriented applications should be runable and can be exited with all three types of exits without any crash.

Update: (SDL2_mixer build from source on macOS/arm64)

The 1, 2, 4, 5 steps are same to Linux host, but a tiny change to step 3:
3. Build (note that this will override the pre-installed SDL2_Mixer)
Might need to install some dependency library(see the error logs if it shows)

$ mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/opt/homebrew -DCMAKE_INSTALL_INCLUDEDIR=include && make -j$(sysctl -n hw.logicalcpu) && sudo make install

jserv

Benchmarks

Benchmark suite	Current: `f20b959`	Previous: `515e011`	Ratio
`Dhrystone`	`1324` Average DMIPS over 10 runs	`1306` Average DMIPS over 10 runs	`0.99`
`Coremark`	`961.974` Average iterations/sec over 10 runs	`963.25` Average iterations/sec over 10 runs	`1.00`

This comment was automatically generated by workflow using github-action-benchmark.

Makefile

src/utils.h

To trap the guest OS SDL applications, virtual memory translation should be handled when accessing the bidirectional event queues in syscall_sdl.c. Additionally, when using the guest OS, the SDL application might be launched and terminated multiple times. To enhance the user experience, it is heuristic to properly handle three main types of exits: 1, SDL application built-in exit: The source code of the SDL application should be modified to call the syscall_exit(syscall number: 93) somewhere when executing cleanup routine before exit(). 2. Ctrl-c (SIGINT) exit: Detect the ctrl-c keycode. 3. SDL window Quit event: The source code of the SDL application should be modified to call the syscall_exit(syscall number: 93) when receiving SDL window Quit event. The main reason for handling these three types of exits is that SDL2_Mixer may malfunction if not properly initialized and destroyed, as it runs on the host side not on the guest side. Additionally, when terminating an SDL application in the guest OS, the previous SDL window should be closed. Moreover, the default audio backend of SDL2_Mixer(downloadable from brew or Linux distro pkg managers), FluidSynth, occasionally generates the warning: "fluidsynth: warning: Ringbuffer full, try increasing synth.polyphony!" This issue causes the audio to hang, leading to an unstable playback experience. Thus, dropping FluidSynth in favor of the Timidity backend, which provides a stable audio playback experience. Known issue: Calling SDL_DestroyWindow() on macOS does not close the previous SDL window. Close: sysprog21#510

ChinYikMing · 2025-02-01T10:08:48Z

Once this merged, I will upload more compact rootfs.cpio to rv32emu-prebuilt.

jserv · 2025-02-01T15:27:35Z

Thank @ChinYikMing for contributing!

User-space emulation has been supported and deployable in WebAssembly since sysprog21#389, but system emulation was not yet available. With sysprog21#508, system emulation was introduced, and later, sysprog21#551 added support for trap-and-emulate of guest Linux SDL syscalls, enabling offloading to the host's SDL backend. Now, it is time to bridge all these components together. Leverage xterm.js as the frontend terminal in the web browser and bridge it with the backend VM shell through custom buffer management. This mechanism handles both standard ASCII input and escape sequences (such as arrow keys), providing a shell experience in the browser that closely resembles a real terminal. The SDL backend is also supported. After booting the guest Linux system, can run doom-riscv, quake, or smolnes to experience graphical applications. Can press Ctrl+C to exit SDL-based programs, or use their built-in exit commands. To reduce the size of the WASM file, the build is now separated into user and system targets. As a result, there are two HTML files: - user.html - system.html and two corresponding preload JavaScript: - user-pre.js and - system-pre.js Tested on latest Chrome, Firefox and Safari.

User-space emulation has been supported and deployable in WebAssembly since sysprog21#389, but system emulation was not yet available. With sysprog21#508, system emulation was introduced, and later, sysprog21#551 added support for trap-and-emulate of guest Linux SDL syscalls, enabling offloading to the host's SDL backend. Now, it is time to bridge all these components together. Leverage xterm.js as the frontend terminal in the web browser and bridge it with the backend VM shell through custom buffer management. This mechanism handles both standard ASCII input and escape sequences (such as arrow keys), providing a shell experience in the browser that closely resembles a real terminal. The SDL backend is also supported. After booting the guest Linux system, can run doom-riscv, quake, or smolnes to experience graphical applications. Can press Ctrl+C to exit SDL-based programs, or use their built-in exit commands. To reduce the size of the WASM file, the build is now separated into user and system targets. As a result, there are two HTML files: - user.html - system.html and two corresponding preload JavaScript: - user-pre.js - system-pre.js Tested on latest Chrome, Firefox and Safari.

User-space emulation has been supported and deployable in WebAssembly since sysprog21#389, but system emulation was not yet available. With sysprog21#508, system emulation was introduced, and later sysprog21#551 added support for trap-and-emulate of guest Linux SDL syscalls, enabling offloading to the host’s SDL backend. This commit bridges these components together to enable full system emulation in the browser. xterm.js is leveraged as the frontend terminal, bridged with the backend VM shell through a custom buffer management mechanism. This mechanism handles both standard ASCII input and escape sequences (such as arrow keys), providing a shell experience in the browser that closely resembles a real terminal. To handle terminal input without blocking the browser’s cooperative multitasking model, the original approach of mapping the read() system call to window.prompt() is avoided. Blocking behavior would freeze the browser’s event loop and degrade responsiveness. Instead, input from xterm.js is stored in a shared input buffer. The rv32emu periodically checks this buffer when handling external interrupts, and if input is available, it is read and consumed by the guest OS shell. The SDL graphic and sound backend are also supported. After booting the guest Linux system, users can run graphical applications such as doom-riscv, quake, or smolnes. These programs can be exited using Ctrl+C or their built-in exit funtionality. To reduce the size of the WebAssembly build and for the sake of the modularity, the project is now separated into user and system targets. As a result, two dedicated HTML files and corresponding preload JavaScript files are introduced: - user.html with user-pre.js - system.html with system-pre.js Navigation buttons are added to the index pages of both user and system demos, allowing users to switch easily between the two modes. Note that these navigation buttons are only functional when both user and system demos are deployed together, otherwise, the target pages may not be reachable. To improve usability, logic is implemented to disable and enable the "Run" button at appropriate times, preventing accidental re-execution when the process is already running. Additional improvements: - Ensure xterm.js uses \r\n instead of \n when logging to correctly move the cursor to the beginning of the line. - Add a new src/emsc.h header to store Emscripten-related variables and function declarations for better management. This implementation has been tested on the latest versions of Chrome, Firefox, and Safari. To serve user space emulation index page: $ make start-web CC=emcc ENABLE_SDL=1 -j8 To serve system emulation index page: $ make start-web CC=emcc ENABLE_SYSTEM=1 ENABLE_SDL=1 INITRD_SIZE=32 -j8

User-space emulation has been supported and deployable in WebAssembly since sysprog21#389, but system emulation was not yet available. With sysprog21#508, system emulation was introduced, and later sysprog21#551 added support for trap-and-emulate of guest Linux SDL syscalls, enabling offloading to the host’s SDL backend. This commit bridges these components together to enable full system emulation in the browser. xterm.js is leveraged as the frontend terminal, bridged with the backend VM shell through a custom buffer management mechanism. This mechanism provides a shell experience in the browser that closely resembles a real terminal. To handle terminal input without blocking the browser’s cooperative multitasking model, the original approach of mapping the read() system call to window.prompt() is avoided. Blocking behavior would freeze the browser’s event loop and degrade responsiveness. Instead, input from xterm.js is stored in a shared input buffer. The rv32emu periodically checks this buffer when handling external interrupts, and if input is available, it is read and consumed by the guest OS shell. The SDL graphic and sound backend are also supported. After booting the guest Linux system, users can run graphical applications such as doom-riscv, quake, or smolnes. These programs can be exited using Ctrl+C or their built-in exit funtionality. To reduce the size of the WebAssembly build and for the sake of the modularity, the project is now separated into user and system targets. As a result, two dedicated HTML files and corresponding preload JavaScript files are introduced: - user.html with user-pre.js - system.html with system-pre.js Navigation buttons are added to the index pages of both user and system demos, allowing users to switch easily between the two modes. Note that these navigation buttons are only functional when both user and system demos are deployed together, otherwise, the target pages may not be reachable. To improve usability, logic is implemented to disable and enable the "Run" button at appropriate times, preventing accidental re-execution when the process is already running. Additional improvements: - Ensure xterm.js uses \r\n instead of \n when logging to correctly move the cursor to the beginning of the line. - Add a new src/emsc.h header to store Emscripten-related variables and function declarations for better management. This implementation has been tested on the latest versions of Chrome, Firefox, and Safari. To serve user space emulation index page: $ make start-web CC=emcc ENABLE_SDL=1 -j8 To serve system emulation index page: $ make start-web CC=emcc ENABLE_SYSTEM=1 ENABLE_SDL=1 INITRD_SIZE=32 -j8

User-space emulation has been supported and deployable in WebAssembly since sysprog21#389, but system emulation was not yet available. With sysprog21#508, system emulation was introduced, and later sysprog21#551 added support for trap-and-emulate of guest Linux SDL syscalls, enabling offloading to the host’s SDL backend. This commit bridges these components together to enable full system emulation in the browser. xterm.js is leveraged as the frontend terminal, bridged with the backend VM shell through a custom buffer management mechanism. This mechanism provides a shell experience in the browser that closely resembles a real terminal. To handle terminal input without blocking the browser’s cooperative multitasking model, the original approach of mapping the read() system call to window.prompt() is avoided. Blocking behavior would freeze the browser’s event loop and degrade responsiveness. Instead, input from xterm.js is stored in a shared input buffer. The rv32emu periodically checks this buffer when handling external interrupts, and if input is available, it is read and consumed by the guest OS shell. The SDL graphic and sound backend are also supported. After booting the guest Linux system, users can run graphical applications such as doom-riscv, quake, or smolnes. These programs can be exited using Ctrl+C or their built-in exit funtionality. To reduce the size of the WebAssembly build and for the sake of the modularity, the project is now separated into user and system targets. As a result, two dedicated HTML files and corresponding preload JavaScript files are introduced: - user.html with user-pre.js - system.html with system-pre.js Navigation buttons are added to the index pages of both user and system demos, allowing users to switch easily between the two modes. Note that these navigation buttons are only functional when both user and system demos are deployed together, otherwise, the target pages may not be reachable. To improve usability, logic is implemented to disable and enable the "Run" button at appropriate times, preventing accidental re-execution when the process is already running. Additional improvements: - Ensure xterm.js uses \r\n instead of \n when logging to correctly move the cursor to the beginning of the line. - Add a new src/emsc.h,c to store Emscripten-related variables and function declarations and implementations for better management. This implementation has been tested on the latest versions of Chrome, Firefox, and Safari. To serve user space emulation index page: $ make start-web CC=emcc ENABLE_SDL=1 -j8 To serve system emulation index page: $ make start-web CC=emcc ENABLE_SYSTEM=1 ENABLE_SDL=1 INITRD_SIZE=32 -j8

User-space emulation has been supported and deployable in WebAssembly since sysprog21#389, but system emulation was not yet available. With sysprog21#508, system emulation was introduced, and later sysprog21#551 added support for trap-and-emulate of guest Linux SDL syscalls, enabling offloading to the host’s SDL backend. This commit bridges these components together to enable full system emulation in the browser. xterm.js is leveraged as the frontend terminal, bridged with the backend VM shell through a custom buffer management mechanism. This mechanism provides a shell experience in the browser that closely resembles a real terminal. To handle terminal input without blocking the browser’s cooperative multitasking model, the original approach of mapping the read() system call to window.prompt() is avoided. Blocking behavior would freeze the browser’s event loop and degrade responsiveness. Instead, input from xterm.js is stored in a shared input buffer. The rv32emu periodically checks this buffer when handling external interrupts, and if input is available, it is read and consumed by the guest OS shell. The SDL graphic and sound backend are also supported. After booting the guest Linux system, users can run graphical applications such as doom-riscv, quake, or smolnes. These programs can be exited using Ctrl+C or their built-in exit funtionality. To reduce the size of the WebAssembly build and for the sake of the modularity, the project is now separated into user and system targets. As a result, two dedicated HTML files and corresponding preload JavaScript files are introduced: - user.html with user-pre.js - system.html with system-pre.js Navigation buttons are added to the index pages of both user and system demos, allowing users to switch easily between the two modes. Note that these navigation buttons are only functional when both user and system demos are deployed together, otherwise, the target pages may not be reachable. To improve usability, logic is implemented to disable and enable the "Run" button at appropriate times, preventing accidental re-execution when the process is already running. Additional improvements: - Ensure xterm.js uses \r\n instead of \n when logging to correctly move the cursor to the beginning of the line. - Add a new src/em_runtime.h,c to store Emscripten-related variables and function declarations and implementations for better management. This implementation has been tested on the latest versions of Chrome, Firefox, and Safari. To serve user space emulation index page: $ make start-web CC=emcc ENABLE_SDL=1 -j8 To serve system emulation index page: $ make start-web CC=emcc ENABLE_SYSTEM=1 ENABLE_SDL=1 INITRD_SIZE=32 -j8

ChinYikMing force-pushed the trap-guestOS-SDL-app branch from 3b4b3b4 to 35c9256 Compare January 31, 2025 23:35

jserv reviewed Jan 31, 2025

View reviewed changes

ChinYikMing force-pushed the trap-guestOS-SDL-app branch 3 times, most recently from 387e715 to a862132 Compare February 1, 2025 04:53

jserv reviewed Feb 1, 2025

View reviewed changes

Makefile Outdated Show resolved Hide resolved

jserv reviewed Feb 1, 2025

View reviewed changes

src/utils.h Outdated Show resolved Hide resolved

jserv added this to the release-2025.1 milestone Feb 1, 2025

ChinYikMing force-pushed the trap-guestOS-SDL-app branch from a862132 to 515e011 Compare February 1, 2025 09:51

Update README to reflect running SDL-oriented applications in guestOS

f20b959

jserv merged commit a8b202c into sysprog21:master Feb 1, 2025
9 checks passed

ChinYikMing deleted the trap-guestOS-SDL-app branch February 1, 2025 19:48

jserv mentioned this pull request Feb 2, 2025

Implement VirtIO sound device sysprog21/semu#50

Open

otteryc mentioned this pull request Mar 15, 2025

Prebuilt Kernel Boot Failure Due to Insufficient INITRD_SIZE #585

Open

ChinYikMing mentioned this pull request May 14, 2025

Unexpected termination of rv32emu when non-SDL Linux userspace programs exit via syscall_exit() #596

Open

ChinYikMing mentioned this pull request Jul 4, 2025

Enable System Emulation in Web Browsers #602

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Trap guestOS to run SDL-oriented applications #551

Trap guestOS to run SDL-oriented applications #551

Uh oh!

ChinYikMing commented Jan 31, 2025 •

edited

Loading

Uh oh!

jserv left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

ChinYikMing commented Feb 1, 2025

Uh oh!

Uh oh!

jserv commented Feb 1, 2025

Uh oh!

Uh oh!

Trap guestOS to run SDL-oriented applications #551

Trap guestOS to run SDL-oriented applications #551

Uh oh!

Conversation

ChinYikMing commented Jan 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Note that the following steps are conducted on Ubuntu 22.04 x86-64 machine

Build SDL2_Mixer from source

Testing procedure

Expectation

Update: (SDL2_mixer build from source on macOS/arm64)

Uh oh!

jserv left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmarks

Uh oh!

Uh oh!

Uh oh!

ChinYikMing commented Feb 1, 2025

Uh oh!

Uh oh!

jserv commented Feb 1, 2025

Uh oh!

Uh oh!

ChinYikMing commented Jan 31, 2025 •

edited

Loading

jserv left a comment •

edited

Loading