WasmFS: Add wasmfs_flush() and use that to get unflushed content checks working (#19448)

We have assertions on unflushed content, which are important as on the Web
we only print full lines with console.log(), so printf("hello\nworld"); only
prints the first word, and the second waits on a newline before we do a
console.log. The assertions see if there is unflushed content and direct the
user to how to fix this potentially confusing problem.

For the assertion to work, we need to flush not only musl's libc buffers but also
our own buffering before console.log. In the JS FS that is done in JS code. For
WasmFS, we need to flush them in C++, which this PR adds.

This makes that flushing method a public API, since it seems like a generally
useful thing.

Author: kripken

emcc.py

@@ -2308,6 +2308,9 @@ def phase_linker_setup(options, state, newargs):
     settings.FILESYSTEM = 1
     settings.SYSCALLS_REQUIRE_FILESYSTEM = 0
     settings.JS_LIBRARIES.append((0, 'library_wasmfs.js'))
+    if settings.ASSERTIONS:
+      # used in assertion checks for unflushed content
+      settings.REQUIRED_EXPORTS += ['wasmfs_flush']
     if settings.FORCE_FILESYSTEM:
       # Add exports for the JS API. Like the old JS FS, WasmFS by default
       # includes just what JS parts it actually needs, and FORCE_FILESYSTEM is

src/postamble.js

@@ -269,6 +269,10 @@ function checkUnflushedContent() {
   try { // it doesn't matter if it fails
 #if SYSCALLS_REQUIRE_FILESYSTEM == 0 && '$flush_NO_FILESYSTEM' in addedLibraryItems
     flush_NO_FILESYSTEM();
+#elif WASMFS && hasExportedSymbol('wasmfs_flush')
+    // In WasmFS we must also flush the WasmFS internal buffers, for this check
+    // to work.
+    _wasmfs_flush();
 #elif hasExportedSymbol('fflush')
     _fflush(0);
 #endif

system/include/emscripten/wasmfs.h

@@ -69,6 +69,15 @@ backend_t wasmfs_create_opfs_backend(void);
 backend_t wasmfs_create_icase_backend(backend_constructor_t create_backend,
                                       void* arg);
 
+// Similar to fflush(0), but also flushes all internal buffers inside WasmFS.
+// This is necessary because in a Web environment we must buffer at an
+// additional level after libc, since console.log() prints entire lines, that
+// is, we can't print individual characters as libc feeds them to us, so we
+// buffer them and call console.log() only after a newline. This function will
+// actually flush all buffers and add newlines as necessary to get everything
+// printed out.
+void wasmfs_flush(void);
+
 // Hooks
 
 // A hook users can do to create the root directory. Overriding this allows the

system/lib/wasmfs/wasmfs.cpp

@@ -59,6 +59,15 @@ WasmFS::WasmFS() : rootDirectory(initRootDirectory()), cwd(rootDirectory) {
 __attribute__((weak)) extern "C" void __lsan_do_leak_check(void) {
 }
 
+extern "C" void wasmfs_flush(void) {
+  // Flush musl libc streams.
+  fflush(0);
+
+  // Flush our own streams. TODO: flush all backends.
+  (void)SpecialFiles::getStdout()->locked().flush();
+  (void)SpecialFiles::getStderr()->locked().flush();
+}
+
 WasmFS::~WasmFS() {
   // See comment above on this function.
   //
@@ -69,17 +78,11 @@ WasmFS::~WasmFS() {
   // time for the checks to run (since right after this nothing can be printed).
   __lsan_do_leak_check();
 
-  // Flush musl libc streams.
-  // TODO: Integrate musl exit() which would call this for us. That might also
-  //       help with destructor priority - we need to happen last.
-  fflush(0);
-
-  // Flush our own streams. TODO: flush all possible streams.
-  // Note that we lock here, although strictly speaking it is unnecessary given
-  // that we are in the destructor of WasmFS: nothing can possibly be running
-  // on files at this time.
-  (void)SpecialFiles::getStdout()->locked().flush();
-  (void)SpecialFiles::getStderr()->locked().flush();
+  // TODO: Integrate musl exit() which would flush the libc part for us. That
+  //       might also help with destructor priority - we need to happen last.
+  //       (But we would still need to flush the internal WasmFS buffers, see
+  //       wasmfs_flush() and the comment on it in the header.)
+  wasmfs_flush();
 
   // Break the reference cycle caused by the root directory being its own
   // parent.

test/test_other.py

@@ -8985,6 +8985,7 @@ def test_fflush(self):
     # fflush without the full filesystem won't quite work
     self.do_other_test('test_fflush.cpp')
 
+  @also_with_wasmfs
   def test_fflush_fs(self):
     # fflush with the full filesystem will flush from libc, but not the JS logging, which awaits a newline
     self.do_other_test('test_fflush_fs.cpp', emcc_args=['-sFORCE_FILESYSTEM'])