[pch/backpressure_2] Merge changes from main and resolve conflicts (#6950)

* Enhance `async` configuration of `bindgen!` macro (#6942)

This commit takes a leaf out of `wiggle`'s book to enable bindings
generation for async host functions where only some host functions are
async instead of all of them. This enhances the `async` key with a few
more options:

    async: {
        except_imports: ["foo"],
        only_imports: ["bar"],
    }

This is beyond what `wiggle` supports where either an allow-list or
deny-list can be specified (although only one can be specified). This
can be useful if either the list of sync imports or the list of async
imports is small.

* cranelift-interpreter: Fix SIMD shifts and rotates (#6939)

* cranelift-interpreter: Fix SIMD `ishl`/`{s,u}`shr

* fuzzgen: Enable a few more ops

* cranelift: Fix tests for {u,s}shr

* fuzzgen: Change pattern matching arms for shifts

Co-Authored-By: Jamey Sharp <jsharp@fastly.com>

---------

Co-authored-by: Jamey Sharp <jsharp@fastly.com>

* Partially revert CLI argument changes from #6737 (#6944)

* Partially revert CLI argument changes from #6737

This commit is a partial revert of #6737. That change was reverted
in #6830 for the 12.0.0 release of Wasmtime and otherwise it's currently
slated to get released with the 13.0.0 release of Wasmtime. Discussion
at today's Wasmtime meeting concluded that it's best to couple this
change with #6925 as a single release rather than spread out across
multiple releases. This commit is thus the revert of #6737, although
it's a partial revert in that I've kept many of the new tests added to
showcase the differences before/after when the change lands.

This means that Wasmtime 13.0.0 will exhibit the same CLI behavior as
12.0.0 and all prior releases. The 14.0.0 release will have both a new
CLI and new argument passing semantics. I'll revert this revert (aka
re-land #6737) once the 13.0.0 release branch is created and `main`
becomes 14.0.0.

* Update release notes

* riscv64: Use `PCRelLo12I` relocation on Loads (#6938)

* riscv64: Use `PCRelLo12I` relocation on Loads

* riscv64: Strenghten pattern matching when emitting Load's

* riscv64: Clarify some of the load address logic

* riscv64: Even stronger matching

* Update Rust in CI to 1.72.0, clarify Wasmtime's MSRV (#6900)

* Update Rust in CI to 1.72.0

* Update CI, tooling, and docs for MSRV

This commit codifies an MSRV policy for Wasmtime at "stable minus two"
meaning that the latest three releases of Rust will be supported. This
is enforced on CI with a full test suite job running on Linux x86_64
with the minimum supported Rust version. The full test suite will use
the latest stable version. A downside of this approach is that new
changes may break MSRV support on non-Linux or non-x86_64 platforms and
we won't know about it, but that's deemed a minor enough risk at this
time.

A minor fix is applied to Wasmtime's `Cargo.toml` to support Rust 1.70.0
instead of requiring Rust 1.71.0

* Fix installation of rust

* Scrape MSRV from Cargo.toml

* Cranelift is the same as Wasmtime's MSRV now, more words too

* Fix a typo

* aarch64: Use `RegScaled*` addressing modes (#6945)

This commit adds a few cases to `amode` construction on AArch64 for
using the `RegScaled*` variants of `AMode`. This won't affect wasm due
to this only matching the sign-extension happening before the shift, but
it should otherwise help non-wasm Cranelift use cases.

Closes #6742

* cranelift: Validate `iconst` ranges (#6850)

* cranelift: Validate `iconst` ranges

Add the following checks:

`iconst.i8`  immediate must be within 0 .. 2^8-1
`iconst.i16` immediate must be within 0 .. 2^16-1
`iconst.i32` immediate must be within 0 .. 2^32-1

Resolves #3059

* cranelift: Parse `iconst` according to its type

Modifies the parser for textual CLIF so that V in `iconst.T V` is
parsed according to T.

Before this commit, something like `iconst.i32 0xffff_ffff_ffff` was
valid because all `iconst` were parsed the same as an
`iconst.i64`. Now the above example will throw an error.

Also, a negative immediate as in `iconst.iN -X` is now converted to
`2^N - X`.

This commit also fixes some broken tests.

* cranelift: Update tests to match new CLIF parser

* Some minor fixes and features for WASI and sockets (#6948)

* Use `command::add_to_linker` in tests to reduce the number of times
  all the `add_to_linker` are listed.
* Add all `wasi:sockets` interfaces currently implemented to both the
  sync and async `command` functions (this enables all the interfaces in
  the CLI for example).
* Use `tokio::net::TcpStream::try_io` whenever I/O is performed on a
  socket, ensuring that readable/writable flags are set/cleared
  appropriately (otherwise once readable a socket is infinitely readable).
* Add a `with_ambient_tokio_runtime` helper function to use when
  creating a `tokio::net::TcpStream` since otherwise it panics due to a
  lack of active runtime in a synchronous context.
* Add `WouldBlock` handling to return a 0-length read.
* Add an `--inherit-network` CLI flag to enable basic usage of sockets
  in the CLI.

This will conflict a small amount with #6877 but should be easy to
resolve, and otherwise this targets different usability points/issues
than that PR.

---------

Co-authored-by: Afonso Bordado <afonso360@users.noreply.github.com>
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
Co-authored-by: Timothée Jourde <timjrd@netc.fr>

Author: 4 people

.github/actions/install-rust/action.yml

@@ -5,7 +5,7 @@ inputs:
   toolchain:
     description: 'Default toolchan to install'
     required: false
-    default: '1.71.0'
+    default: 'default'
   lockfiles:
     description: 'Path glob for Cargo.lock files to use as cache keys'
     required: false
@@ -14,13 +14,28 @@ inputs:
 runs:
   using: composite
   steps:
+    - name: Install Rust
+      shell: bash
+      id: select
+      run: |
+        # Determine MSRV as N in `1.N.0` by looking at the `rust-version`
+        # located in the root `Cargo.toml`.
+        msrv=$(grep 'rust-version.*1' Cargo.toml | sed 's/.*\.\([0-9]*\)\..*/\1/')
+
+        if [ "${{ inputs.toolchain }}" = "default" ]; then
+          echo "version=1.$((msrv+2)).0" >> "$GITHUB_OUTPUT"
+        elif [ "${{ inputs.toolchain }}" = "msrv" ]; then
+          echo "version=1.$msrv.0" >> "$GITHUB_OUTPUT"
+        else
+          echo "version=${{ inputs.toolchain }}" >> "$GITHUB_OUTPUT"
+        fi
 
     - name: Install Rust
       shell: bash
       run: |
         rustup set profile minimal
-        rustup update "${{ inputs.toolchain }}" --no-self-update
-        rustup default "${{ inputs.toolchain }}"
+        rustup update "${{ steps.select.outputs.version }}" --no-self-update
+        rustup default "${{ steps.select.outputs.version }}"
 
         # Save disk space by avoiding incremental compilation. Also turn down
         # debuginfo from 2 to 0 to help save disk space.
@@ -31,11 +46,7 @@ runs:
         EOF
 
         # Deny warnings on CI to keep our code warning-free as it lands in-tree.
-        # Don't do this on nightly though, since there's a fair amount of
-        # warning churn there.
-        if [[ "${{ inputs.toolchain }}" != nightly* ]]; then
-          echo RUSTFLAGS="-D warnings" >> "$GITHUB_ENV"
-        fi
+        echo RUSTFLAGS="-D warnings" >> "$GITHUB_ENV"
 
         if [[ "${{ runner.os }}" = "macOS" ]]; then
           cat >> "$GITHUB_ENV" <<EOF

.github/workflows/main.yml

@@ -383,6 +383,8 @@ jobs:
       with:
         submodules: true
     - uses: ./.github/actions/install-rust
+      with:
+        toolchain: ${{ matrix.rust }}
 
     # Install targets in order to build various tests throughout the repo
     - run: rustup target add wasm32-wasi wasm32-unknown-unknown ${{ matrix.target }}

Cargo.toml

@@ -123,7 +123,9 @@ exclude = [
 version = "13.0.0"
 authors = ["The Wasmtime Project Developers"]
 edition = "2021"
-rust-version = "1.71.0"
+# Wasmtime's current policy is that this number can be no larger than the
+# current stable release of Rust minus 2.
+rust-version = "1.70.0"
 
 [workspace.dependencies]
 wasmtime-wmemcheck = { path = "crates/wmemcheck", version = "=13.0.0" }

RELEASES.md

@@ -80,13 +80,6 @@ Unreleased.
   These methods do not affect the size of the pre-allocated pool.
   [#6835](https://github.com/bytecodealliance/wasmtime/pull/6835)
 
-* Options to the `wasmtime` CLI for Wasmtime itself must now come before the
-  WebAssembly module. For example `wasmtime run foo.wasm --disable-cache` now
-  must be specified as `wasmtime run --disable-cache foo.wasm`. Any
-  argument/option after the WebAssembly module is now interpreted as an argument
-  to the wasm module itself.
-  [#6737](https://github.com/bytecodealliance/wasmtime/pull/6737)
-
 * Builder methods for WASI contexts onw use `&mut self` instead of `self`.
   [#6770](https://github.com/bytecodealliance/wasmtime/pull/6770)
 

ci/build-test-matrix.js

@@ -32,13 +32,22 @@ const names = fs.readFileSync(process.argv[3]).toString();
 //   on CI.
 // * `isa` - changes to `cranelift/codegen/src/$isa` will automatically run this
 //   test suite.
+// * `rust` - the Rust version to install, and if unset this'll be set to
+//   `default`
 const array = [
   {
     "os": "ubuntu-latest",
     "name": "Test Linux x86_64",
     "filter": "linux-x64",
     "isa": "x64"
   },
+  {
+    "os": "ubuntu-latest",
+    "name": "Test MSRV on Linux x86_64",
+    "filter": "linux-x64",
+    "isa": "x64",
+    "rust": "msrv",
+  },
   {
     "os": "macos-latest",
     "name": "Test macOS x86_64",
@@ -90,6 +99,12 @@ const array = [
   }
 ];
 
+for (let config of array) {
+  if (config.rust === undefined) {
+    config.rust = 'default';
+  }
+}
+
 function myFilter(item) {
   if (item.isa && names.includes(`cranelift/codegen/src/isa/${item.isa}`)) {
     return true;

cranelift/codegen/src/isa/aarch64/inst.isle

@@ -3122,15 +3122,37 @@
 (rule 5 (amode ty (iadd (sextend x @ (value_type $I32)) y) offset)
         (AMode.RegExtended (amode_add y offset) x (ExtendOp.SXTW)))
 
+;; `RegScaled*` rules where this matches an addition of an "index register" to a
+;; base register. The index register is shifted by the size of the type loaded
+;; in bytes to enable this mode matching.
+;;
+;; Note that this can additionally bundle an extending operation but the
+;; extension must happen before the shift. This will pattern-match the shift
+;; first and then if that succeeds afterwards try to find an extend.
+(rule 6 (amode ty (iadd x (ishl y (iconst (u64_from_imm64 n)))) offset)
+        (if-let $true (u64_eq (ty_bytes ty) (u64_shl 1 n)))
+        (amode_reg_scaled (amode_add x offset) y ty))
+(rule 7 (amode ty (iadd (ishl y (iconst (u64_from_imm64 n))) x) offset)
+        (if-let $true (u64_eq (ty_bytes ty) (u64_shl 1 n)))
+        (amode_reg_scaled (amode_add x offset) y ty))
+
+(decl amode_reg_scaled (Reg Value Type) AMode)
+(rule 0 (amode_reg_scaled base index ty)
+        (AMode.RegScaled base index ty))
+(rule 1 (amode_reg_scaled base (uextend index @ (value_type $I32)) ty)
+        (AMode.RegScaledExtended base index ty (ExtendOp.UXTW)))
+(rule 2 (amode_reg_scaled base (sextend index @ (value_type $I32)) ty)
+        (AMode.RegScaledExtended base index ty (ExtendOp.SXTW)))
+
 ;; Small optimizations where constants found in `iadd` are folded into the
 ;; `offset` immediate.
 ;;
 ;; NB: this should probably be done by mid-end optimizations rather than here
 ;; in the backend, but currently Cranelift doesn't do that.
-(rule 6 (amode ty (iadd x (iconst (simm32 y))) offset)
+(rule 8 (amode ty (iadd x (iconst (simm32 y))) offset)
         (if-let new_offset (s32_add_fallible y offset))
         (amode ty x new_offset))
-(rule 7 (amode ty (iadd (iconst (simm32 x)) y) offset)
+(rule 9 (amode ty (iadd (iconst (simm32 x)) y) offset)
         (if-let new_offset (s32_add_fallible x offset))
         (amode ty y new_offset))
 

cranelift/codegen/src/isa/riscv64/inst/args.rs

@@ -161,6 +161,18 @@ impl AMode {
         }
     }
 
+    /// Retrieve a MachLabel that corresponds to this addressing mode, if it exists.
+    pub(crate) fn get_label_with_sink(&self, sink: &mut MachBuffer<Inst>) -> Option<MachLabel> {
+        match self {
+            &AMode::Const(addr) => Some(sink.get_label_for_constant(addr)),
+            &AMode::Label(label) => Some(label),
+            &AMode::RegOffset(..)
+            | &AMode::SPOffset(..)
+            | &AMode::FPOffset(..)
+            | &AMode::NominalSPOffset(..) => None,
+        }
+    }
+
     pub(crate) fn to_string_with_alloc(&self, allocs: &mut AllocationConsumer<'_>) -> String {
         format!("{}", self.clone().with_allocs(allocs))
     }

cranelift/codegen/src/isa/riscv64/inst/emit.rs

@@ -620,18 +620,49 @@ impl MachInstEmit for Inst {
                 let base = from.get_base_register();
                 let offset = from.get_offset_with_state(state);
                 let offset_imm12 = Imm12::maybe_from_i64(offset);
+                let label = from.get_label_with_sink(sink);
 
-                // TODO: We shouldn't just fall back to `LoadAddr` immediately. For `MachLabel`s
-                // we should try to emit the `auipc` and add a relocation on this load.
-                let (addr, imm12) = match (base, offset_imm12) {
-                    // If the offset fits into an imm12 we can directly encode it.
-                    (Some(base), Some(imm12)) => (base, imm12),
-                    // Otherwise load the address it into a reg and load from it.
-                    _ => {
+                let (addr, imm12) = match (base, offset_imm12, label) {
+                    // When loading from a Reg+Offset, if the offset fits into an imm12 we can directly encode it.
+                    (Some(base), Some(imm12), None) => (base, imm12),
+
+                    // Otherwise, if the offset does not fit into a imm12, we need to materialize it into a
+                    // register and load from that.
+                    (Some(_), None, None) => {
                         let tmp = writable_spilltmp_reg();
                         Inst::LoadAddr { rd: tmp, mem: from }.emit(&[], sink, emit_info, state);
                         (tmp.to_reg(), Imm12::zero())
                     }
+
+                    // If the AMode contains a label we can emit an internal relocation that gets
+                    // resolved with the correct address later.
+                    (None, Some(imm), Some(label)) => {
+                        debug_assert_eq!(imm.as_i16(), 0);
+
+                        // Get the current PC.
+                        sink.use_label_at_offset(sink.cur_offset(), label, LabelUse::PCRelHi20);
+                        Inst::Auipc {
+                            rd,
+                            imm: Imm20::from_bits(0),
+                        }
+                        .emit(&[], sink, emit_info, state);
+
+                        // Emit a relocation for the load. This patches the offset into the instruction.
+                        sink.use_label_at_offset(sink.cur_offset(), label, LabelUse::PCRelLo12I);
+
+                        // Imm12 here is meaningless since it's going to get replaced.
+                        (rd.to_reg(), Imm12::zero())
+                    }
+
+                    // These cases are impossible with the current AModes that we have. We either
+                    // always have a register, or always have a label. Never both, and never neither.
+                    (None, None, None)
+                    | (None, Some(_), None)
+                    | (Some(_), None, Some(_))
+                    | (Some(_), Some(_), Some(_))
+                    | (None, None, Some(_)) => {
+                        unreachable!("Invalid load address")
+                    }
                 };
 
                 let srcloc = state.cur_srcloc();
@@ -650,8 +681,6 @@ impl MachInstEmit for Inst {
                 let offset = to.get_offset_with_state(state);
                 let offset_imm12 = Imm12::maybe_from_i64(offset);
 
-                // TODO: We shouldn't just fall back to `LoadAddr` immediately. For `MachLabel`s
-                // we should try to emit the `auipc` and add a relocation on this store.
                 let (addr, imm12) = match (base, offset_imm12) {
                     // If the offset fits into an imm12 we can directly encode it.
                     (Some(base), Some(imm12)) => (base, imm12),

cranelift/codegen/src/legalizer/mod.rs

@@ -16,7 +16,7 @@
 use crate::cursor::{Cursor, FuncCursor};
 use crate::flowgraph::ControlFlowGraph;
 use crate::ir::immediates::Imm64;
-use crate::ir::types::{I128, I64};
+use crate::ir::types::{self, I128, I64};
 use crate::ir::{self, InstBuilder, InstructionData, MemFlags, Value};
 use crate::isa::TargetIsa;
 use crate::trace;
@@ -38,7 +38,17 @@ fn imm_const(pos: &mut FuncCursor, arg: Value, imm: Imm64, is_signed: bool) -> V
             let imm = pos.ins().iconst(I64, imm);
             pos.ins().uextend(I128, imm)
         }
-        _ => pos.ins().iconst(ty.lane_type(), imm),
+        _ => {
+            let bits = imm.bits();
+            let unsigned = match ty.lane_type() {
+                types::I8 => bits as u8 as i64,
+                types::I16 => bits as u16 as i64,
+                types::I32 => bits as u32 as i64,
+                types::I64 => bits,
+                _ => unreachable!(),
+            };
+            pos.ins().iconst(ty.lane_type(), unsigned)
+        }
     }
 }
 

cranelift/codegen/src/prelude.isle

@@ -87,6 +87,7 @@
 
 (decl pure u16_as_u64 (u16) u64)
 (extern constructor u16_as_u64 u16_as_u64)
+(convert u16 u64 u16_as_u64)
 
 (decl pure u32_as_u64 (u32) u64)
 (extern constructor u32_as_u64 u32_as_u64)