GSoC 2025: GPU-accelerated raster ops

[Firestar99]

Problem

Graphite used not have any hardware acceleration and compose the final image entirely on the CPU,
but in the past year, the vello renderer was implemented to accelerate the image composition on the
GPU. However, vello is only well suited for composing vector elements and plain raster images. It
does not implement any operations for (post-)processing raster images, like adjusting hue,
saturation, blending images, a posterization effect, etc. Currently, these “raster ops” are still
evaluated on the CPU, given that Graphite runs within the Web Assembly (wasm) environment of
the browser, limited to a single thread processing one pixel at a time. This significantly slows editing
with any raster-based content being present, in practice 2-3 fps on a typical desktop system.

Benefit

I want to accelerate the processing of rasterization ops using the GPU to achieve acceptable levels of
performance.
Instead of porting the existing raster ops written in Rust to a shader language, I want to use the
rust-gpu shader compiler to compile the existing Rust-based raster ops into shaders. It allows
Graphite to continue using the existing CPU path if the new WebGPU API WebGL is not available, without
duplicating the code across several languages.

Constraints

• rust-gpu integration should be invisible to other contributors
  • the project must build on stable
  • rust-gpu toolchain must be automagically installed
  • rust-gpu codegen backend must be automagically build and used to compile shaders
  • if compile times are bad, may switch from build scripts to on-demand cmdline
• must run on WebGL
  • vello recently added WebGL support
  • no compute shaders, everything must be done in fragment shaders
  • transpile spirv to wgsl

Deliverables

• prepare rust-gpu for integration
  • instead of backporting graphite to edition 2021, help out upgrading rust-gpu to edition 2024
  • rustup: update to nightly-2025-05-09 (~1.88) and Rust 2024 edition. Rust-GPU/rust-gpu#249
• cargo-gpu allows using rust-gpu on a stable toolchain, manages toolchain install and codegen backend build
  • refactor: use SpirvBuilder directly, just manage rustc_backend_spirv dylibs Rust-GPU/cargo-gpu#69
  • Make cargo-gpu a library, usable from build scripts Rust-GPU/cargo-gpu#71
  • bug: parallel cargo-gpu installing crashes Rust-GPU/cargo-gpu#72 (comment)
  • allow different target jsons for different codegen backend versions, blocker for rust-gpu 2024, see comment
  • integrate naga spirv -> wgsl transpiler
• first per-pixel op: invert
  • images should be propagated as wgpu images
  • invert op should launch a draw
• port simple per-pixel ops to shaders
  • eg. Threshold, Hue, Saturation, Brightness, Posterization
  • support uniforms to pass values to shaders
• port convolution kernels to shaders
  • eg. blur, bloom, edge detection
  • may access neighboring pixels to evaluate their kernel
  • may add faster compute shader variant, if supported
• extension: prototype brush engine
• extension: shader compositing

[Firestar99]

Preparation

Anything before week 1

• help debug rust-gpu edition 2024: rustup: update to nightly-2025-05-09 (~1.88) and Rust 2024 edition. Rust-GPU/rust-gpu#249
• full refactor of cargo gpu to simplify it a lot: refactor: use SpirvBuilder directly, just manage rustc_backend_spirv dylibs Rust-GPU/cargo-gpu#69
• begin port to a lib: Make cargo-gpu a library, usable from build scripts Rust-GPU/cargo-gpu#71

[Firestar99]

Week 1: being sick

Am sick this week, likely caught something last week in the (amazing) rustweek conference. Still made some progress:

• The rust-gpu edition 2024 PR needs to change the "rustc target json files", as rustc's validator would otherwise complain. cargo-gpu was never build to deal with different target jsons, and always uses the one it's shipped with. To not have a hard break between before / after that PR, I refactored it to have per-install (of our codegen backend) target jsons, extracted from our codegen backend. Includes some refactor from the lib port.
  • Updating target json support Rust-GPU/rust-gpu#256
  • Updating target json support Rust-GPU/cargo-gpu#75
• minor
  • reviewed Fix windows builds Rust-GPU/rust-gpu#252
  • reviewed fix: buff the help menu for the wgpu-runner Rust-GPU/rust-gpu#254

[Firestar99]

Week 2: cargo-gpu naga transpile vertical slice

Demo

A vertical slice of using cargo gpu to compile rust-gpu shaders and transpile them to wgsl with naga, so they can be passed to wgpu. Based on a GameJam game of mine with just 4 shaders, small enough to allow quick iteration and API exploration.
https://github.com/Firestar99/colorbubble/tree/cargo-gpu
Branch cargo-gpu

Progress

• rust-gpu 2024 prep: Was asked to put the target specs into a new rustc_codegen_spirv-target-specs crate:
  • Updating target json support Rust-GPU/cargo-gpu#75
  • Updating target json support Rust-GPU/rust-gpu#256
  • published on crates: https://crates.io/crates/rustc_codegen_spirv-target-specs
• lib port mostly done Make cargo-gpu a library, usable from build scripts Rust-GPU/cargo-gpu#71
• started on Naga transpiling to wgsl Rust-GPU/cargo-gpu#76
• found a bug that may bite us later: build script: cargo clippy fails Rust-GPU/cargo-gpu#77
• debugging why hypercube has issues building the demo on fedora, randomly fixed?
• some initial code investigation on how to integrate this into graphite
• minor
  • update compiletest_rs Rust-GPU/rust-gpu#261 to fix no $HOST env var at compile time Rust-GPU/rust-gpu#259
  • Upgrade more dependencies Rust-GPU/rust-gpu#262
  • ci: ensure lockfile is up-to-date Rust-GPU/cargo-gpu#78

[Firestar99]

Week 3: invert_gpu

Demo

A full vertical slice with a new gpu invert node in this draft PR! I hope the image isn't too confusing, this is a screenshot of the node graph, which I then inserted as an image and inverted using the gpu invert node, creating a node graph just like the original screenshot. Also we're suffering from memory leaks, which I'll have to debug soonish.

Progress

• Demo GPU invert node #2690
  • debugging why cargo-gpu's build of rustc_codegen_spirv has wasm compile flags leak into it, failing ci. Can't repo locally, and the runner seems to be down rn.
• Various small fixes for Graphite Rust-GPU/cargo-gpu#80
• review and work on difftesting for rust-gpu Add differential testing harness Rust-GPU/rust-gpu#216
  It's been in the pipeline for quite a while and we finally want to get it underway. Having this we would have probably noticed things like misimplemented intrinsics way earlier.... and noticed that the fix itself was indeed buggy as well
• speedup rust-gpu ci: Split compiletests and difftests into parallel CI jobs Rust-GPU/rust-gpu#272
• make NonUniform merge-able behind feature gate Non uniform for everything! Rust-GPU/rust-gpu#177
• minor
  • change cargo-gpu to Apache and MIT licence, same as rust-gpu Rust-GPU/cargo-gpu#82
  • review and fixup show-targets #36 Rust-GPU/cargo-gpu#79
  • review Fix inverted conditions in count_(leading|trailing)_zeros for 64-bit … Rust-GPU/rust-gpu#268
  • review Fix union field access by representing unions as structs Rust-GPU/rust-gpu#269

[Firestar99]

Week 4: spirv-unknown-wgsl

Progress

• add spirv-unknown-wgsl target via naga Rust-GPU/rust-gpu#280
• learn how to add new targets with add targets vulkan1.3, vulkan1.4 and spv1.6 Rust-GPU/rust-gpu#271
  • required updating CI tooling for vulkan1.4 ci: replace spirv-tools prebuilds with vulkan sdk from lunarg Rust-GPU/rust-gpu#279
  • which also unblocked Update mac target in CI to aarch64-apple-darwin. Rust-GPU/rust-gpu#258
• minor
  • Compiletest docs Rust-GPU/rust-gpu#275
  • review fix: array stride validation errors Rust-GPU/rust-gpu#273
  • review Add query_size_lod and query_size methods to SampledImage Rust-GPU/rust-gpu#281

[Firestar99]

Week 5: refactoring gcore

Progress

• graphite gcore refactoring
  • Put all #[test] in a mod tests #2728
  • Bulk remove old code for legacy GPU node implementations #2722
  • Remove unused dependencies from the Cargo.toml files via cargo shear #2723
  • Fix profiling CI action failing #2734
  • make editor only depend on gstd
    • Re-export all of gcore in gstd and rename 'Image' node to 'Image Value' #2733
    • Bulk rename gcore -> gstd in editor #2729
  • Make gcore a std library
    • Make gcore a std library and remove std, alloc, and more unused features #2724
    • Bulk gcore cleanup, replace core and alloc with std #2735
  • Unify feature dependencies with workspace dependencies #2736
  • Remove the serde feature and instead just always require it #2737
  • Extract gapplication_io from gcore #2742
• cleanup and compiletest add spirv-unknown-wgsl target via naga Rust-GPU/rust-gpu#280
• minor
  • ci: resolve target tuple from rustc Rust-GPU/rust-gpu#286
  • fix compiletests Rust-GPU/rust-gpu#290

Away for an event during the extended weekend.

[Firestar99]

Week 6: refactoring gcore even more

Progress

• experimenting with parallel frontend, results in discord
• graphite gcore refactoring
  • Extract gapplication_io from gcore #2742
  • Make wasm only depend on gstd, not gcore #2745
  • Prep gcore splitup: move various symbols into their own modules #2746
  • Prep gcore splitup 2: Vector extension traits #2759
  • Stacked pr ci #2763
  • Extract gpath_bool from mod gstd::ops path_bool-related nodes #2762
  • Extract gsvg_renderer from gcore, remove gcore/vello feature  #2760
  • Extract gmath_nodes from gcore::ops, create mods debug and extract_xy #2764
  • Extract gelement-nodes for nodes accepting vector, raster or element #2767
  • node macro: copy visibility from node function #2782
  • Extract graster-nodes #2783
  • Extract gbrush #2784
• minor
  • rebase and merge Various small fixes for Graphite Rust-GPU/cargo-gpu#80

[Firestar99]

Week 7

Progress

• graphite gcore refactoring
  • quite some merge conflicts from last week's PRs
• rust-gpu spirv-unknown-wgsl target
  • split out Upgrade wgpu 23 -> 25 Rust-GPU/rust-gpu#309
  • split out Make read_clock_khr always available Rust-GPU/rust-gpu#310
  • split out and finish off enum SpirvTargetEnv
    • enum SpirvTargetEnv containing all available targets Rust-GPU/rust-gpu#311
    • target enum: fixup cargo gpu Rust-GPU/cargo-gpu#92
  • finish off add spirv-unknown-wgsl target via naga Rust-GPU/rust-gpu#280
• minor
  • Fix test without --workspace failing #2805
  • review Optimize constant casts to avoid unnecessary capabilities Rust-GPU/rust-gpu#302
  • review and fixup Remove unsafe from unconditionally safe subgroup ops Rust-GPU/rust-gpu#306