Version 0.8.0 Beta 2: nvFatbin support, CUDA 12.x features, sync-async op unification, etc.

[eyalroz]

Changes since v0.7.1:

Support for the nvFatbin library (#681)

• The API wrappers now support NVIDIA's "fat binary" file format creation/marshalling library, nvFatbin. It is supported via a cuda::fatbin_builder_t class: One creates a builder, adds various fragments of fatbin-contained content (cubin, PTX, LTO IR etc.), then finally uses the build() or build_at() method to obtain the completed, final, tabin file data, in a region of memory.
• The project's CMake now exports a new target, cuda-api-wrappers::fatbin , which one should depend on when actually using the builder.
• NVIDIA has not fully documented this library, so some functionality is not fully articulated, and some is only partially supported (specifically, passing extra options when adding LTO IR or PTX)

Support for more CUDA 12.x features

• Support Module function enumeration #669 : Can now obtain the kernels available in a given cuda::module_t, with the method unique_span<kernel_t> get_kernels() const.
• Support obtaining a kernel's name, module by its handle #670 : Can now obtain a kernel's name and the module containing it via the kernel's handle; but - only the mangled kernel name is accessible, so giving that an appropriate method name: kernel_t::mangled_name() (regards Make mangled and unmangled (kernel) names more difficult to mix up #674)
• Support querying the CUDA module loading mode #675 : Can now query CUDA's module loading mode (lazy or eager)

(Note these features are not accessible if you're using the wrappers with CUDA 11.x)

More unique_span class changes

Like a recently-cut gem, one slowly polishes it until it gains it shines brightly... we had some work on unique_span in version 0.7.1 as well, and it continues:

• Change unique_span to take a deleter object taking a span, not a template param accepting taking pointer #678 : The deleter is now instance-specific, so it is possible to allocate in more than one way depending even on the span size - and also have the use of such unique-spans decoupled from the allocation decisions. Also, the deleter takes a span, not just a pointer, so it can make decisions based on the allocation size.
• Address unique_span issues from cr.SX #665 :
  • Simplified the swap() implementation
  • Removed some redundant code
  • Shortened some code
  • Can now properly convert from a span of T to a span of const T.
  • Neither release(), nor our move construction, can be noexcept - removed that marking based only on optimism

optional_ref & partial unification of async and non-async memory operations

• Add an optional_ref class #691 : Added an optional_ref class, for passing optional arguments which are references. See this blog post by Foonathan about the problems of putting references in C++ optional's.
• Unify async and non-async operations using optional_ref<const stream_t> optional arguments #689 : Many memory-related operations which had a cuda::memory::foo() and cuda::memory::async::foo() variants now have a single variant, cuda::memory::foo(), which takes an extra optional_ref<stream_t> parameter: When it's not set, it's a synchronous(ish) operation; when it is set - the operation is asynchronous and scheduled on the stream.
• Missing support for asynchronous copying using CUDA_MEMCPY2D and CUDA_MEMCPY3D_PEER parameters #688 : Now support asynchronously copying 2D and 3D data using "copy parameters" structures
• sync and async versions of copy_single disagree - pointers vs references #687 : The synchronous and asynchronous variants of copy_single() had disagreed - one took a pointer, the other a reference. With their unification, they now agree (and take a pointer).

Bug fixes

The poor man's optional class

• msvc build fix #664, poor man's optional const&& constructor causes trouble in some cases #666 : Tweaked the class to avoid build failure in MSVC
• optional::value_or() doesn't return a value #676 : value_or() now returns a value...
• poor_mans_option::value_or() should be const #682 : value_or() is now const

In example programs

• simpleCudaGraphs fails with illegal memory access + weird invalid parameter setting #672 : The simpleCudaGraphs example program was crashing due to a gratuitous setting of launch config settings
• Potential use-before-initialization in simpleIPC.cu #673 : Potential use-before-initialization in the simpleIPC exampl;e

Other changes

Build mechanism

• When building example programs, make sure kernels are built for _all_ relevant architectures #694 : Now properly building fatbin files on systems with multiple GPUs of different compute capabilities

In the wrapper APIs themselves

• Some methods in the dimensions classes are missing noexcept and/or constexpr #667 : Some dimension-classes methods are missing noexcept designators
• Rename macro to be less-clashy #671 : A bit of macro renaming to avoid clashing with other libraries
• Take more launch-config-related sizes as size_t #684 : Taking more linear sizes as size_t's in launch_config_builder_t's methods - so as to prevent narrowing-cast warnings and checking limits ourselves.
• Support getting kernels from a library with a target context in which they should be available #686 : When loading a kernel from a library, one can now specify which context to obtain the kernel.

In example programs

• Add a matrixMulCUBLAS example program #680 : Added an example program invoking cuBLAS for matrix multplication (cuBLAS is not wrapped by this library)
• Unify the random values generator across vectorAdd* examples #683 : The random number generation is now the same in all vectorAdd example programs

---

This discussion was created from the release Version 0.8.0 Beta 2: nvFatbin support, CUDA 12.x features, sync-async op unification, etc..