Add devcontainers

[bdice]

Adds rapidsmpf devcontainers.

[jameslamb]

I generally support adding these here, thanks! If this works, fine with me to merge.

[bdice]

This is pretty close now. It's just failing to find ucxx on pip devcontainers. I am going to pause work on this until #269 is merged, since there will be quite a few merge conflicts. @gforsyth Please feel free to pick this up once #269 is done, if you're able.

[trxcllnt]

@bdice I filed this PR to fix the issue with the older version of DFG, but I noticed the current behavior worked in rapidsai/devcontainers#519.

I'm curious, what changed in newer DFG versions to make it generate CUDA 12.8 dependencies even though rapidsmpf only defines the matrix for 12.9?

[bdice]

@bdice I filed this PR to fix the issue with the older version of DFG, but I noticed the current behavior worked in in rapidsai/devcontainers#519.

I'm curious, what changed in newer DFG versions to make it generate CUDA 12.8 dependencies even though rapidsmpf only defines the matrix for 12.9?

I made basically the same change in this PR. I'm generating only 12.8 environments. We concluded that we didn't want to have developer environments for CUDA 12.9 anywhere in RAPIDS until we resolve the blocking build issue we saw for CUDA 12.9 with cudf, so I changed the environments in this PR from 12.9 to 12.8.

[bdice]

I'll rebase this once #318 goes in.

[bdice]

pre-commit run --all-files fails for me without this configuration. The failure is:

.devcontainer/README.md:3: implementors ==> implementers

But that's a proper name in that file, so it's not a typo.

[bdice]

rapidsai/devcontainers#523 should fix the pip failures here. I'll retry in a bit.

[trxcllnt]

Any idea why the pip devcontainers can't FindPackage(ucxx)? Is libucxx missing from the dependencies, or does the wheel not include ucxx-config.cmake in a location CMake knows about?

[gforsyth]

or does the wheel not include ucxx-config.cmake in a location CMake knows about?

I thought we fixed that in rapidsai/ucxx#429

But that has definitely been a problem

[gforsyth]

Pulling in changes from the default branch to see if that resolves the issue -- I think we should already have the updated configs in place, but it's worth a shot

[trxcllnt]

@gforsyth when I configure locally with --debug-find-pkg=ucxx, I see CMake searching for $VIRTUAL_ENV/lib/python3.13/site-packages/cmake/data/ucxx-config.cmake, but not $VIRTUAL_ENV/lib/python3.13/site-packages/libucxx/lib64/cmake/ucxx/ucxx-config.cmake where the file actually is:

  find_package considered the following locations for ucxx's Config module:

    /home/coder/rapidsmpf/cpp/build/pip/cuda-12.8/release/CMakeFiles/pkgRedirects/ucxxConfig.cmake
    /home/coder/rapidsmpf/cpp/build/pip/cuda-12.8/release/CMakeFiles/pkgRedirects/ucxx-config.cmake
    /home/coder/.local/share/venvs/rapids/ucxxConfig.cmake
    /home/coder/.local/share/venvs/rapids/ucxx-config.cmake
    /vscode/vscode-server/bin/linux-x64/258e40fedc6cb8edf399a463ce3a9d32e7e1f6f3/bin/remote-cli/ucxxConfig.cmake
    /vscode/vscode-server/bin/linux-x64/258e40fedc6cb8edf399a463ce3a9d32e7e1f6f3/bin/remote-cli/ucxx-config.cmake
    /home/coder/.local/ucxxConfig.cmake
    /home/coder/.local/ucxx-config.cmake
    /usr/local/cuda/ucxxConfig.cmake
    /usr/local/cuda/ucxx-config.cmake
    /usr/local/python/current/ucxxConfig.cmake
    /usr/local/python/current/ucxx-config.cmake
    /usr/local/ucxxConfig.cmake
    /usr/local/ucxx-config.cmake
    /usr/ucxxConfig.cmake
    /usr/ucxx-config.cmake
    /ucxxConfig.cmake
    /ucxx-config.cmake
    /home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/cmake/data/ucxxConfig.cmake
    /home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/cmake/data/ucxx-config.cmake
    /opt/ucxxConfig.cmake
    /opt/ucxx-config.cmake

  The file was not found.

$ find $VIRTUAL_ENV/ -type f -name ucxx-config.cmake
/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/libucxx/lib64/cmake/ucxx/ucxx-config.cmake

Does rapidsmpf need to augment FindPackage when searching for C++ libraries in a virtual env?

[gforsyth]

Huh, well, I'm not clear on why the rest of the rapidsmpf builds are working, but we only updated the cmake.prefix for ucxx, not libucxx

[trxcllnt]

not clear on why the rest of the rapidsmpf builds are working

The CMake package files are located somewhere else in conda vs. pip. Is this the first package that's attempting to FindPackage(ucxx) from the wheel installation?

[gforsyth]

The CMake package files are located somewhere else in conda vs. pip. Is this the first package that's attempting to FindPackage(ucxx) from the wheel installation?

I think it is. The rapidsmpf Python wheel does a rapids_find_package(ucxx-python) but not of ucxx

[trxcllnt]

@gforsyth I succesfully configured in a venv by augmenting CMAKE_PREFIX_PATH:

$ configure-rapidsmpf-cpp \
    -Wno-dev \
    -DBUILD_BENCHMARKS=ON \
    -DBUILD_TESTS=OFF \
    -DCMAKE_PREFIX_PATH="$(sp="$(python -c 'import site; print(site.getsitepackages()[0])')"; declare -a ar="($(echo "$sp"/{libcudf,rapids_logger,librmm,libucxx}/lib{,64}/cmake))"; IFS=';'; echo "${ar[*]}")"

The expression above expands to:

$ echo "$(sp="$(python -c 'import site; print(site.getsitepackages()[0])')"; declare -a ar="($(echo "$sp"/{libcudf,rapids_logger,librmm,libucxx}/lib{,64}/cmake))"; IFS=';'; echo "${ar[*]}")"
/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/libcudf/lib/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/libcudf/lib64/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/rapids_logger/lib/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/rapids_logger/lib64/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/librmm/lib/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/librmm/lib64/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/libucxx/lib/cmake;/home/coder/.local/share/venvs/rapids/lib/python3.13/site-packages/libucxx/lib64/cmake

There's probably a way to do this in CMake, but would need to use FindPython.cmake and determine the site-packages dir ourselves :-(