Causing issues with celldex

[Demond-dev]

Hi I can't install any versions of celldex because all versions of this package are failing to build. I was wondering if this was something that would be addressed soon?

[jkanche]

Hi, We need more information on how you are installing this. Please provide OS, architecture, your conda/mama configuration (if exists) and anything else that might help debug this issue.

I would state that we have not tested the package on windows machines. But the wheels should work on mac and *nix based systems.

[Demond-dev]

(singler_env) [williamsd7@cn0114 ~]$ pip install --no-cache-dir celldex
Collecting celldex
  Downloading celldex-0.3.0-py3-none-any.whl.metadata (8.6 kB)
Collecting dolomite_base (from celldex)
  Downloading dolomite_base-0.4.2.tar.gz (57 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting dolomite_matrix (from celldex)
  Downloading dolomite_matrix-0.4.0-py3-none-any.whl.metadata (4.9 kB)
Collecting dolomite_sce>=0.1.2 (from celldex)
  Downloading dolomite_sce-0.3.0-py3-none-any.whl.metadata (3.8 kB)
Collecting gypsum_client>=0.1.3 (from celldex)
  Downloading gypsum_client-0.2.0.tar.gz (55 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: delayedarray>=0.5.1 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from celldex) (0.6.0)
Requirement already satisfied: summarizedexperiment in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from celldex) (0.5.3)
Requirement already satisfied: singlecellexperiment in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from celldex) (0.5.7)
Requirement already satisfied: biocframe in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from celldex) (0.6.2)
Requirement already satisfied: numpy in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from delayedarray>=0.5.1->celldex) (2.2.3)
Requirement already satisfied: biocutils in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from delayedarray>=0.5.1->celldex) (0.2.2)
Collecting dolomite-se>=0.3.0 (from dolomite_sce>=0.1.2->celldex)
  Downloading dolomite_se-0.3.0-py3-none-any.whl.metadata (4.0 kB)
Collecting jsonschema (from dolomite_base->celldex)
  Downloading jsonschema-4.23.0-py3-none-any.whl.metadata (7.9 kB)
Collecting dolomite-schemas>=0.0.8 (from dolomite_base->celldex)
  Downloading dolomite-schemas-0.0.8.tar.gz (43 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting h5py (from dolomite_base->celldex)
  Downloading h5py-3.13.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Requirement already satisfied: requests in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from gypsum_client>=0.1.3->celldex) (2.32.3)
Requirement already satisfied: filelock in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from gypsum_client>=0.1.3->celldex) (3.17.0)
Collecting appdirs (from gypsum_client>=0.1.3->celldex)
  Downloading appdirs-1.4.4-py2.py3-none-any.whl.metadata (9.0 kB)
Requirement already satisfied: genomicranges>=0.5.1 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from summarizedexperiment->celldex) (0.6.2)
Collecting hdf5array (from dolomite_matrix->celldex)
  Downloading hdf5array-0.5.0-py3-none-any.whl.metadata (6.0 kB)
Collecting dolomite-ranges>=0.2.0 (from dolomite-se>=0.3.0->dolomite_sce>=0.1.2->celldex)
  Downloading dolomite_ranges-0.2.1-py3-none-any.whl.metadata (4.1 kB)
Requirement already satisfied: iranges>=0.4.2 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from genomicranges>=0.5.1->summarizedexperiment->celldex) (0.4.2)
Collecting attrs>=22.2.0 (from jsonschema->dolomite_base->celldex)
  Downloading attrs-25.1.0-py3-none-any.whl.metadata (10 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema->dolomite_base->celldex)
  Downloading jsonschema_specifications-2024.10.1-py3-none-any.whl.metadata (3.0 kB)
Collecting referencing>=0.28.4 (from jsonschema->dolomite_base->celldex)
  Downloading referencing-0.36.2-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema->dolomite_base->celldex)
  Downloading rpds_py-0.23.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from requests->gypsum_client>=0.1.3->celldex) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from requests->gypsum_client>=0.1.3->celldex) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from requests->gypsum_client>=0.1.3->celldex) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from requests->gypsum_client>=0.1.3->celldex) (2025.1.31)
Requirement already satisfied: ncls in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from iranges>=0.4.2->genomicranges>=0.5.1->summarizedexperiment->celldex) (0.0.68)
Requirement already satisfied: typing-extensions>=4.4.0 in /data/williamsd7/conda/envs/singler_env/lib/python3.12/site-packages (from referencing>=0.28.4->jsonschema->dolomite_base->celldex) (4.12.2)
Downloading celldex-0.3.0-py3-none-any.whl (15 kB)
Downloading dolomite_sce-0.3.0-py3-none-any.whl (6.2 kB)
Downloading dolomite_matrix-0.4.0-py3-none-any.whl (21 kB)
Downloading dolomite_se-0.3.0-py3-none-any.whl (8.8 kB)
Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Downloading h5py-3.13.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 192.3 MB/s eta 0:00:00
Downloading hdf5array-0.5.0-py3-none-any.whl (10 kB)
Downloading jsonschema-4.23.0-py3-none-any.whl (88 kB)
Downloading attrs-25.1.0-py3-none-any.whl (63 kB)
Downloading dolomite_ranges-0.2.1-py3-none-any.whl (9.4 kB)
Downloading jsonschema_specifications-2024.10.1-py3-none-any.whl (18 kB)
Downloading referencing-0.36.2-py3-none-any.whl (26 kB)
Downloading rpds_py-0.23.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (391 kB)
Building wheels for collected packages: dolomite_base, gypsum_client, dolomite-schemas
  Building wheel for dolomite_base (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for dolomite_base (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [102 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/__init__.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/_utils_factor.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/_utils_misc.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/_utils_string.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/alt_read_object.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/alt_save_object.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/choose_missing_placeholder.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/load_vector_from_hdf5.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/read_atomic_vector.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/read_data_frame.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/read_object.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/read_object_file.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/read_simple_list.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/read_string_factor.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/save_atomic_vector.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/save_data_frame.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/save_object.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/save_object_file.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/save_simple_list.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/save_string_factor.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/validate_object.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      copying src/dolomite_base/write_vector_to_hdf5.py -> build/lib.linux-x86_64-cpython-312/dolomite_base
      running egg_info
      writing src/dolomite_base.egg-info/PKG-INFO
      writing dependency_links to src/dolomite_base.egg-info/dependency_links.txt
      writing requirements to src/dolomite_base.egg-info/requires.txt
      writing top-level names to src/dolomite_base.egg-info/top_level.txt
      ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
      reading manifest file 'src/dolomite_base.egg-info/SOURCES.txt'
      adding license file 'LICENSE.txt'
      writing manifest file 'src/dolomite_base.egg-info/SOURCES.txt'
      running build_ext
      cmake -S lib -B build/temp.linux-x86_64-cpython-312 -Dpybind11_DIR=/tmp/pip-build-env-wds83gaq/overlay/lib/python3.12/site-packages/pybind11/share/cmake/pybind11 -DPYTHON_EXECUTABLE=/data/williamsd7/conda/envs/singler_env/bin/python3.12 -DCMAKE_BUILD_TYPE=Release -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/lib.linux-x86_64-cpython-312/dolomite_base
      -- The CXX compiler identification is GNU 8.5.0
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")
      -- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
      -- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
      -- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
      -- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
      CMake Warning (dev) at /tmp/pip-build-env-wds83gaq/overlay/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake:101 (message):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning, or
        preferably upgrade to using FindPython, either by calling it explicitly
        before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
      Call Stack (most recent call first):
        /tmp/pip-build-env-wds83gaq/overlay/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
        /tmp/pip-build-env-wds83gaq/overlay/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/pybind11Common.cmake:228 (include)
        /tmp/pip-build-env-wds83gaq/overlay/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/pybind11Config.cmake:250 (include)
        CMakeLists.txt:57 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Found PythonInterp: /data/williamsd7/conda/envs/singler_env/bin/python3.12 (found suitable version "3.12.9", minimum required is "3.7")
      -- Found PythonLibs: /data/williamsd7/conda/envs/singler_env/lib/libpython3.12.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Found pybind11: /tmp/pip-build-env-wds83gaq/overlay/lib/python3.12/site-packages/pybind11/include (found version "2.13.6")
      -- Configuring done (3.8s)
      -- Generating done (0.0s)
      -- Build files have been written to: /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312
      cmake --build build/temp.linux-x86_64-cpython-312
      [ 25%] Building CXX object CMakeFiles/dolomite_base.dir/src/load_list.cpp.o
      In file included from /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/uzuki2.hpp:17,
                       from /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/lib/src/load_list.cpp:3:
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/parse_json.hpp: In lambda function:
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/parse_json.hpp:348:26: error: ‘ritsuko’ has not been declared
                           if (!ritsuko::is_date(x.c_str(), x.size())) {
                                ^~~~~~~
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/parse_json.hpp: In lambda function:
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/parse_json.hpp:354:26: error: ‘ritsuko’ has not been declared
                           if (!ritsuko::is_rfc3339(x.c_str(), x.size())) {
                                ^~~~~~~
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/parse_json.hpp: In function ‘uzuki2::ParsedList uzuki2::json::parse(byteme::Reader&, Externals, uzuki2::json::Options)’:
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/build/temp.linux-x86_64-cpython-312/_deps/uzuki2-src/include/uzuki2/parse_json.hpp:443:25: error: ‘ritsuko’ has not been declared
                   auto vraw = ritsuko::parse_version_string(vstr.c_str(), vstr.size(), /* skip_patch = */ true);
                               ^~~~~~~
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/lib/src/load_list.cpp: In function ‘pybind11::object load_list_hdf5(std::__cxx11::string, std::__cxx11::string, pybind11::list)’:
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/lib/src/load_list.cpp:251:27: error: ‘uzuki2::hdf5’ has not been declared
           auto parsed = uzuki2::hdf5::parse<PythonProvisioner>(path, name, PythonExternals(children));
                                 ^~~~
      /tmp/pip-install-bt6e55ck/dolomite-base_9f1dd06098624ab6ba6a20f5ce016896/lib/src/load_list.cpp:251:56: error: expected primary-expression before ‘>’ token
           auto parsed = uzuki2::hdf5::parse<PythonProvisioner>(path, name, PythonExternals(children));
                                                              ^
      gmake[2]: *** [CMakeFiles/dolomite_base.dir/build.make:79: CMakeFiles/dolomite_base.dir/src/load_list.cpp.o] Error 1
      gmake[1]: *** [CMakeFiles/Makefile2:273: CMakeFiles/dolomite_base.dir/all] Error 2
      gmake: *** [Makefile:136: all] Error 2


      An error occurred while building the project, please ensure you have the most updated version of setuptools, setuptools_scm and wheel with:
         pip install -U setuptools setuptools_scm wheel


      error: command '/tmp/pip-build-env-wds83gaq/overlay/bin/cmake' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for dolomite_base
  Building wheel for gypsum_client (pyproject.toml) ... done
  Created wheel for gypsum_client: filename=gypsum_client-0.2.0-py3-none-any.whl size=41851 sha256=95e1c5c3bd5082424857dab2001bd58899ab365a81e4f9420bc99258f702cc9e
  Stored in directory: /tmp/pip-ephem-wheel-cache-q_bbf1l6/wheels/07/47/29/ab55959fd5a24093130202e679fcb47a3e4323f5fbd26c6358
  Building wheel for dolomite-schemas (pyproject.toml) ... done
  Created wheel for dolomite-schemas: filename=dolomite_schemas-0.0.8-py3-none-any.whl size=77492 sha256=cd054b728b73f46e9b23ad4bc5e6222c693dd99c7564069f99ad54a45c55a9d7
  Stored in directory: /tmp/pip-ephem-wheel-cache-q_bbf1l6/wheels/23/65/a3/eb58129cd3e99e5892e83f1511b533f2887c1212483a90103a
Successfully built gypsum_client dolomite-schemas
Failed to build dolomite_base
ERROR: Failed to build installable wheels for some pyproject.toml based projects (dolomite_base)

Installing using pip on hpc at my institute.

I've tried remaking my mamba environment with python from 3.9-3.12 since the latest versions of singlerpy have removed support for 3.8. I've tried installing older versions of celldex but run into the same issue of dolomite_base 0.4.2 failing to build regardless of what version of python or of celldex I am attempting to install.

[jkanche]

This issue is being masked (since i ran into a similar run the other day) by missing HDF5 libraries:

-- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
-- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
-- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)
-- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C CXX)

Does you HPC provide a module for HDF5 libraries? If not you need to install and load that module before installing dolomite/celldex

[jkanche]

if its helps, you could also try to modify the environment here for the packages you need: https://github.com/BiocPy/environments/

[jkanche]

@Demond-dev : can you help us identify what targets pip downloads on your systems, the snippet below should provide the output we need.

from pip._internal.models.target_python import TargetPython
target_python = TargetPython()
target_python.get_sorted_tags()

I think we figured this out. We are using a newer version of manylinux that anaconda has not updated to. I would love to confirm thats the case so that we can generate builds for multiple versions of manylinux.