'_lasy_init' in 'silx/opencl/common.py' returns 'pyopencl._cl.LogicError: clGetDeviceIDs failed: INVALID_VALUE' in MacOS GitHub test workflow

[tieneupin]

Describe the bug
Our team uses silx and pyopencl as part of our package, and we are encountering an error when running them as part of our MacOS test workflow on GitHub (this doesn't occur for Windows or Ubuntu).

This is the error log returned by pytest when running tests in the GitHub workflow.

/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/detectors/__init__.py:41: in <module>
    from ._common import *
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/detectors/_common.py:49: in <module>
    from .. import io
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/io/__init__.py:59: in <module>
    from .integration_config import WorkerConfig
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/io/integration_config.py:84: in <module>
    from ..integrator import load_engines as load_integrators
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/integrator/__init__.py:43: in <module>
    from . import load_engines
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/integrator/load_engines.py:233: in <module>
    from ..opencl import ocl
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyFAI/opencl/__init__.py:61: in <module>
    ocl = common.ocl    # /!\ lasy loading of ocl at the silx level !!!
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/silx/opencl/common.py:681: in __getattr__
    if ocl.nb_devices == 0:
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/silx/opencl/common.py:406: in nb_devices
    self._lasy_init()
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/silx/opencl/common.py:422: in _lasy_init
    for idd, device in enumerate(platform.get_devices()):
E   pyopencl._cl.LogicError: clGetDeviceIDs failed: INVALID_VALUE

Environment (please complete the following information):

• OS: "MacOS-latest" (should map to macOS 14 (Sonoma))
• ICD Loader and version: Unsure
• ICD and version: Unsure
• CPU/GPU: Unsure
• Python version: 3.9 - 3.12
• PyOpenCL version: 2025.1

[inducer]

This looks like an error returned by your ICD loader or ICD, so you'll need to get clarity on what those are. Did you take specific steps to install OpenCL? How did you install PyOpenCL? Judging from the paths above, it seems like silx and pyopencl were installed into the Mac system Python? If so, that's a terrible idea/a great way to break your system. My suggestion would be to learn about conda and/or virtual environments.

[tieneupin]

Hi @inducer , thanks for replying.

This installation was done in an automated GitHub test workflow to check that the code base I'm helping to manage works as intended. I think the workflow spins up a MacOS virtual machine, installs necessary dependencies, runs the specified tests on the package, then is deleted. Because it's only temporary, I haven't bothered to add additional instructions to install it in a virtualenv in the test environment (I do agree with avoiding installing stuff in the system Python if the environment is used repeatedly).

This would be what the relevant section of my GitHub workflow YAML file look like at the moment:

jobs:
  build:
    strategy:
      fail-fast: false
      matrix:
        os: [macos-latest, ubuntu-latest, windows-2019, windows-2022]
        python-version: ["3.9", "3.10", "3.11", "3.12"]

    runs-on: ${{ matrix.os }}

    steps:
      - uses: actions/checkout@v4
      - name: Install Cairo library (Ubuntu only)
        # Cairo library needs to be manually installed on Ubuntu in order for
        # pyCairo to be installed
        if: runner.os == 'Linux'
        run: |
          sudo apt update
          sudo apt install -y libcairo2-dev
      - name: Disable OpenCL for pyFAI and silx (MacOS only)
        # There is currently an error with getting device IDs in pyOpenCL on MacOS
        # $ pyopencl._cl.LogicError: clGetDeviceIDs failed: INVALID_VALUE
        # Disabling the use of OpenCL seems to work for now
        if: runner.os == 'macOS'
        run: |
          echo "PYFAI_NO_OPENCL=1" >> $GITHUB_ENV
          echo "SILX_OPENCL=0" >> $GITHUB_ENV
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v3
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Python dependencies
        run: |
          python -m pip install --upgrade pip
          python -m pip install pytest
          python -m pip install .[dev]
      - name: Run 'pytest' tests
        run: |
          pytest

It was a very straightforward install, where pyopencl and silx were installed with no further arguments as part of the dependencies list for our package. I had to add that section to disable OpenCL in the macOS test workflow only, as it works fine on the Ubuntu and Windows workflows. I'm not sure why that is the case, so any pointers in this matter would be much appreciated.

[inducer]

PyOpenCL needs an OpenCL ICD (a "driver") to operate. There exist ICDs (and an ICD loader) built into the OS on mac, but these are (typically) badly broken. What you're seeing is evidence of that: macOS's OpenCL framework gets asked "what devices do you have?", and instead of "none", it spews an error. If you would like a functioning OpenCL install on macOS, there is advice here:

https://documen.tician.de/pyopencl/

[kif]

Hi Andreas,
In what sense installing silx in addition to pyopencl is a bad idea ? Since silx depends on pyopencl for supporting OpenCL computing ? (and pyFAI depends on silx).
We recently discovered an error in the management of (opencl context) caches in the silx layer ...
silx-kit/silx#4245
But I doubt it is related. I agree getting reliably OpenCL working on macos is getting more and more difficult to maintain.

[inducer]

In what sense installing silx in addition to pyopencl is a bad idea ?

Sorry about not having been clear. What I meant was that I think it's a bad idea to install packages into the system global system Python interpreter (instead of, say a virtualenv or a conda environment). Based on the module paths (e.g. /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/silx, this is what seemed to have happened here. I did not mean to imply anything whatsoever about silx.

But I doubt it is related. I agree getting reliably OpenCL working on macos is getting more and more difficult to maintain.

I actually think it's fine as long as you don't want to talk to a GPU. :) The conda-based install flow (with pocl) seems reasonably reliable.