JPype "unable to convert" 2d boolean NumPy array as of NumPy 2.3

[john-hen]

Not sure if it's a problem with JPype or NumPy (could be the latter), but as of NumPy 2.3, the following script

import jpype
from jpype import JArray, JBoolean
from numpy import array

jpype.startJVM()
bool_2d = JArray(JBoolean, dims=2)(array([[True, False], [False, True]]))

fails with

Traceback (most recent call last):
  File "convert_bool2d.py", line 6, in <module>
    bool_2d = JArray(JBoolean, dims=2)(array([[True, False], [False, True]]))
TypeError: Unable to convert

It does not fail with NumPy 2.2.6, but does with NumPy 2.3.0 and 2.3.1. (Regardless of the JPype version, 1.5.2 or 1.6.0.)

It still works for 1d boolean arrays (dims=1), and for JInt and JDouble arrays of any dimension (that I've tested).

The work-around is to call .tolist() on the boolean NumPy array object, then it works again. My test suite caught this, and yes, I could just use the .tolist() call going forward. Just wanted to bring it to your attention, maybe there's a reason for this. If it is a bug in NumPy, then I wouldn't know how to describe it.

Tested on Windows with Java 17 (via winget install Azul.ZuluFX.17.JDK).

[Thrameos]

It is suddenly failed then most likely it is change in the buffer protocol implementation. While it may be clear JPype actually has no integration with numpy whatsoever (we used to). Instead we just request a buffer of a specific type and see if the type responds. We made the switch circa 0.7. There was a change in 1.6 to allow readonly buffers to be requested (or was it reverse I forget). But either way we don’t even know what numpy is and depend entirely on the buffer conversions. Of course it was supposed to just report fail so we can try other methods. If they fail on that the process would stop. What this means is that you can use memoryview or other Python native method to trigger this error by requesting the buffer yourself. (assuming that error came from numpy and not us). So first confirm which party produced the error. Then use memory view to trigger it in the new (and hopefully not the old). Then you will have something you can meaningfully report. It could be that we requested “byte” conversion and they now consider “bool” != “byte” but that is only speculation.

…

--Karl

[john-hen]

Thanks for the advice. I'll try to dig a little deeper on what has changed with the memory views in NumPy 2.3.

[marscher]

This is the tracing output using Numpy 2.3.1

< PyJPModule_newArrayType
> PyJPArray_new
  > PyJPValue_alloc
    PyJPValue_alloc: alloc boolean[][] 0x7fca3af3b250
  < PyJPValue_alloc
< PyJPArray_new
> PyJPArray_init
  PyJPArray_init: Sequence
  > JPArray::JPArray
    JPArray::JPArray: class [[Z
  < JPArray::JPArray
  > JPArray::setRange
    JPArray::setRange: Verify lengths 2 2
    JPArray::setRange: Call component set range
    > JPClass::setArrayRange
      JPClass::setArrayRange: Verify argument types
      > JPArrayClass::findJavaConversion
        > JPConversionNull::matches
        < JPConversionNull::matches
        > JPConversionObject::matches
        < JPConversionObject::matches
        > JPConversionBuffer::matches
          > JPBooleanType::findJavaConversion id="0x55abf86daf60"
          < JPBooleanType::findJavaConversion
        < JPConversionBuffer::matches
      < JPArrayClass::findJavaConversion
      JPClass::setArrayRange: EXCEPTION THROWN 0x55abd85b57c0 Unable to convert
      JPClass::setArrayRange: EXCEPTION FROM:  native/common/jp_class.cpp 265
      JPClass::setArrayRange: EXCEPTION FROM:  native/common/jp_class.cpp 277
    EXCEPTION! JPClass::setArrayRange
    JPArray::setRange: EXCEPTION FROM:  native/common/jp_array.cpp 99
  EXCEPTION! JPArray::setRange

This is with Numpy-2.0

< PyJPModule_newArrayType
> PyJPArray_new
  > PyJPValue_alloc
    PyJPValue_alloc: alloc boolean[][] 0x7f9b797775c0
  < PyJPValue_alloc
< PyJPArray_new
> PyJPArray_init
  PyJPArray_init: Sequence
  > JPArray::JPArray
    JPArray::JPArray: class [[Z
  < JPArray::JPArray
  > JPArray::setRange
    JPArray::setRange: Verify lengths 2 2
    JPArray::setRange: Call component set range
    > JPClass::setArrayRange
      JPClass::setArrayRange: Verify argument types
      > JPArrayClass::findJavaConversion
        > JPConversionNull::matches
        < JPConversionNull::matches
        > JPConversionObject::matches
        < JPConversionObject::matches
        > JPConversionBuffer::matches
          > JPBooleanType::findJavaConversion id="0x556b9389a2e0"
          < JPBooleanType::findJavaConversion
        < JPConversionBuffer::matches
      < JPArrayClass::findJavaConversion
      > JPArrayClass::findJavaConversion
        > JPConversionNull::matches
        < JPConversionNull::matches
        > JPConversionObject::matches
        < JPConversionObject::matches
        > JPConversionBuffer::matches
          > JPBooleanType::findJavaConversion id="0x556b9389a2e0"
          < JPBooleanType::findJavaConversion
        < JPConversionBuffer::matches
      < JPArrayClass::findJavaConversion
      JPClass::setArrayRange: Copy
      > JPArrayClass::findJavaConversion
        > JPConversionNull::matches
        < JPConversionNull::matches
        > JPConversionObject::matches
        < JPConversionObject::matches
        > JPConversionBuffer::matches
          > JPBooleanType::findJavaConversion id="0x556b9389a2e0"
          < JPBooleanType::findJavaConversion
        < JPConversionBuffer::matches
      < JPArrayClass::findJavaConversion
      > JPConversionBuffer::convert
        > JPBooleanType::setArrayRange
        < JPBooleanType::setArrayRange
      < JPConversionBuffer::convert
      > JPArrayClass::findJavaConversion
        > JPConversionNull::matches
        < JPConversionNull::matches
        > JPConversionObject::matches
        < JPConversionObject::matches
        > JPConversionBuffer::matches
          > JPBooleanType::findJavaConversion id="0x556b9389a2e0"
          < JPBooleanType::findJavaConversion
        < JPConversionBuffer::matches
      < JPArrayClass::findJavaConversion
      > JPConversionBuffer::convert
        > JPBooleanType::setArrayRange
        < JPBooleanType::setArrayRange
      < JPConversionBuffer::convert
    < JPClass::setArrayRange
  < JPArray::setRange
< PyJPArray_init

[Thrameos]

Seems like adding diagostics at that point is needed to determine what the reported. Most likely the changed type codes to something not in our convertion table. Though it is usually not hard to add another to the table once we recognize it.