JNI Error Handling

[iprovalo]

I am curious if it would be beneficial to add some exception handling at the JNI layer in order to avoid the app crashes.

For example, in the offline-recognizer.cc:

SHERPA_ONNX_EXTERN_C
JNIEXPORT void JNICALL Java_com_k2fsa_sherpa_onnx_OfflineRecognizer_decode(
    JNIEnv *env, jobject /*obj*/, jlong ptr, jlong streamPtr) {
  auto recognizer = reinterpret_cast<sherpa_onnx::OfflineRecognizer *>(ptr);
  auto stream = reinterpret_cast<sherpa_onnx::OfflineStream *>(streamPtr);

  try {
    recognizer->DecodeStream(stream);
  } catch (const std::exception& e) {
      jclass exClass = env->FindClass("java/lang/RuntimeException");
      if (exClass != nullptr) {
          env->ThrowNew(exClass, e.what());
      }
  }
} 

I am observing some crashes like this on Android:

  #00  pc 0x0000000000b5def4  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #01  pc 0x0000000000b49894  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #02  pc 0x0000000000b7599c  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #03  pc 0x0000000000b75558  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #04  pc 0x0000000000b76b78  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #05  pc 0x0000000000459c04  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #06  pc 0x000000000045b67c  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #07  pc 0x000000000042eb9c  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)
  #08  pc 0x00000000000e9b60  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libsherpa-onnx-jni.so (BuildId: c816740f0b120054222ce04e10479ec3252b4474)
  #09  pc 0x00000000000e9a94  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libsherpa-onnx-jni.so (BuildId: c816740f0b120054222ce04e10479ec3252b4474)
  #10  pc 0x00000000000fa7cc  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libsherpa-onnx-jni.so (BuildId: c816740f0b120054222ce04e10479ec3252b4474)
  #11  pc 0x00000000000da578  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libsherpa-onnx-jni.so (BuildId: c816740f0b120054222ce04e10479ec3252b4474)
  #12  pc 0x000000000008a020  /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libsherpa-onnx-jni.so (Java_com_k2fsa_sherpa_onnx_OfflineRecognizer_decode+52) (BuildId: c816740f0b120054222ce04e10479ec3252b4474)
  #13  pc 0x00000000003343c0  /data/misc/apexdata/com.android.art/dalvik-cache/arm64/boot.oat (art_jni_trampoline+112)
  #14  pc 0x000000000077fc80  /apex/com.android.art/lib64/libart.so (nterp_helper+5648)

[csukuangfj]

In general, recognizer.decode() won't throw unless

• 1. Your input audio is super short, e.g., less than 0.1 seconds
• 2. Or your input audio is super long, e.g., several minutes long.

Could you describe how you get the above error?

[csukuangfj]

I have two API's right now frequently throwing a similar exception where the crash stack ends with some onnxruntime:

#00 pc 0x0000000000b5def4 /data/app/~~er-YO3C0U2mhaw4uNlItFg==/com.___.android-5IuAA6NTrxfLlXlcKIMYtQ==/split_config.arm64_v8a.apk!libonnxruntime.so (BuildId: 96aa9c7a798685f988b107ba72fae8918594328d)

It's Java_com_k2fsa_sherpa_onnx_OfflineRecognizer_decode and Java_com_k2fsa_sherpa_onnx_Vad_acceptWaveform.

Another frequent crash is with Java_com_k2fsa_sherpa_onnx_Vad_reset. This one is not related to onnxruntime.

By the way, are you using multithreads?

Vad and Stream are not thread safe.

[iprovalo]

It's a kotlin background IO thread which invokes the listening method. UI stays locked for any further actions. I keep the vad and recognizer within a synchronized method. Technically, there should not be any two threads accessing the APIs in my case. I will double check all of these objects again.

[csukuangfj]

Also, please don't forget that the stream object is also not thread safe.

[iprovalo]

Thank you! I will double-check everything.

What about adding the native debug symbols file for more detailed crash report? Do you have the instructions for for doing that by chance?

[csukuangfj]

Thank you! I will double-check everything.

What about adding the native debug symbols file for more detailed crash report? Do you have the instructions for for doing that by chance?

Sorry, I don't have much experience in debugging C++ code with Android.

[iprovalo]

@csukuangfj I was able to create the debug symbols file for sherpa-onnx project, but I am missing the onnxruntime debug symbols library. Can you please point me to the native build for onnxruntime, so I can create one as well?

[csukuangfj]

sorry，i don't know about it.

By the way, if you need to debug onnxruntime， it indicates there are issues about your code.

[iprovalo]

@csukuangfj it is most likely some problem with my code, but this issue is happening on a small percentage of a wide range of devices in the field. It is a condition which is hard to debug and I have not been able to reproduce it on any of my devices. Adding the symbolized non-stripped builds is one way to get to the bottom of this.

This is what I was looking for:
https://github.com/csukuangfj/onnxruntime-libs/blob/master/.github/workflows/aarch64-shared.yaml#L28

[csukuangfj]

it is not for Android.

We use the pre built onnxruntime lib for Android from Microsoft.