One-time crash on Quest 2 / Quest 3 after around 20 minutes of playtime

[electrum-bowie]

Cesium for Unity 1.7.1.

16 people using a quest vr device (android) experienced this exact crash on their own device atleast once:

Abort Reason: terminating (libCesiumForUnityNative-Runtime.so)

[electrum-bowie]

Update:
i am using Google's P3DT, i have 336870912 allocated bytes in the faulty scene,
i optimized some scripts and updated some packages and the crash seems to occur more rarely than before, i also found that i had Incremental Garbage Collector TURNED OFF. which i turned on right after.
i will update on this tommorow or the day after.

[csciguy8]

Very likely related to CesiumGS/cesium-native#739

[kring]

It's hard to say what's going on here. The issue @csciguy8 linked could indeed be the problem, though in that case you should see signs the device ran out of memory. It seems we're in CesiumForUnityNative::Cesium3DTilesetImpl::Update and then "something" goes wrong and there's a crash, but the call stack in between is not very useful so it's very hard to tell what happened. Even the 356 listed doesn't make much sense as a line number, so I'm assuming that's a byte offset in the built code and I don't have an easy way to resolve that to specific code. Can you try running a debug build on the device and see if it provides more useful information?

[electrum-bowie]

It's hard to say what's going on here. The issue @csciguy8 linked could indeed be the problem, though in that case you should see signs the device ran out of memory.

The device does not run out of memory during or before the crash, not even close actually, in fact the device's system software shows a notification that makes me sure the crash was because the headset was out of available memory, if that was the case. (which I previously ran into months ago)

[electrum-bowie]

Can you try running a debug build on the device and see if it provides more useful information?

Will do that when I get the time.

[electrum-bowie]

Best possible workaround for now 😁

#447

[csciguy8]

One actionable thing we could do is put some exception handling (try catch) in Cesium3DTilesetImpl::Update, link.

It would give us a chance to catch any known types of exceptions and log. This might help distinguish out of memory problems vs other problems.

[meta-meta]

Are there any workarounds you can recommend? In Unity on the Quest, it's inevitable that spending enough time in the app will produce a crash due to running out of memory. We've attempted to hook into Unity's Application.lowMemory event and destroy/recreate the CesiumGeoreference and Cesium3DTileset. That seems to buy some time but in practice, Unity's event happens long before a crash would have happened so it becomes a UX nuisance and the app will still crash.

[kring]

@meta-meta the OP seemed to think the crash described in this issue was not related to running out of memory. In which case you're presumably running into a different problem. The first thing to try if you're running out of memory is to set the Maximum Cached Bytes property on the tileset to zero. If that doesn't help, please post to the community forum with as much detail as you can about your situation. In particular, what tilesets are you using?

[meta-meta]

Oh whoops, I missed that message. At any rate, between updating Unity to latest LTS and making efforts to reduce the amount of memory our app uses, we may have eliminated the crashes.

[electrum-bowie]

My app doesn't run out of memory, and isn't even close to doing so..

Stack trace:

`Thread 4043
#00 pc 000000000008b0a0 /apex/com.android.runtime/lib64/bionic/libc.so (abort+168) (BreakpadBuildId: 52a17d79c7474226ba96a3be5f0e981c0)
#1 pc 00000000022f4ed8 libil2cpp.so (libil2cpp.so (0x22f4ed8)) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#2 pc 00000000022f5028 libil2cpp.so (libil2cpp.so (0x22f5028)) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#3 pc 00000000022f4f90 libil2cpp.so (libil2cpp.so (0x22f4f90)) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#4 pc 00000000022f4f14 libil2cpp.so (std::terminate()+36) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#5 pc 00000000022f5138 libil2cpp.so (libil2cpp.so (0x22f5138)) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#6 pc 0000000000a62cc4 /data/app/~~ZP_082MwIodNHODKs5sHUA==/com.bowie.earthquest-zXNvxKN0uEIqq_lA6BzX9g==/lib/arm64/libCesiumForUnityNative-Runtime.so (std::__terminate(void ()())+12)
#7 pc 0000000000a63504 /data/app/~~ZP_082MwIodNHODKs5sHUA==/com.bowie.earthquest-zXNvxKN0uEIqq_lA6BzX9g==/lib/arm64/libCesiumForUnityNative-Runtime.so (__cxxabiv1::call_terminate(bool, _Unwind_Exception)+36)
#8 pc 0000000000a634a8 /data/app/~~ZP_082MwIodNHODKs5sHUA==/com.bowie.earthquest-zXNvxKN0uEIqq_lA6BzX9g==/lib/arm64/libCesiumForUnityNative-Runtime.so (__cxxabiv1::scan_eh_tab(__cxxabiv1::(anonymous namespace)::scan_results&, _Unwind_Action, bool, _Unwind_Exception*, _Unwind_Context*)+1508)
#9 pc 0000000000a62d8c /data/app/~~ZP_082MwIodNHODKs5sHUA==/com.bowie.earthquest-zXNvxKN0uEIqq_lA6BzX9g==/lib/arm64/libCesiumForUnityNative-Runtime.so (__gxx_personality_v0+104)
#10 pc 000000000230c3c8 libil2cpp.so (std::__terminate) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#11 pc 00000000022f4480 libil2cpp.so (__cxa_throw+116) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#12 pc 0000000002273424 libil2cpp.so (<__cxxabiv1::scan_eh_tab>) (BreakpadBuildId: 0cf42e90989c1d5200000000000000000)
#13 pc 00000000022734b8 libil2cpp.so (<__gxx_personality_v0>) (...

IMAGE WITH MORE INSIGHT

[electrum-bowie]

v1.12.0 Cesium for Unity

v1.7.1 returns the same crash type.

[electrum-bowie]

Unity 2023.1.0f1

[kring]

Thanks for the call stack, @electrum-bowie, but I don't think it adds any new insights for us on this one. Cesium3DTilesetImpl::Update is apparently throwing an exception (or perhaps causing an access violation?), but we don't know why, or even on what line. Some questions to help narrow it down a bit:

1. Can you reproduce this consistently? Or does it only happen randomly on end-user devices?
2. Can you reproduce it with the Cesium for Unity Samples project, or only in your own project? (I realize if the answer to the above is no, then this is likely to be no as well)
3. Can you reproduce it on any other devices, or only on Quest 2/3?