RxJava2Debug causing `StackOverflowError` on too complex `CompositeException`

[mkoslacz]

Hi,

Recently I have encountered a problem using RxJava2Debug on Android.

In my project I sometimes got OOM errors when trying to log too complex CompositeExceptions from external components using Android Log.e method. To avoid it I strip these exceptions to have 5 original exceptions max using stripCompositeException method included in the gist I linked at the end of this issue.

In the regular setup this approach works correctly, but using RxJava2Debug, on less performant devices and older apis that have older memory management algorithms (in my case API 17) I got uncaught StackOverflowError that crashes my app. I include the error I'm encountering in the attached test case:

01-12 15:57:08.835 3908-3908/? E/AndroidRuntime: FATAL EXCEPTION: main
                                                 java.lang.StackOverflowError
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:57)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onSubscribe(ObservableDoOnEach.java:73)
                                                     at io.reactivex.internal.observers.BasicFuseableObserver.onSubscribe(BasicFuseableObserver.java:66)
                                                     at io.reactivex.internal.operators.observable.ObservableJust.subscribeActual(ObservableJust.java:34)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at hu.akarnokd.rxjava2.debug.ObservableOnAssemblyScalarCallable.subscribeActual(ObservableOnAssemblyScalarCallable.java:41)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach.subscribeActual(ObservableDoOnEach.java:42)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at hu.akarnokd.rxjava2.debug.ObservableOnAssembly.subscribeActual(ObservableOnAssembly.java:41)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach.subscribeActual(ObservableDoOnEach.java:42)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at hu.akarnokd.rxjava2.debug.ObservableOnAssembly.subscribeActual(ObservableOnAssembly.java:41)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at io.reactivex.internal.operators.observable.ObservableDoOnEach.subscribeActual(ObservableDoOnEach.java:42)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                     at hu.akarnokd.rxjava2.debug.ObservableOnAssembly.subscribeActual(ObservableOnAssembly.java:41)
                                                     at io.reactivex.Observable.subscribe(Observable.java:10903)
                                                 	at io.reactivex.internal.operators.observable.ObservableDoOnEach.subscribeActual(ObservableDoOnEach.java:4

Although, on API 25 RxJava2Debug won't crash the app and will log the exception correctly, but still, in such case the printed stacktrace is so huge that it fills up whole Android Logcat console and eventually breaks it with read: unexpected EOF! what also makes logging useless.

Is it possible to create some constraints in RxJava2Debug that would prevent such issues?

I have created the test case here. Let me know if you wouldn't be able to reproduce the issue.

BTW - It's the exactly the same case I have reported in Traceur, which is an another attempt to do what RxJava2Debug does: T-Spoon/Traceur#4

[akaita]

I love that test case :)

Indeed, RxJava2Debug will not drop any of the causes. That's fine for the vast majority of cases, but could be an issue for overly complex scenarios. I think RxJava2Debug should support those scenarios, though.

I'll give it a thought.
Give me a shout if you have some solution or expected-behaviour in mind; input is always welcome 👍

[mkoslacz]

Yeah, I agree with you that those scenarios should be supported, especially because we had this kind of crashes on production already ;)

It seems that the most straightforward behavior would be to simply limit the included causes count to some reasonable number to avoid this kind of crashes. Anyway, I'll think about it and let you know if something will occur to me.

[mkoslacz]

Well, sorry, I don't have a better idea for now. What do you think about my previous proposal?

[akaita]

So far, I see three issues:

1. StackOverflow while creating the enhanced stack
2. OOM while creating the enhanced stack
3. StackOverflow while printing using Android.Log in API 16

I understand your proposal would fix the third issue. But the that is really not a RxJava2Debug issue, but an Android issue. I don't think that is this library's responsibility to solve.

The other two issue I want to find a fix for.
The problem is that the way it is currently coded, each RxJava2 operation is treated with no knowledge of what other operations previously did. I want to think about a way to solve the issue in a way that's not specific to RxJava2Debug's use case, so I can then contribute back upstream.

I'm still thinking about the best way to solve this 🤔

[akaita]

just an update for any observers of the issue:

I'm scratching my head about how to support streams with over 95 operators in Android's VM. Can't find a way, though.
I need to continue wrapping observers so I can override onError implementations to hand the assembled stacktrace over...

I'm thinking that maybe giving the ability to disable RxJava2Debug just for a specific stream could be the way to go. Sadly, still gotta keep thinking about it.

[RikkiGibson]

Would the idea be that you would opt-out of assembly tracing on a particular large workflow to avoid the StackOverFlowException?

[akaita]

Yeah. I don't particularly like the idea, but that's the best I got now.
I'm banking on the assumption that there is just a couple of really extensive streams in the app. Is that correct?

[RikkiGibson]

The trouble is, I'd guess that those streams are the most likely to encounter a problem you'll want a trace for. I'm very interested in finding some kind of a solution for debugging async Rx workflows as well so I'll let you know if I come up with anything.

[akaita]

True. The more complex the stream is, the higher the chances of misbehaviour.
I see no other option than:

• a workaround to avoid the tools itself generating an issue
• refactoring streams to keep them simple (50 operators are fine for older devices)

Could you dig up one of the "real-world" failing streams and check how many operators it is composed of?

[IDan14]

Hi,
I have encountered the same error on some specific device:

Caused by java.lang.StackOverflowError: stack size 8MB
       at java.lang.StringBuffer.append(StringBuffer.java:278)
	…

A solution I think would be to have in RxJavaAssemblyException public Throwable appendLast(Throwable ex) method only append first (N) exception(s) and be able to optionally enable this from RxJava2Debug class.
Is this library still in active development?
Thank you,
Dan I.

[akaita]

Hi @IDan14 , it was on hold until I got some more time.
I think I already tried your proposal and didn't work. I think the problem was really rxjava2-extensions, which I couldn't figure out how to solve.

I'll give your proposal a go, because tbh my memories about this are a bit fuzzy