JsonSyntaxException when Nakadi's response is not JSON

[ePaul]

Current behavior

When Nakadi replies with an error which can't be parsed as application/problem+json, then Nakadi-Java throws just a NakadiException wrapping a JsonSyntaxException, without any trace of the original error message by Nakadi.

 17:41:41.909 2020-07-09 15:41:41.909 ERROR 7 --- [-java-compute-0] c.z.p.nakadi.AbstractEventObserver       : An error occurred while committing subscription cursor: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $;  (400)
17:41:41.909 nakadi.NakadiException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $;  (400)
 17:41:41.909 	at nakadi.SubscriptionOffsetCheckpointer.checkpointInner(SubscriptionOffsetCheckpointer.java:149) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.SubscriptionOffsetCheckpointer.checkpoint(SubscriptionOffsetCheckpointer.java:95) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.SubscriptionOffsetObserver.checkpoint(SubscriptionOffsetObserver.java:28) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.SubscriptionOffsetObserver.onNext(SubscriptionOffsetObserver.java:21) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at com.zalando.po_comparator.nakadi.AbstractEventObserver.commitSubscriptionCursor(AbstractEventObserver.java:71) ~[classes!/:na]
 17:41:41.909 	at com.zalando.po_comparator.nakadi.AbstractEventObserver.onNext(AbstractEventObserver.java:47) ~[classes!/:na]
 17:41:41.909 	at nakadi.StreamBatchRecordSubscriber.onNext(StreamBatchRecordSubscriber.java:47) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.StreamBatchRecordSubscriber.onNext(StreamBatchRecordSubscriber.java:7) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.shadow.io.reactivex.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:400) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.shadow.io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.909 	at nakadi.shadow.io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:260) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:225) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
 17:41:41.910 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
 17:41:41.910 	at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
 17:41:41.910 Caused by: nakadi.shadow.com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $
 17:41:41.910 	at nakadi.shadow.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:224) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.com.google.gson.Gson.fromJson(Gson.java:887) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.com.google.gson.Gson.fromJson(Gson.java:852) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.com.google.gson.Gson.fromJson(Gson.java:801) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.com.google.gson.Gson.fromJson(Gson.java:773) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.GsonSupport.fromJson(GsonSupport.java:73) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.ProblemSupport.toProblem(ProblemSupport.java:9) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.OkHttpResource.handleError(OkHttpResource.java:314) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.OkHttpResource.throwIfError(OkHttpResource.java:301) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.OkHttpResource.requestThrowingInner(OkHttpResource.java:168) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.OkHttpResource.lambda$requestThrowing$3(OkHttpResource.java:123) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.io.reactivex.internal.operators.observable.ObservableDefer.subscribeActual(ObservableDefer.java:32) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.io.reactivex.Observable.subscribe(Observable.java:10842) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.io.reactivex.Observable.blockingFirst(Observable.java:4727) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.OkHttpResource.requestThrowing(OkHttpResource.java:123) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.SubscriptionResourceReal.lambda$checkpoint$3(SubscriptionResourceReal.java:204) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.SubscriptionResourceReal.timed(SubscriptionResourceReal.java:43) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.SubscriptionResourceReal.checkpoint(SubscriptionResourceReal.java:204) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.SubscriptionResourceReal.checkpoint(SubscriptionResourceReal.java:145) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.SubscriptionOffsetCheckpointer.checkpointInner(SubscriptionOffsetCheckpointer.java:156) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.SubscriptionOffsetCheckpointer.checkpointInner(SubscriptionOffsetCheckpointer.java:105) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	... 14 common frames omitted
 17:41:41.910 Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $
 17:41:41.910 	at nakadi.shadow.com.google.gson.stream.JsonReader.beginObject(JsonReader.java:385) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	at nakadi.shadow.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:213) ~[nakadi-java-client-0.14.0.jar!/:0.14.0]
 17:41:41.910 	... 34 common frames omitted

(This might currently be happening due to rate limiting on Nakadi-Staging at Zalando.)

Expected behavior

• When the error response does not have content-type application/problem+json, don't try to parse it as such.
• When there is no body, have an error message which includes at least the HTTP status.
• When the error response body is non-empty and can't be parsed, still include the body (or the start of it, if it's long) in any error message thrown.

[dehora]

When Nakadi replies with an error which can't be parsed as application/problem+json,

@adyach is there a bug filed against Nakadi for this? The server shouldn't be emitting junk entities.

[adyach]

@dehora it looks like it can not be covered by Nakadi, because response can be made from load balancer for example which can be arbitrary text.