java.lang.NumberFormatException: For input string: "1.547993531819E9 #1023

oripwk · 2019-01-20T14:30:26Z

When using amazon-kinesis-client, I'm getting

Unexpected exception was thrown. This could probably be an issue or a bug. Please search for the exception/error online to check what is going on. If the issue persists or is a recurring problem, feel free to open an issue on, https://github.com/awslabs/amazon-kinesis-client.

Current Behavior

The ShardRecordProcessor fails when reading records from Kinesis stream with

java.lang.NumberFormatException: For input string: "1.547993531819E9"

Stack trace

java.lang.NumberFormatException: For input string: "1.547993531819E9"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_181]
	at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_181]
	at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_181]
	at software.amazon.awssdk.utils.DateUtils.parseUnixTimestampMillisInstant(DateUtils.java:146) ~[utils-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.core.StringToInstant.lambda$safeParseDate$0(StringToInstant.java:72) ~[protocol-core-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.core.StringToInstant.convert(StringToInstant.java:56) ~[protocol-core-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.core.StringToInstant.convert(StringToInstant.java:32) ~[protocol-core-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller$SimpleTypeJsonUnmarshaller.unmarshall(JsonProtocolUnmarshaller.java:160) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallStructured(JsonProtocolUnmarshaller.java:210) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallStructured(JsonProtocolUnmarshaller.java:114) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.lambda$unmarshallList$2(JsonProtocolUnmarshaller.java:143) ~[aws-json-protocol-2.3.2.jar:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_181]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_181]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_181]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_181]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_181]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_181]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_181]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallList(JsonProtocolUnmarshaller.java:145) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshallStructured(JsonProtocolUnmarshaller.java:210) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshall(JsonProtocolUnmarshaller.java:197) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonProtocolUnmarshaller.unmarshall(JsonProtocolUnmarshaller.java:168) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:79) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:36) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonResponseHandler.handle(AwsJsonResponseHandler.java:43) ~[aws-json-protocol-2.3.2.jar:?]
	at software.amazon.awssdk.core.client.handler.BaseClientHandler.lambda$interceptorCalling$2(BaseClientHandler.java:133) ~[sdk-core-2.3.2.jar:?]
	at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:40) ~[sdk-core-2.3.2.jar:?]
	at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:28) ~[sdk-core-2.3.2.jar:?]
	at software.amazon.awssdk.core.internal.http.async.SyncResponseHandlerAdapter.lambda$prepare$0(SyncResponseHandlerAdapter.java:85) ~[sdk-core-2.3.2.jar:?]
	at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952) ~[?:1.8.0_181]
	at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_181]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_181]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_181]
	at software.amazon.awssdk.core.internal.http.async.SyncResponseHandlerAdapter$BaosSubscriber.onComplete(SyncResponseHandlerAdapter.java:127) ~[sdk-core-2.3.2.jar:?]
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.runAndLogError(ResponseHandler.java:158) ~[netty-nio-client-2.3.2.jar:?]
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.access$700(ResponseHandler.java:64) ~[netty-nio-client-2.3.2.jar:?]
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$PublisherAdapter$1.onComplete(ResponseHandler.java:268) ~[netty-nio-client-2.3.2.jar:?]
	at com.typesafe.netty.HandlerPublisher.complete(HandlerPublisher.java:408) ~[netty-reactive-streams-2.0.0.jar:?]
	at com.typesafe.netty.HandlerPublisher.handlerRemoved(HandlerPublisher.java:395) ~[netty-reactive-streams-2.0.0.jar:?]
	at io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:670) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:505) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:451) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at com.typesafe.netty.http.HttpStreamsHandler.removeHandlerIfActive(HttpStreamsHandler.java:328) ~[netty-reactive-streams-http-2.0.0.jar:?]
	at com.typesafe.netty.http.HttpStreamsHandler.handleReadHttpContent(HttpStreamsHandler.java:189) ~[netty-reactive-streams-http-2.0.0.jar:?]
	at com.typesafe.netty.http.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:165) ~[netty-reactive-streams-http-2.0.0.jar:?]
	at com.typesafe.netty.http.HttpStreamsClientHandler.channelRead(HttpStreamsClientHandler.java:148) ~[netty-reactive-streams-http-2.0.0.jar:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at software.amazon.awssdk.http.nio.netty.internal.FutureCancelHandler.channelRead0(FutureCancelHandler.java:35) ~[netty-nio-client-2.3.2.jar:?]
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:591) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:508) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]

Possible Solution

No possible solution

Steps to Reproduce (for bugs)

Context

This issue may be related: #970

Your Environment

AWS Java SDK version used:
- amazon-kinesis-client:2.1.0
- kinesis:2.3.2
- dynamodb:2.3.2
- cloudwatch:2.3.2
JDK version used: 1.8
Operating System and version: Alpine

The text was updated successfully, but these errors were encountered:

oripwk · 2019-01-20T16:06:55Z

This is clearly a bug.

According to Kinesis docs, ApproximateArrivalTimestamp can be written in exponential notation, and in this specific case, it doesn't event evaluate to a long, but to double:

1.441215410867E9 == 1441215410.867

The stack trace shows that the parseLong(…) method is used, and there it fails.

This is a very critical bug and we'd appreciate if you can issue a fix.

Thanks

Edit

The root cause is that TimestampFormatTrait.Format evaluates to UNIX_TIMESTAMP_MILLIS instead of UNIX_TIMESTAMP. If it would have been the latter, the parseUnixTimestampInstant() method would be used instead of parseUnixTimestampMillisInstant() and it would have worked.

aesteve · 2019-01-21T08:09:16Z

Maybe it's related to the bug I reported in #1021 ? What do you think ?

oripwk · 2019-01-21T08:29:17Z

@aesteve it might be the case, since we also disabled CBOR for localstack.

aesteve · 2019-01-22T07:31:50Z

I'll add a comment on #1021 to add your EDIT, it's probably gonna help fixing the issue.

oripwk · 2019-01-22T11:08:41Z

@zoewangg

Sorry to nudge, but this is critical. If someone can look into it we'll appreciate.

zoewangg · 2019-01-22T17:58:10Z

Thank you for reporting. I'm working on the fix right now.

zoewangg · 2019-01-22T23:51:22Z

The fix is merged and will be included in the next release.

zoewangg · 2019-01-24T18:58:51Z

The fix has been released in 2.3.7. Could you try with latest version and verify?

Feel free to reopen if you have further question.

oripwk · 2019-01-24T19:08:59Z

We've deployed the fix 12 hours ago and there haven't been any NumberFormatExceptions since. It depends on traffic and it always happened sporadically so I'm not 100% sure. But so far looks good.

Thank you!

aesteve · 2019-01-25T11:22:21Z

Can confirm my integration tests with localstack are all green now.

Thanks for your fix.

aesteve mentioned this issue Jan 21, 2019

KinesisAsyncClient doesn't work with CBOR disabled (since 2.3.0 I guess) #1021

Closed

zoewangg added the bug This issue is a bug. label Jan 22, 2019

oripwk mentioned this issue Jan 22, 2019

java.lang.NumberFormatException: For input string: "1.547993531819E9 awslabs/amazon-kinesis-client#485

Closed

zoewangg mentioned this issue Jan 22, 2019

Fixed the time marshalling issue when CBOR is disabled. See #1023 #1027

Merged

13 tasks

zoewangg added a commit that referenced this issue Jan 22, 2019

Fixed the time marshalling issue when CBOR is disabled. See #1023

b4f795d

zoewangg added the awaiting-review label Jan 22, 2019

zoewangg added a commit that referenced this issue Jan 22, 2019

Fixed the time marshalling issue when CBOR is disabled. See #1023

ec0d95c

zoewangg added awaiting-release and removed awaiting-review labels Jan 22, 2019

zoewangg closed this as completed Jan 24, 2019

justnance added pending-release This issue will be fixed by an approved PR that hasn't been released yet. and removed awaiting-release labels Apr 19, 2019

danielmewes mentioned this issue Jan 17, 2023

Json Unmarshal Exception with AWS java SDK 2.X mhart/kinesalite#108

Open

debora-ito mentioned this issue Jun 28, 2024

Kinesis::GetShardIterator with AT_TIMESTAMP does not work when CBOR is disabled aws/aws-sdk-java#2588

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java.lang.NumberFormatException: For input string: "1.547993531819E9 #1023

java.lang.NumberFormatException: For input string: "1.547993531819E9 #1023

oripwk commented Jan 20, 2019 •

edited

Loading

oripwk commented Jan 20, 2019 •

edited

Loading

aesteve commented Jan 21, 2019

oripwk commented Jan 21, 2019

aesteve commented Jan 22, 2019

oripwk commented Jan 22, 2019

zoewangg commented Jan 22, 2019

zoewangg commented Jan 22, 2019

zoewangg commented Jan 24, 2019

oripwk commented Jan 24, 2019

aesteve commented Jan 25, 2019

java.lang.NumberFormatException: For input string: "1.547993531819E9 #1023

java.lang.NumberFormatException: For input string: "1.547993531819E9 #1023

Comments

oripwk commented Jan 20, 2019 • edited Loading

Current Behavior

Stack trace

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

oripwk commented Jan 20, 2019 • edited Loading

Edit

aesteve commented Jan 21, 2019

oripwk commented Jan 21, 2019

aesteve commented Jan 22, 2019

oripwk commented Jan 22, 2019

zoewangg commented Jan 22, 2019

zoewangg commented Jan 22, 2019

zoewangg commented Jan 24, 2019

oripwk commented Jan 24, 2019

aesteve commented Jan 25, 2019

oripwk commented Jan 20, 2019 •

edited

Loading

oripwk commented Jan 20, 2019 •

edited

Loading