Skip to content

KAFKA-6741: Disable Selector's idle connection timeout in testNetworkThreadTimeRecorded() test#4824

Merged
rajinisivaram merged 2 commits into
apache:trunkfrom
omkreddy:SSLTransport
Apr 5, 2018
Merged

KAFKA-6741: Disable Selector's idle connection timeout in testNetworkThreadTimeRecorded() test#4824
rajinisivaram merged 2 commits into
apache:trunkfrom
omkreddy:SSLTransport

Conversation

@omkreddy

@omkreddy omkreddy commented Apr 4, 2018

Copy link
Copy Markdown
Contributor

No description provided.

@omkreddy

omkreddy commented Apr 4, 2018 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@rajinisivaram trace logs shows that connection is getting closed due to idle time out. increased the idle connection timeout to 10secs for this test alone. Pls take a look.

[2018-04-05 00:21:35,390] TRACE About to close the idle connection from 0 due to being idle for 5170 millis (org.apache.kafka.common.network.Selector:639)
[2018-04-05 00:21:35,393] DEBUG Connection with /127.0.0.1 disconnected (org.apache.kafka.common.network.Selector:527)
java.io.IOException: Broken pipe
	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
	at org.apache.kafka.common.network.SslTransportLayer.flush(SslTransportLayer.java:211)
	at org.apache.kafka.common.network.SslTransportLayer.write(SslTransportLayer.java:644)
	at org.apache.kafka.common.network.SslTransportLayer.write(SslTransportLayer.java:677)
	at org.apache.kafka.common.network.SslTransportLayer.write(SslTransportLayer.java:702)
	at org.apache.kafka.common.network.ByteBufferSend.writeTo(ByteBufferSend.java:60)
	at org.apache.kafka.common.network.KafkaChannel.send(KafkaChannel.java:239)
	at org.apache.kafka.common.network.KafkaChannel.write(KafkaChannel.java:210)
	at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:509)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:424)
	at org.apache.kafka.common.network.NioEchoServer.run(NioEchoServer.java:145)

@omkreddy omkreddy changed the title KAFKA-6741: Increase Selector's idle connection timeout of testNetwokkThreadTimeRecorded() test KAFKA-6741: Increase Selector's idle connection timeout of testNetworkThreadTimeRecorded() test Apr 4, 2018
hachikuji
hachikuji reviewed Apr 5, 2018
View reviewed changes
Comment thread clients/src/test/java/org/apache/kafka/common/network/SslTransportLayerTest.java Outdated
public void testNetworkThreadTimeRecorded() throws Exception {
selector.close();
this.selector = new Selector(NetworkReceive.UNLIMITED, 5000, new Metrics(), Time.SYSTEM,
this.selector = new Selector(NetworkReceive.UNLIMITED, 10000, new Metrics(), Time.SYSTEM,

@hachikuji hachikuji Apr 5, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to disable idle timeout with NO_IDLE_TIMEOUT_MS for this test? Do we want it to be at play?

@omkreddy omkreddy Apr 5, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, We can set timeout to NO_IDLE_TIMEOUT_MS for this test.

@omkreddy omkreddy changed the title KAFKA-6741: Increase Selector's idle connection timeout of testNetworkThreadTimeRecorded() test KAFKA-6741: Disable Selector's idle connection timeout in testNetworkThreadTimeRecorded() test Apr 5, 2018
@rajinisivaram

rajinisivaram commented Apr 5, 2018

Copy link
Copy Markdown
Contributor

@omkreddy Is the test failing consistently without this fix and passing consistently with this fix? The test is waiting for a receive and if it fails to receive, then it would hit idle timeout since there is no other traffic, but I am not sure idle timeout is the cause of the failure. There could be some other timing issue that is causing the receive to fail.

@omkreddy

omkreddy commented Apr 5, 2018

Copy link
Copy Markdown
Contributor Author

@rajinisivaram Yes, test is failing most of the times without this patch and passing consistently with the fix. ran 100 times without any error. so i thought, may be system slowness/load is causing the test to fail.

rajinisivaram
rajinisivaram approved these changes Apr 5, 2018
View reviewed changes

@rajinisivaram rajinisivaram left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@omkreddy Thanks for the PR, LGTM.

@rajinisivaram

rajinisivaram commented Apr 5, 2018

Copy link
Copy Markdown
Contributor

@omkreddy Thanks for the PR, merging to trunk.

@rajinisivaram
rajinisivaram merged commit 77c79df into apache:trunk Apr 5, 2018
@omkreddy
omkreddy deleted the SSLTransport branch July 3, 2018 15:44
ying-zheng pushed a commit to ying-zheng/kafka that referenced this pull request Jul 6, 2018
@omkreddy @ying-zheng
KAFKA-6741: Disable Selector's idle connection timeout in testNetwork…
1bc993a 
…ThreadTimeRecorded() test (apache#4824)

Reviewers: Jason Gustafson <jason@confluent.io>, Rajini Sivaram <rajinisivaram@googlemail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hachikuji hachikuji hachikuji left review comments
@rajinisivaram rajinisivaram rajinisivaram approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@omkreddy @rajinisivaram @hachikuji