Skip to content

Conversation

@original-brownbear
Copy link
Contributor

  • The static threadpool leaks a lot of memory in these tests because it prevents things like the connect listeners from org.elasticsearch.transport.TcpTransport#initiateConnection
    to be GCed between tests (since they keep being referenced by the threadpool) which in turn reference channels and their underlying buffers
    • I could not find any slowdown in executing these tests from this change, if anything they are slightly faster now on my machine
  • Relates TransportTasksActionTests testTaskLevelActionFailures #36906 (which may be caused by slowness from leaking memory and also becomes testable in a loop by this change)

* The static threadpool leaks a lot of memory in these tests because it prevents things like the connect listeners from `org.elasticsearch.transport.TcpTransport#initiateConnection`
to be GCed between tests (since they keep being referenced by the threadpool) which in turn reference channels and their underlying buffers
  * I could not find any slowdown in executing these tests from this change, if anything they are slightly faster now on my machine
* Relates #36906 (which may be caused by slowness from leaking memory and also becomes testable in a loop by this change)
@original-brownbear original-brownbear added >test Issues or PRs that are addressing/adding tests :Core/Infra/Core Core issues without another label v7.0.0 v6.7.0 labels Dec 24, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

public abstract class TaskManagerTestCase extends ESTestCase {

protected static ThreadPool threadPool;
public static final Settings CLUSTER_SETTINGS = Settings.builder().put("cluster.name", "test-cluster").build();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was just unused

@original-brownbear
Copy link
Contributor Author

Jenkins run gradle build tests 2 please

@original-brownbear
Copy link
Contributor Author

Jenkins run gradle build tests 1 please

Copy link
Member

@danielmitterdorfer danielmitterdorfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@original-brownbear original-brownbear merged commit 675ea4c into elastic:master Jan 3, 2019
@original-brownbear original-brownbear deleted the 36906 branch January 3, 2019 14:19
@original-brownbear
Copy link
Contributor Author

@danielmitterdorfer thanks!

@romseygeek
Copy link
Contributor

This appears to be causing test failures in CancellableTasksTests in master. The following seed reproduces:

./gradlew :server:unitTest \
  -Dtests.seed=FF91D1A6CF41E535 \
  -Dtests.class=org.elasticsearch.action.admin.cluster.node.tasks.CancellableTasksTests \
  -Dtests.method="testChildTasksCancellation" \
  -Dtests.security.manager=true \
  -Dtests.locale=und \
  -Dtests.timezone=America/Indiana/Vevay

@original-brownbear
Copy link
Contributor Author

@romseygeek looking into it.

@original-brownbear
Copy link
Contributor Author

original-brownbear commented Jan 3, 2019

@romseygeek Fixed in #37123

original-brownbear added a commit that referenced this pull request Jan 3, 2019
* If the threadpool gets shut down before the testnodes we run into an error => fixed by moving to single `After` method
* Relates #36976
original-brownbear added a commit that referenced this pull request Jan 4, 2019
* The static threadpool leaks a lot of memory in these tests because it prevents things like the connect listeners from `org.elasticsearch.transport.TcpTransport#initiateConnection`
to be GCed between tests (since they keep being referenced by the threadpool) which in turn reference channels and their underlying buffers
  * I could not find any slowdown in executing these tests from this change, if anything they are slightly faster now on my machine
* Relates #36906 (which may be caused by slowness from leaking memory and also becomes testable in a loop by this change)
original-brownbear added a commit that referenced this pull request Jan 4, 2019
* If the threadpool gets shut down before the testnodes we run into an error => fixed by moving to single `After` method
* Relates #36976
@jimczi jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Core Core issues without another label >test Issues or PRs that are addressing/adding tests v6.7.0 v7.0.0-beta1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants