Skip to content

Comments

fix unnecessary synchronized block#27348

Closed
tinder-xli wants to merge 1 commit intoelastic:masterfrom
tinder-xli:master
Closed

fix unnecessary synchronized block#27348
tinder-xli wants to merge 1 commit intoelastic:masterfrom
tinder-xli:master

Conversation

@tinder-xli
Copy link
Contributor

During our perf test against elasticsearch, we noticed two synchronized block in the call stack of fetching doc values, which i think is necessary and cause a serious gc issue for us, especially when "size" param is large and fetch docvalue fields.

  1. Logger object in DocValuesIndexFieldData.java class
    Each doc value fetch will essentially create an instance of DocValuesIndexFieldData class, which will create a logger object in its constructor. However in PrefixLogger class, there is a synchronized block around marker which will block the initialization of DocValuesIndexFieldData class. We noticed a lot of thread block in PrefixLogger from JFR recording, which we believe is the root cause for super long ParNew GC:
    screen shot 2017-11-10 at 4 07 06 pm
    We noticed that among all the classes which extend DocValuesIndexFieldData, only SortedSetDVOrdinalsIndexFieldData class actually uses the logger, thus we are proposing moving the logger creation to this class.
    We also noticed that there are quite a few other places where loggers get initialized in constructor, e.g. BulkRequestHandler.java and AbstractComponent.java. We think it could also potentially be an issue if lots of instances of such classes are created in a short period of time and suggest revisit such use cases.

  2. getForField method in IndexFieldDataService.java
    There is a synchronized block for getting from fieldDataCaches map, which is unnecessary when cache != null, we suggest changing fieldDataCaches from HashMap to ConcurrentHashMap thus we don't need any explicit synchronized logic at all.
    We see similar behavior(threads waiting on getForField method) in our JFR too.
    screen shot 2017-11-10 at 4 23 39 pm

gradle test all passed in my local.

For reference, below is a sample query we used for testing:

{
  "stored_fields": "_none_",
  "docvalue_fields": ["user_number"],
  "size": 33000,
  "query": {
    "bool": {
    	 "must": [
    	   {
    	   	"match_all":{}
    	   }
    	 ]
    }
  }
}

@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@jasontedor
Copy link
Member

This looks like good work, but there are two separate changes here. Would you please split this into two PRs so the changes can be reviewed separately? The logging change can be integrated almost immediately but I would prefer a second set of eyes from @jpountz on the index field data service change.

@tinder-xli
Copy link
Contributor Author

@jasontedor as you suggested I'm going to create two separate PR. Closing this one

@tinder-xli tinder-xli closed this Nov 11, 2017
@tinder-xli
Copy link
Contributor Author

Two new PR created:
#27349
#27350

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants