Allow routing of bulk API actions #631
Conversation
The elasticsearch bulk API allows for each item to have a routing value which helps prevent bulk actions from being broadcast to all shards. http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-routing
If all ids to be deleted during a bulk API request are routed to the same shard allow that routing value to be set so that the delete command is not broadcast to all shards.
| @@ -527,6 +528,9 @@ public function deleteIds(array $ids, $index, $type) | |||
| $action = new Action(Action::OP_TYPE_DELETE); | |||
| $action->setId($id); | |||
|
|
|||
| if (!empty($routing)) | |||
There was a problem hiding this comment.
Can you please use { } also if it only one line inside the if clause?
|
Thx for the addition. For the test changes, I would suggest to add new additional tests instead of changing the existing tests (if not needed). So we can make sure, the old behaviour still exists, but a new one was added. Also instead of having large tests, we can have very small unit tests that only test one thing (if possible). |
Allow routing of bulk API actions
|
Thanks @ruflin! I didn't get a chance to cleanup the tests like you suggested yesterday, let me know if you still want new tests for this change as opposed to the changed tests. |
|
@xyu Yes, it would be very nice if you could also update the tests and make an new pull request. I merged it yesterday as other pull requests came in an I wanted to make sure not to have any conflicts. |
The elasticsearch bulk API allows for each item to have a routing value. This is especially helpful for bulk delete calls, which when set helps prevent those actions from being broadcast to all shards needlessly.
This change adds a optional parameter to the
deleteIds()method which when set will route all deletes for the given id with the given routing value.I also thought about perhaps allowing the ids array to be an array of id / routing key pairs however that seemed needlessly complicated and users can simply call
deleteIds()multiple times with grouping ids with the same routing value if needed. (For our use case when we are bulk deleting by ids we are bulk deleting a bunch of related documents which are routed to the same shard.)