Upgrade to Elasticsearch 5.0 #1184

ruflin · 2016-10-11T06:49:36Z

* Replace remove count search type by query_then_fetch * Replace optimize by forcemerge as optimize was removed * Rename getMock() to createMock() to be compatible with newer phpunit versions * Convert string mapping to text * Change delete-by-query implementation to use 5.0 api * Remove delete-by-query plugin checks as now installed by default * Change script implementation to new 5.0 implementation * Simplify update-document * Update test environment to use self built elasticsearch instance * Improve node info fetching to use new format * Skip plugin tests if no plugin available * Fix list of tests * Apply liniting * Fix script tests for new script structure * Remove search_type from scroll as not needed anymore Part of #1184

* Replace remove count search type by query_then_fetch * Replace optimize by forcemerge as optimize was removed * Rename getMock() to createMock() to be compatible with newer phpunit versions * Convert string mapping to text * Change delete-by-query implementation to use 5.0 api * Remove delete-by-query plugin checks as now installed by default * Change script implementation to new 5.0 implementation * Simplify update-document * Update test environment to use self built elasticsearch instance * Improve node info fetching to use new format * Skip plugin tests if no plugin available * Fix list of tests * Apply liniting * Fix script tests for new script structure * Remove search_type from scroll as not needed anymore * Update README.md with new dependency Part of #1184

In this first step, all tests are checked which are not compatible with elasticsearch 5.0. Either the tests are directly fixed or they are temporarely skipped and have to be cleaned up later. In general it seems as most of the things will just keep working with elasticsearch 5.0 except for the things that were changed / removed in elasticsearch 5.0 anyways (like mapping changes from string to text / keyword). * Replace remove count search type by query_then_fetch * Replace optimize by forcemerge as optimize was removed * Rename getMock() to createMock() to be compatible with newer phpunit versions * Convert string mapping to text * Change delete-by-query implementation to use 5.0 api * Remove delete-by-query plugin checks as now installed by default * Change script implementation to new 5.0 implementation * Simplify update-document * Update test environment to use self built elasticsearch instance * Improve node info fetching to use new format * Skip plugin tests if no plugin available * Fix list of tests * Apply liniting * Fix script tests for new script structure * Remove search_type from scroll as not needed anymore * Update README.md with new dependency Part of #1184

* Remove outdated QueryBuilder. Update 2.4 query builder to only use queries * Remove deprecated Script*.php classes Part of #1184

Zyqsempai · 2016-10-18T08:51:33Z

Hi there, i will be glad to help you with this migration, please give me some short description, for example for "Remove image plugin support", do you want to completely delete it from project? Or give some other task!

ruflin · 2016-10-18T18:32:36Z

@Zyqsempai Great to hear. What I already did is removing all the filters and skip all the tests which do not pass at the moment. I added a message to each of this tests on why they don't pass (see https://travis-ci.org/ruflin/Elastica/jobs/167274669#L2135). We need to go through each of this test an figure out how it should be fixed. Most of them should be an easy fix.

To start simple: I think the image plugin is not under active development any more. So we should remove this one completely from the code and tests.

Zyqsempai · 2016-10-19T06:39:13Z

ok then i will start with image plugin a then move on to tests

- Remove Elastica/Query/Image.php - Remove test/Elastica/Query/ImageTest.php - Changelog update

This commit is for Travis rebuild.

Zyqsempai · 2016-10-19T13:57:27Z

So, what about this type of skipped tests, "script.inline is not enabled. This is required for this test", i checked your elasticsearch.yml and i found this "script.inline: on" i also tried to add something like this "script.engine.groovy.inline.aggs: on" and no results, do you know source of this problem?

Zyqsempai · 2016-10-19T14:15:36Z

Also, this one "Skipped because of 5.0: unknown setting [index.merge.policy.merge_factor] did you mean [index.merge.policy.max_merge_at_once]" this is strange because according to this https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_20_setting_changes.html#_merge_and_merge_throttling_settings , this error mast appear after 2.4 version migration, anyway it seams like we can't fix it for Version > 2.4

- Remove Elastica/Query/Filtered.php - Remove test/Elastica/Query/FilteredTest.php - Remove all references for filtered query from code - Changelog update

…ce [geo_distance_range] queries with geo distance sorting.

ruflin · 2016-10-24T07:06:31Z

@Zyqsempai Sorry for the delay in answering, was on vacation last week. About the issues above: Didn't check them in detail yet. For some option it can be that they don't exist anymore and we have to remove them or rename them.

- Remove Elastica/Query/Image.php - Remove test/Elastica/Query/ImageTest.php - Changelog update

…o_distance_range] queries with geo distance sorting. (#1191)

- Remove Elastica/Query/Filtered.php - Remove test/Elastica/Query/FilteredTest.php - Remove all references for filtered query from code - Changelog update

p365labs · 2016-10-24T14:33:20Z

Hi @ruflin regarding this error
I have a look here https://www.elastic.co/guide/en/elasticsearch/reference/1.7/index-modules-merge.html, seems that merge_factor has been removed and the PolicyFactor has been deprecated since 1.6 elasticsearch version. I had a look at source code of ES and in ES5.0\ES5.x\master this feature is not present.
I've opened a PR --> #1192 for this

p365labs · 2016-10-25T04:47:22Z

dear @ruflin and @Zyqsempai I'm trying to help you too, if I can :) now I'm working on remove all the deprecations. I would like to understand better another point of this issue : "Double check which PHP versions should be supported" how do you want to deal with it ? If you have a look at here http://php.net/supported-versions.php, php 5.4 and 5.5 has been dropped active versions are 7.0 and 5.6 (active support will end januare the 2017). I will drop 5.4 for sure and evaluate dropping also 5.5, but leaving 5.6. What do u guess ?

ruflin · 2016-10-25T08:01:03Z

The above topics have been discussed here: https://gitter.im/ruflin/Elastica?at=580c7bc4305a10335db2114f

…ies (ruflin#1184)

…ies (#1184) (#1208)

…ields instead (ruflin#1184)

* fielddata_fields has been deprecated in ES5, use parameter docvalue_fields instead (#1184) * Fix Set fielddata=true on [answer] (#1184)

…n suggester (ruflin#1184)

…m ElasticSearch 5.0 - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0

…n suggester (ruflin#1184)

…n suggester (#1184) (#1213)

…apping 'store' (ruflin#1184)

…apping 'store' (#1184) (#1215)

…for store instaed yes/no (ruflin#1184)

…for store instaed yes/no (#1184) (#1218)

…1220)

…m ElasticSearch 5.0 - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0

* fielddata_fields has been deprecated in ES5, use parameter docvalue_fields instead (ruflin#1184) * Fix Set fielddata=true on [answer] (ruflin#1184) Change to official docker image (ruflin#1210) * Switch test environment to official docker image. * Disable tests with multiple elasticsearch instances. Update PHPUnit to the most recent version (ruflin#1211) Prepare release 5.0.0-beta1 (ruflin#1212) updated CompletionTest, payload and output are removed from completion suggester (ruflin#1184) (ruflin#1213) store_fields it will no longer extract values from the _source, use mapping 'store' (ruflin#1184) (ruflin#1215) update tests : use text or keyword instead of string and use boolean for store instaed yes/no (ruflin#1184) (ruflin#1218) remove shutdown Node|Cluster as _shutdown has been removed (ruflin#1184) (ruflin#1220) Squashed commits - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0 - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0

ruflin mentioned this issue Oct 11, 2016

First step to upgrade Elastic to elasticsearch 5.0 #1114

Merged

ruflin added a commit that referenced this issue Oct 12, 2016

Remove filters from Elastica as filers are removed in ES 5.0

e5786fa

* Remove outdated QueryBuilder. Update 2.4 query builder to only use queries * Remove deprecated Script*.php classes Part of #1184

ruflin added a commit that referenced this issue Oct 12, 2016

Remove filters from Elastica as filers are removed in ES 5.0

60a995a

* Remove outdated QueryBuilder. Update 2.4 query builder to only use queries * Remove deprecated Script*.php classes Part of #1184

ruflin mentioned this issue Oct 12, 2016

Remove filters from Elastica as filers are removed in ES 5.0 #1186

Merged

ruflin added a commit that referenced this issue Oct 13, 2016

Remove filters from Elastica as filers are removed in ES 5.0 (#1186)

00e097c

* Remove outdated QueryBuilder. Update 2.4 query builder to only use queries * Remove deprecated Script*.php classes Part of #1184

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Oct 19, 2016

ruflin#1184 Remove image plugin:

917c9a1

- Remove Elastica/Query/Image.php - Remove test/Elastica/Query/ImageTest.php - Changelog update

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Oct 19, 2016

ruflin#1184 Remove image plugin:

6775b9f

This commit is for Travis rebuild.

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Oct 19, 2016

ruflin#1184 Remove Filtered query:

df22a11

- Remove Elastica/Query/Filtered.php - Remove test/Elastica/Query/FilteredTest.php - Remove all references for filtered query from code - Changelog update

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Oct 21, 2016

ruflin#1184 Fix for skipped GeoDistanceRangeTest::testGeoPoint, repla…

186bdc4

…ce [geo_distance_range] queries with geo distance sorting.

ruflin pushed a commit that referenced this issue Oct 24, 2016

#1184 Remove image plugin: (#1188)

2e606f2

- Remove Elastica/Query/Image.php - Remove test/Elastica/Query/ImageTest.php - Changelog update

ruflin pushed a commit that referenced this issue Oct 24, 2016

#1184 Fix for skipped GeoDistanceRangeTest::testGeoPoint, replace [ge…

de90664

…o_distance_range] queries with geo distance sorting. (#1191)

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Oct 24, 2016

ruflin#1184 Remove Filtered query:

dcaa8ce

- Remove Elastica/Query/Filtered.php - Remove test/Elastica/Query/FilteredTest.php - Remove all references for filtered query from code - Changelog update

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Oct 24, 2016

ruflin#1184 Remove Filtered query:

d6cb8a9

- Remove Elastica/Query/Filtered.php - Remove test/Elastica/Query/FilteredTest.php - Remove all references for filtered query from code - Changelog update

ruflin pushed a commit that referenced this issue Oct 24, 2016

#1184 Remove Filtered query: (#1189)

a55f66d

- Remove Elastica/Query/Filtered.php - Remove test/Elastica/Query/FilteredTest.php - Remove all references for filtered query from code - Changelog update

p365labs added a commit to p365labs/Elastica that referenced this issue Oct 27, 2016

Remove all depercated methods as this is a major release (ruflin#1184)

42ba7d4

p365labs added a commit to p365labs/Elastica that referenced this issue Oct 28, 2016

update test env from ES 5.0.0-rc1 ES 5.0.0 (ruflin#1184)

b9f51bd

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 9, 2016

removed logging.yml from docker ES config. ES uses now log4j2.propert…

97dbe2f

…ies (ruflin#1184)

ruflin pushed a commit that referenced this issue Nov 9, 2016

removed logging.yml from docker ES config. ES uses now log4j2.propert…

16764ce

…ies (#1184) (#1208)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 9, 2016

fielddata_fields has been deprecated in ES5, use parameter docvalue_f…

3c124b5

…ields instead (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 9, 2016

Fix Set fielddata=true on [answer] (ruflin#1184)

b040b0d

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 9, 2016

fielddata_fields has been deprecated in ES5, use parameter docvalue_f…

776016f

…ields instead (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 9, 2016

Fix Set fielddata=true on [answer] (ruflin#1184)

930c94f

ruflin pushed a commit that referenced this issue Nov 9, 2016

Remove fielddata fields and fix Fix Set fielddata=true (#1209)

f1d9c0e

* fielddata_fields has been deprecated in ES5, use parameter docvalue_fields instead (#1184) * Fix Set fielddata=true on [answer] (#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 9, 2016

updated CompletionTest, payload and output are removed from completio…

e935358

…n suggester (ruflin#1184)

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Nov 9, 2016

ruflin#1184 - Removed "payload option" from suggestions - removed fro…

38a5a60

…m ElasticSearch 5.0 - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 10, 2016

updated CompletionTest, payload and output are removed from completio…

5eba8b7

…n suggester (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 10, 2016

updated CompletionTest, payload and output are removed from completio…

9f68781

…n suggester (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 10, 2016

updated CompletionTest, payload and output are removed from completio…

edb4031

…n suggester (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 10, 2016

updated CompletionTest, payload and output are removed from completio…

c451798

…n suggester (ruflin#1184)

ruflin pushed a commit that referenced this issue Nov 10, 2016

updated CompletionTest, payload and output are removed from completio…

a204fe6

…n suggester (#1184) (#1213)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 11, 2016

store_fields it will no longer extract values from the _source, use m…

1626058

…apping 'store' (ruflin#1184)

ruflin pushed a commit that referenced this issue Nov 12, 2016

store_fields it will no longer extract values from the _source, use m…

c72a75b

…apping 'store' (#1184) (#1215)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 14, 2016

update tests : use text or keyword instead of string and use boolean …

ec47606

…for store instaed yes/no (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 16, 2016

update tests : use text or keyword instead of string and use boolean …

85359de

…for store instaed yes/no (ruflin#1184)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 16, 2016

remove shutdown Node|Cluster as _shutdown has been removed (ruflin#1184)

53597e1

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 16, 2016

remove shutdown Node|Cluster as _shutdown has been removed (ruflin#1184)

083d7f1

ruflin pushed a commit that referenced this issue Nov 17, 2016

update tests : use text or keyword instead of string and use boolean …

c44ec85

…for store instaed yes/no (#1184) (#1218)

p365labs added a commit to p365labs/Elastica that referenced this issue Nov 17, 2016

remove shutdown Node|Cluster as _shutdown has been removed (ruflin#1184)

30c581f

ruflin pushed a commit that referenced this issue Nov 18, 2016

remove shutdown Node|Cluster as _shutdown has been removed (#1184) (#…

683f7d9

…1220)

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Nov 18, 2016

ruflin#1184 - Removed "payload option" from suggestions - removed fro…

5c90bfe

…m ElasticSearch 5.0 - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Nov 18, 2016

ruflin#1184 - Added Regex suggestions

4cba0fe

Zyqsempai pushed a commit to Zyqsempai/Elastica that referenced this issue Nov 19, 2016

ruflin#1184 - Added Regex options

83744f1

Tobion mentioned this issue Jan 31, 2017

Finalize elasticsearch 5 migration #1242

Merged

7 tasks

Tobion closed this as completed Feb 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to Elasticsearch 5.0 #1184

Upgrade to Elasticsearch 5.0 #1184

ruflin commented Oct 11, 2016 •

edited

Loading

Zyqsempai commented Oct 18, 2016

ruflin commented Oct 18, 2016

Zyqsempai commented Oct 19, 2016

Zyqsempai commented Oct 19, 2016

Zyqsempai commented Oct 19, 2016

ruflin commented Oct 24, 2016

p365labs commented Oct 24, 2016

p365labs commented Oct 25, 2016

ruflin commented Oct 25, 2016

Upgrade to Elasticsearch 5.0 #1184

Upgrade to Elasticsearch 5.0 #1184

Comments

ruflin commented Oct 11, 2016 • edited Loading

Zyqsempai commented Oct 18, 2016

ruflin commented Oct 18, 2016

Zyqsempai commented Oct 19, 2016

Zyqsempai commented Oct 19, 2016

Zyqsempai commented Oct 19, 2016

ruflin commented Oct 24, 2016

p365labs commented Oct 24, 2016

p365labs commented Oct 25, 2016

ruflin commented Oct 25, 2016

ruflin commented Oct 11, 2016 •

edited

Loading