Remove fielddata fields and fix Fix Set fielddata=true (ruflin#1209)

* fielddata_fields has been deprecated in ES5, use parameter docvalue_fields instead (ruflin#1184) * Fix Set fielddata=true on [answer] (ruflin#1184) Change to official docker image (ruflin#1210) * Switch test environment to official docker image. * Disable tests with multiple elasticsearch instances. Update PHPUnit to the most recent version (ruflin#1211) Prepare release 5.0.0-beta1 (ruflin#1212) updated CompletionTest, payload and output are removed from completion suggester (ruflin#1184) (ruflin#1213) store_fields it will no longer extract values from the _source, use mapping 'store' (ruflin#1184) (ruflin#1215) update tests : use text or keyword instead of string and use boolean for store instaed yes/no (ruflin#1184) (ruflin#1218) remove shutdown Node|Cluster as _shutdown has been removed (ruflin#1184) (ruflin#1220) Squashed commits - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0 - Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0
Zyqsempai · Nov 21, 2016 · e3f9342 · e3f9342
1 parent 16764ce
commit e3f9342
Show file tree

Hide file tree

Showing 37 changed files with 247 additions and 329 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,11 +1,28 @@
 # Change Log
 All notable changes to this project will be documented in this file based on the [Keep a Changelog](http://keepachangelog.com/) Standard. This project adheres to [Semantic Versioning](http://semver.org/).
 
-## [Unreleased](https://github.com/ruflin/Elastica/compare/3.2.3...HEAD)
+## [Unreleased](https://github.com/ruflin/Elastica/compare/5.0.0-beta1...master)
 
 ### Backward Compatibility Fixes
+- Updated Elastica\Test\Suggest\CompletionTest now payload and output are removed
+- Updated Elastica\Test\TypeTest::testGetDocumentWithFieldsSelection The stored_fields parameter will only return stored fields — it will no longer extract values from the _source
+- remove _shutdown for Node and Cluster as deprecated
+- Removed "payload option" from suggestions - removed from ElasticSearch 5.0
+- Changed "text" field to "prefix" in suggestions - changed in ElasticSearch 5.0
+
+### Bugfixes
+
+### Added
+
+### Improvements
+
+## Deprecated
+
+
+## [5.0.0-beta1](https://github.com/ruflin/Elastica/compare/3.2.3...5.0.0-beta1)
+
+### Backward Compatibility Breaks
 - Update elasticsearch dependency to 5.0
-- Composer installations will no longer include tests and other development files.
 - Replace flush refresh param with a options array
 - Rename Mapping::setFields to Mapping::setStoredFields
 - Removing all deprecated filters including tests. Use queries instead.
@@ -14,63 +31,39 @@ All notable changes to this project will be documented in this file based on the
 - Remove Elastica/Query/Filtered.php and test/Elastica/Query/FilteredTest.php and all uses from code.
 - Remove index.merge.policy.merge_factor, and set/get MergePolicy as it looks deprecated from ES 1.6
 - Add new "Percolate query" functionality and tests
-- removed in Elastica\AbstractUpdateAction Option "percolate", getter and setter as deprecated as of ES 1.3. Use Percolator instead.
-- removed in Elastica\Aggregation\DateHistogram Option "pre_zone" is deprecated as of ES 1.5. Use "time_zone" instead
-- removed in Elastica\Aggregation\DateHistogram Option "post_zone" is deprecated as of ES 1.5. Use "time_zone" instead.
-- removed in Elastica\Aggregation\DateHistogram Option "pre_zone_adjust_large_interval" is deprecated as of ES 1.5. Use "time_zone" instead.
-- removed in Elastica\Aggregation\DateHistogram Option "pre_offset" is deprecated as of ES 1.5. Use "offset" instead.
-- removed in Elastica\Aggregation\DateHistogram Option "post_offset" is deprecated as of ES 1.5. Use "offset" instead.
-- removed Elastica\Document::set as deprecated. Use Elastica\Document::set instead
-- removed Elastica\Document::setScript() is no longer available as of 0.90.2. See http://elastica.io/migration/0.90.2/upsert.html to migrate.
-- removed Elastica\Document::getScript() is no longer available as of 0.90.2. See http://elastica.io/migration/0.90.2/upsert.html to migrate.
-- removed Elastica\Document::hasScript() is no longer available as of 0.90.2. See http://elastica.io/migration/0.90.2/upsert.html to migrate.
-- removed Elastica/Query::setLimit as deprecated. Use the Elastica/Query::setSize() method
-- removed Elastica\Query\Builder
-- removed Elastica\Query\Fuzzy::addField as deprecated. Use Elastica\Query\Fuzzy::setField and Elastica\Query\FuzzysetFieldOption instead.
-- removed Elastica\Query::setIds as deprecated. Use Elastica\Query::like instead.
-- removed Elastica\Query::setLikeText as deprecated. Use Elastica\Query::like instead.
-- removed Elastica\Query Option "percent_terms_to_match" is deprecated as of ES 1.5. Use "minimum_should_match" instead.
-- removed Elastica\QueryBuilder\DSL\Query "More Like This Field" query is deprecated as of ES 1.4. Use MoreLikeThis query instead.
-- changed visibility from protected to private Elastica\ResultSet::$_position as accessing this property in an extended class is deprecated.
-- changed visibility from protected to private Elastica\ResultSet::$_response as accessing this property in an extended class is deprecated.
-- changed visibility from protected to private Elastica\ResultSet::$_query as accessing this property in an extended class is deprecated.
-- changed visibility from protected to private Elastica\ResultSet::$_results as accessing this property in an extended class is deprecated.
-- removed Elastica\ResultSet::$_timedOut as deprecated. Use ResultSet->hasTimedOut() instead.
-- removed Elastica\ResultSet::$_took as deprecated. Use ResultSet->hasTimedOut() instead.
-- removed Elastica\ResultSet::$_totalHits as deprecated. Use ResultSet->hasTimedOut() instead.
-- removed Elastica\Type::delete() It is no longer possible to delete the mapping for a type. Instead you should delete the index and recreate it with the new mappings.
-- removed Elastica\Query\Builder as deprecated. Use new Elastica\QueryBuilder instead.
-- removed Elastica\Percolator as deprecated. Use new Elastica\Query\Percolate instead.
-- changed Elastica\Index::deleteByQuery() to use new API https://www.elastic.co/guide/en/elasticsearch/reference/5.0/docs-delete-by-query.html
-- updated Dockerfile and elasticsearch.yml to allow inline.script: true
-- updated some Script function to use groovy as now default scripting is painless
-    - Elastica\Test\Aggregation\ScriptTest::testAggregationScript
-    - Elastica\Test\Aggregation\ScriptTest::testAggregationScriptAsString
-    - Elastica\Test\Query\FunctionScoreTest::testScriptScore
-    - Elastica\Test\BulkTest::testUpdate
-    - Elastica\Test\ClientTest::testUpdateDocumentByScript
-    - Elastica\Test\ClientTest::testUpdateDocumentByScriptWithUpsert
-    - Elastica\Test\ClientTest::testUpdateDocumentPopulateFields
-    - Elastica\Test\ClientTest::testUpdateDocumentPopulateFields
-    - Elastica\Test\TypeTest::testUpdateDocument
-    - Elastica\Test\TypeTest::testUpdateDocumentWithIdForwardSlashes
-    - Elastica\Test\TypeTest::testUpdateDocumentWithParameter
-    - Elastica\Test\TypeTest::testUpdateDocumentWithFieldsSource
-- linted some files:
-    - Elastica\Index.php
-    - Elastica\Query\Percolate.php
-    - Elastica\QueryBuilder\DSL\Query.php
-    - Elastica\Test\IndexTest.php
-    - Elastica\Test\Node\InfoTest.php
-    - Elastica\Test\Query\PercolateTest.php
-- removed Elastica\ScanAndScroll and test, Scan search type is removed from ElasticSearch 5.0.
+- Remove in Elastica\AbstractUpdateAction Option "percolate", getter and setter as deprecated as of ES 1.3. Use Percolator instead.
+- Remove in Elastica\Aggregation\DateHistogram Option "pre_zone" is deprecated as of ES 1.5. Use "time_zone" instead
+- Remove in Elastica\Aggregation\DateHistogram Option "post_zone" is deprecated as of ES 1.5. Use "time_zone" instead.
+- Remove in Elastica\Aggregation\DateHistogram Option "pre_zone_adjust_large_interval" is deprecated as of ES 1.5. Use "time_zone" instead.
+- Remove in Elastica\Aggregation\DateHistogram Option "pre_offset" is deprecated as of ES 1.5. Use "offset" instead.
+- Remove in Elastica\Aggregation\DateHistogram Option "post_offset" is deprecated as of ES 1.5. Use "offset" instead.
+- Remove Elastica\Document::set as deprecated. Use Elastica\Document::set instead
+- Remove Elastica\Document::setScript() is no longer available as of 0.90.2. See http://elastica.io/migration/0.90.2/upsert.html to migrate.
+- Remove Elastica\Document::getScript() is no longer available as of 0.90.2. See http://elastica.io/migration/0.90.2/upsert.html to migrate.
+- Remove Elastica\Document::hasScript() is no longer available as of 0.90.2. See http://elastica.io/migration/0.90.2/upsert.html to migrate.
+- Remove Elastica/Query::setLimit as deprecated. Use the Elastica/Query::setSize() method
+- Remove Elastica\Query\Builder
+- Remove Elastica\Query\Fuzzy::addField as deprecated. Use Elastica\Query\Fuzzy::setField and Elastica\Query\FuzzysetFieldOption instead.
+- Remove Elastica\Query::setIds as deprecated. Use Elastica\Query::like instead.
+- Remove Elastica\Query::setLikeText as deprecated. Use Elastica\Query::like instead.
+- Remove Elastica\Query Option "percent_terms_to_match" is deprecated as of ES 1.5. Use "minimum_should_match" instead.
+- Remove Elastica\QueryBuilder\DSL\Query "More Like This Field" query is deprecated as of ES 1.4. Use MoreLikeThis query instead.
+- Changed visibility from protected to private Elastica\ResultSet::$_position as accessing this property in an extended class is deprecated.
+- Changed visibility from protected to private Elastica\ResultSet::$_response as accessing this property in an extended class is deprecated.
+- Changed visibility from protected to private Elastica\ResultSet::$_query as accessing this property in an extended class is deprecated.
+- Changed visibility from protected to private Elastica\ResultSet::$_results as accessing this property in an extended class is deprecated.
+- Removed Elastica\ResultSet::$_timedOut as deprecated. Use ResultSet->hasTimedOut() instead.
+- Removed Elastica\ResultSet::$_took as deprecated. Use ResultSet->hasTimedOut() instead.
+- Removed Elastica\ResultSet::$_totalHits as deprecated. Use ResultSet->hasTimedOut() instead.
+- Removed Elastica\Type::delete() It is no longer possible to delete the mapping for a type. Instead you should delete the index and recreate it with the new mappings.
+- Removed Elastica\Query\Builder as deprecated. Use new Elastica\QueryBuilder instead.
+- Removed Elastica\Percolator as deprecated. Use new Elastica\Query\Percolate instead.
+- Changed Elastica\Index::deleteByQuery() to use new API https://www.elastic.co/guide/en/elasticsearch/reference/5.0/docs-delete-by-query.html
+- Remove Elastica\ScanAndScroll and test, Scan search type is removed from ElasticSearch 5.0.
 - Remove support for PHP 5.4 and 5.5. Require at least PHP 5.6 #1202
-- removed groovy as default scripting language
-- implemented painless as default scripting language in tests
-- removed search_type=count is removed in Elasticsearch 5.0
-- removed logging.yml as now ES 5.0 uses log4j2.properties (leaving defaults)
-
-### Bugfixes
+- Remove groovy as default scripting language
+- Remove search_type=count as it is removed in Elasticsearch 5.0
+- Remove fielddata_fields as it has been deprecated in ES5, use parameter docvalue_fields instead
 
 ### Added
 - Elastica\QueryBuilder\DSL\Query::exists
@@ -85,9 +78,23 @@ $cluster->getHealth()->getIndices()[$indexName]
 $indices = $cluster->getHealth()->getIndices();
 $indices[$indexName]
 ```
-- Added a `Query::setTrackScores` method 
-
-## Deprecated
+- Added a `Query::setTrackScores` method
+- Implemented painless as default scripting language in tests
+- Updated Dockerfile and elasticsearch.yml to allow inline.script: true
+- Updated some Script function to use groovy as now default scripting is painless
+    - Elastica\Test\Aggregation\ScriptTest::testAggregationScript
+    - Elastica\Test\Aggregation\ScriptTest::testAggregationScriptAsString
+    - Elastica\Test\Query\FunctionScoreTest::testScriptScore
+    - Elastica\Test\BulkTest::testUpdate
+    - Elastica\Test\ClientTest::testUpdateDocumentByScript
+    - Elastica\Test\ClientTest::testUpdateDocumentByScriptWithUpsert
+    - Elastica\Test\ClientTest::testUpdateDocumentPopulateFields
+    - Elastica\Test\ClientTest::testUpdateDocumentPopulateFields
+    - Elastica\Test\TypeTest::testUpdateDocument
+    - Elastica\Test\TypeTest::testUpdateDocumentWithIdForwardSlashes
+    - Elastica\Test\TypeTest::testUpdateDocumentWithParameter
+    - Elastica\Test\TypeTest::testUpdateDocumentWithFieldsSource
+- Composer installations will no longer include tests and other development files.
 
 
 ## [3.2.3](https://github.com/ruflin/Elastica/compare/3.2.2...3.2.3)

diff --git a/Makefile b/Makefile
@@ -95,9 +95,9 @@ build:
 
 .PHONY: setup
 setup: build
-	docker-compose scale elasticsearch=3
+	#docker-compose scale elasticsearch=3
 	# TODO: Makes the snapshot directory writable for all instances. Nicer solution needed.
-	docker-compose run elasticsearch chmod -R 777 /tmp/backups/
+	#docker-compose run elasticsearch chmod -R 777 /tmp/backups/
 
 .PHONY: start
 start:

diff --git a/composer.json b/composer.json
@@ -37,7 +37,7 @@
     },
     "extra": {
         "branch-alias": {
-            "dev-master": "3.2.x-dev"
+            "dev-master": "5.0.x-dev"
         }
     }
 }
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -14,7 +14,12 @@ elasticsearch:
   #image: ruflin/elasticsearch-elastica
   volumes_from:
     - data
-  command: elasticsearch -Ehttp.host=0.0.0.0
+  environment:
+    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+    - "network.host="
+    - "transport.host=127.0.0.1"
+    - "http.host=0.0.0.0"
+    - "xpack.security.enabled=false"
 
 nginx:
   build: ./env/nginx/

diff --git a/env/elastica/composer.json b/env/elastica/composer.json
@@ -18,7 +18,7 @@
         "pdepend/pdepend":"~2.2",
         "phploc/phploc":"~2.1",
         "phpmd/phpmd":"~2.3",
-        "phpunit/phpunit":"~4.8",
+        "phpunit/phpunit":"~5.6",
         "sebastian/phpcpd":"~2.0",
         "squizlabs/php_codesniffer":"~2.5"
     }

diff --git a/env/elasticsearch/Dockerfile b/env/elasticsearch/Dockerfile
@@ -1,54 +1,12 @@
-FROM java:8-jre
+FROM docker.elastic.co/elasticsearch/elasticsearch:latest
 MAINTAINER Nicolas Ruflin <[email protected]>
 
-ENV VERSION 5.0.0
-
-ENV URL https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-${VERSION}.tar.gz
-
-# Cache variable can be set during building to invalidate the build cache with `--build-arg CACHE=$(date +%s) .`
-ARG CACHE=1
-
-ENV ESHOME /opt/elasticsearch-${VERSION}
-ENV ES_PLUGIN_BIN ${ESHOME}/bin/elasticsearch-plugin
-
-# grab gosu for easy step-down from root
-RUN gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
-RUN arch="$(dpkg --print-architecture)" \
-	&& set -x \
-	&& curl -o /usr/local/bin/gosu -fSL "https://github.com/tianon/gosu/releases/download/1.3/gosu-$arch" \
-	&& curl -o /usr/local/bin/gosu.asc -fSL "https://github.com/tianon/gosu/releases/download/1.3/gosu-$arch.asc" \
-	&& gpg --verify /usr/local/bin/gosu.asc \
-	&& rm /usr/local/bin/gosu.asc \
-	&& chmod +x /usr/local/bin/gosu
-
-RUN groupadd -r elasticsearch && useradd -r -m -g elasticsearch elasticsearch
-
-RUN set -x && \
-	cd /opt && \
-	wget -qO elasticsearch.tar.gz "$URL?${CACHE}" && \
-	tar xzvf elasticsearch.tar.gz && \
-	chown -R elasticsearch:elasticsearch ${ESHOME}
-
-ENV PATH ${ESHOME}/bin:$PATH
-
-VOLUME ${ESHOME}/data
-
-ENV ES_JAVA_OPTS="-Xms512m -Xmx512m"
-
-RUN ${ES_PLUGIN_BIN} install mapper-attachments
+RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install mapper-attachments
 
 # Copy config files
-COPY elasticsearch.yml /opt/elasticsearch-${VERSION}/config/elasticsearch.yml
-COPY scripts/* /opt/elasticsearch-${VERSION}/config/scripts/
-
-COPY docker-entrypoint.sh /
-
-ENTRYPOINT ["/docker-entrypoint.sh"]
+COPY elasticsearch.yml /usr/share/elasticsearch/config/elasticsearch.yml
+COPY scripts/* /usr/share/elasticsearch/config/scripts/
 
 
 RUN mkdir -p /tmp/backups/backup1
 RUN mkdir -p /tmp/backups/backup2
-
-EXPOSE 9200 9300
-
-CMD ["elasticsearch"]
diff --git a/lib/Elastica/Aggregation/TopHits.php b/lib/Elastica/Aggregation/TopHits.php
@@ -155,6 +155,6 @@ public function setHighlight(array $highlightArgs)
      */
     public function setFieldDataFields(array $fields)
     {
-        return $this->setParam('fielddata_fields', $fields);
+        return $this->setParam('docvalue_fields', $fields);
     }
 }
diff --git a/lib/Elastica/Cluster.php b/lib/Elastica/Cluster.php
@@ -166,20 +166,4 @@ public function getSettings()
     {
         return new Settings($this->getClient());
     }
-
-    /**
-     * Shuts down the complete cluster.
-     *
-     * @link https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-shutdown.html
-     *
-     * @param string $delay OPTIONAL Seconds to shutdown cluster after (default = 1s)
-     *
-     * @return \Elastica\Response
-     */
-    public function shutdown($delay = '1s')
-    {
-        $path = '_shutdown?delay='.$delay;
-
-        return $this->_client->request($path, Request::POST);
-    }
 }
diff --git a/lib/Elastica/Node.php b/lib/Elastica/Node.php
@@ -142,20 +142,4 @@ public function refresh()
         $this->_stats = null;
         $this->_info = null;
     }
-
-    /**
-     * Shuts this node down.
-     *
-     * @param string $delay OPTIONAL Delay after which node is shut down (default = 1s)
-     *
-     * @return \Elastica\Response
-     *
-     * @link https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-shutdown.html
-     */
-    public function shutdown($delay = '1s')
-    {
-        $path = '_cluster/nodes/'.$this->getId().'/_shutdown?delay='.$delay;
-
-        return $this->_client->request($path, Request::POST);
-    }
 }
diff --git a/lib/Elastica/Query.php b/lib/Elastica/Query.php
@@ -267,7 +267,7 @@ public function setStoredFields(array $fields)
      */
     public function setFieldDataFields(array $fieldDataFields)
     {
-        return $this->setParam('fielddata_fields', $fieldDataFields);
+        return $this->setParam('docvalue_fields', $fieldDataFields);
     }
 
     /**

diff --git a/lib/Elastica/Query/InnerHits.php b/lib/Elastica/Query/InnerHits.php
@@ -166,6 +166,6 @@ public function setHighlight(array $highlightArgs)
      */
     public function setFieldDataFields(array $fields)
     {
-        return $this->setParam('fielddata_fields', $fields);
+        return $this->setParam('docvalue_fields', $fields);
     }
 }
-Original file line number
+Diff line change
@@ Expand Up / @@ -267,7 +267,7 @@ public function setStoredFields(array $fields) @@
          */
         public function setFieldDataFields(array $fieldDataFields)
         {
-            return $this->setParam('fielddata_fields', $fieldDataFields);
+            return $this->setParam('docvalue_fields', $fieldDataFields);
         }
         /**
@@ Expand Down @@