Skip to content

Commit 873d380

Browse files
albertzaharovitsalbertzaharovits
committed
Repository GCS plugin new client library (#30168)
This does away with the deprecated `com.google.api-client:google-api-client:1.23` and replaces it with `com.google.cloud:google-cloud-storage:1.28.0`. It also changes security permissions for the repository-gcs plugin. Closes: #29259
1 parent 017779b commit 873d380

File tree

78 files changed

+3403
-981
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+3403
-981
lines changed

‎docs/plugins/repository-gcs.asciidoc‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -84,11 +84,7 @@ A service account file looks like this:
8484
"private_key_id": "...",
8585
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
8686
"client_email": "service-account-for-your-repository@your-project-id.iam.gserviceaccount.com",
87-
"client_id": "...",
88-
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
89-
"token_uri": "https://accounts.google.com/o/oauth2/token",
90-
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
91-
"client_x509_cert_url": "..."
87+
"client_id": "..."
9288
}
9389
----
9490
// NOTCONSOLE
@@ -178,6 +174,12 @@ are marked as `Secure`.
178174
a custom name can be useful to authenticate your cluster when requests
179175
statistics are logged in the Google Cloud Platform. Default to `repository-gcs`
180176

177+
`project_id`::
178+
179+
The Google Cloud project id. This will be automatically infered from the credentials file but
180+
can be specified explicitly. For example, it can be used to switch between projects when the
181+
same credentials are usable for both the production and the development projects.
182+
181183
[[repository-gcs-repository]]
182184
==== Repository Settings
183185

‎plugins/repository-gcs/build.gradle‎

Lines changed: 191 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -22,38 +22,207 @@ esplugin {
2222
classname 'org.elasticsearch.repositories.gcs.GoogleCloudStoragePlugin'
2323
}
2424

25-
versions << [
26-
'google': '1.23.0',
27-
]
28-
2925
dependencies {
30-
compile "com.google.apis:google-api-services-storage:v1-rev115-${versions.google}"
31-
compile "com.google.api-client:google-api-client:${versions.google}"
32-
compile "com.google.oauth-client:google-oauth-client:${versions.google}"
33-
compile "org.apache.httpcomponents:httpclient:${versions.httpclient}"
34-
compile "org.apache.httpcomponents:httpcore:${versions.httpcore}"
35-
compile "commons-logging:commons-logging:${versions.commonslogging}"
36-
compile "commons-codec:commons-codec:${versions.commonscodec}"
37-
compile "com.google.http-client:google-http-client:${versions.google}"
38-
compile "com.google.http-client:google-http-client-jackson2:${versions.google}"
26+
compile 'com.google.cloud:google-cloud-storage:1.28.0'
27+
compile 'com.google.cloud:google-cloud-core:1.28.0'
28+
compile 'com.google.cloud:google-cloud-core-http:1.28.0'
29+
compile 'com.google.auth:google-auth-library-oauth2-http:0.9.1'
30+
compile 'com.google.auth:google-auth-library-credentials:0.9.1'
31+
compile 'com.google.oauth-client:google-oauth-client:1.23.0'
32+
compile 'com.google.http-client:google-http-client:1.23.0'
33+
compile 'com.google.http-client:google-http-client-jackson:1.23.0'
34+
compile 'com.google.http-client:google-http-client-jackson2:1.23.0'
35+
compile 'com.google.http-client:google-http-client-appengine:1.23.0'
36+
compile 'com.google.api-client:google-api-client:1.23.0'
37+
compile 'com.google.api:gax:1.25.0'
38+
compile 'com.google.api:gax-httpjson:0.40.0'
39+
compile 'com.google.api:api-common:1.5.0'
40+
compile 'com.google.api.grpc:proto-google-common-protos:1.8.0'
41+
compile 'com.google.guava:guava:20.0'
42+
compile 'com.google.apis:google-api-services-storage:v1-rev115-1.23.0'
43+
compile 'org.codehaus.jackson:jackson-core-asl:1.9.13'
44+
compile 'io.grpc:grpc-context:1.9.0'
45+
compile 'io.opencensus:opencensus-api:0.11.1'
46+
compile 'io.opencensus:opencensus-contrib-http-util:0.11.1'
47+
compile 'org.threeten:threetenbp:1.3.6'
3948
}
4049

4150
dependencyLicenses {
42-
mapping from: /google-.*/, to: 'google'
51+
mapping from: /google-cloud-.*/, to: 'google-cloud'
52+
mapping from: /google-auth-.*/, to: 'google-auth'
53+
mapping from: /google-http-.*/, to: 'google-http'
54+
mapping from: /opencensus.*/, to: 'opencensus'
4355
}
4456

4557
thirdPartyAudit.excludes = [
58+
// uses internal java api: sun.misc.Unsafe
59+
'com.google.common.cache.Striped64',
60+
'com.google.common.cache.Striped64$1',
61+
'com.google.common.cache.Striped64$Cell',
62+
'com.google.common.hash.LittleEndianByteArray$UnsafeByteArray$1',
63+
'com.google.common.hash.LittleEndianByteArray$UnsafeByteArray$2',
64+
'com.google.common.hash.LittleEndianByteArray$UnsafeByteArray$3',
65+
'com.google.common.util.concurrent.AbstractFuture$UnsafeAtomicHelper',
66+
'com.google.common.util.concurrent.AbstractFuture$UnsafeAtomicHelper$1',
67+
'com.google.common.hash.LittleEndianByteArray$UnsafeByteArray',
68+
'com.google.common.primitives.UnsignedBytes$LexicographicalComparatorHolder$UnsafeComparator',
69+
'com.google.common.primitives.UnsignedBytes$LexicographicalComparatorHolder$UnsafeComparator$1',
4670
// classes are missing
47-
'com.google.common.base.Splitter',
48-
'com.google.common.collect.Lists',
49-
'javax.servlet.ServletContextEvent',
50-
'javax.servlet.ServletContextListener',
51-
'org.apache.avalon.framework.logger.Logger',
52-
'org.apache.log.Hierarchy',
53-
'org.apache.log.Logger',
71+
'com.google.appengine.api.datastore.Blob',
72+
'com.google.appengine.api.datastore.DatastoreService',
73+
'com.google.appengine.api.datastore.DatastoreServiceFactory',
74+
'com.google.appengine.api.datastore.Entity',
75+
'com.google.appengine.api.datastore.Key',
76+
'com.google.appengine.api.datastore.KeyFactory',
77+
'com.google.appengine.api.datastore.PreparedQuery',
78+
'com.google.appengine.api.datastore.Query',
79+
'com.google.appengine.api.memcache.Expiration',
80+
'com.google.appengine.api.memcache.MemcacheService',
81+
'com.google.appengine.api.memcache.MemcacheServiceFactory',
82+
'com.google.appengine.api.urlfetch.FetchOptions$Builder',
83+
'com.google.appengine.api.urlfetch.FetchOptions',
84+
'com.google.appengine.api.urlfetch.HTTPHeader',
85+
'com.google.appengine.api.urlfetch.HTTPMethod',
86+
'com.google.appengine.api.urlfetch.HTTPRequest',
87+
'com.google.appengine.api.urlfetch.HTTPResponse',
88+
'com.google.appengine.api.urlfetch.URLFetchService',
89+
'com.google.appengine.api.urlfetch.URLFetchServiceFactory',
90+
'com.google.gson.Gson',
91+
'com.google.gson.GsonBuilder',
92+
'com.google.gson.TypeAdapter',
93+
'com.google.gson.stream.JsonReader',
94+
'com.google.gson.stream.JsonWriter',
95+
'com.google.iam.v1.Binding$Builder',
96+
'com.google.iam.v1.Binding',
97+
'com.google.iam.v1.Policy$Builder',
98+
'com.google.iam.v1.Policy',
99+
'com.google.protobuf.AbstractMessageLite$Builder',
100+
'com.google.protobuf.AbstractParser',
101+
'com.google.protobuf.Any$Builder',
102+
'com.google.protobuf.Any',
103+
'com.google.protobuf.AnyOrBuilder',
104+
'com.google.protobuf.AnyProto',
105+
'com.google.protobuf.Api$Builder',
106+
'com.google.protobuf.Api',
107+
'com.google.protobuf.ApiOrBuilder',
108+
'com.google.protobuf.ApiProto',
109+
'com.google.protobuf.ByteString',
110+
'com.google.protobuf.CodedInputStream',
111+
'com.google.protobuf.CodedOutputStream',
112+
'com.google.protobuf.DescriptorProtos',
113+
'com.google.protobuf.Descriptors$Descriptor',
114+
'com.google.protobuf.Descriptors$EnumDescriptor',
115+
'com.google.protobuf.Descriptors$EnumValueDescriptor',
116+
'com.google.protobuf.Descriptors$FieldDescriptor',
117+
'com.google.protobuf.Descriptors$FileDescriptor$InternalDescriptorAssigner',
118+
'com.google.protobuf.Descriptors$FileDescriptor',
119+
'com.google.protobuf.Descriptors$OneofDescriptor',
120+
'com.google.protobuf.Duration$Builder',
121+
'com.google.protobuf.Duration',
122+
'com.google.protobuf.DurationOrBuilder',
123+
'com.google.protobuf.DurationProto',
124+
'com.google.protobuf.EmptyProto',
125+
'com.google.protobuf.Enum$Builder',
126+
'com.google.protobuf.Enum',
127+
'com.google.protobuf.EnumOrBuilder',
128+
'com.google.protobuf.ExtensionRegistry',
129+
'com.google.protobuf.ExtensionRegistryLite',
130+
'com.google.protobuf.FloatValue$Builder',
131+
'com.google.protobuf.FloatValue',
132+
'com.google.protobuf.FloatValueOrBuilder',
133+
'com.google.protobuf.GeneratedMessage$GeneratedExtension',
134+
'com.google.protobuf.GeneratedMessage',
135+
'com.google.protobuf.GeneratedMessageV3$Builder',
136+
'com.google.protobuf.GeneratedMessageV3$BuilderParent',
137+
'com.google.protobuf.GeneratedMessageV3$FieldAccessorTable',
138+
'com.google.protobuf.GeneratedMessageV3',
139+
'com.google.protobuf.Internal$EnumLite',
140+
'com.google.protobuf.Internal$EnumLiteMap',
141+
'com.google.protobuf.Internal',
142+
'com.google.protobuf.InvalidProtocolBufferException',
143+
'com.google.protobuf.LazyStringArrayList',
144+
'com.google.protobuf.LazyStringList',
145+
'com.google.protobuf.MapEntry$Builder',
146+
'com.google.protobuf.MapEntry',
147+
'com.google.protobuf.MapField',
148+
'com.google.protobuf.Message',
149+
'com.google.protobuf.MessageOrBuilder',
150+
'com.google.protobuf.Parser',
151+
'com.google.protobuf.ProtocolMessageEnum',
152+
'com.google.protobuf.ProtocolStringList',
153+
'com.google.protobuf.RepeatedFieldBuilderV3',
154+
'com.google.protobuf.SingleFieldBuilderV3',
155+
'com.google.protobuf.Struct$Builder',
156+
'com.google.protobuf.Struct',
157+
'com.google.protobuf.StructOrBuilder',
158+
'com.google.protobuf.StructProto',
159+
'com.google.protobuf.Timestamp$Builder',
160+
'com.google.protobuf.Timestamp',
161+
'com.google.protobuf.TimestampProto',
162+
'com.google.protobuf.Type$Builder',
163+
'com.google.protobuf.Type',
164+
'com.google.protobuf.TypeOrBuilder',
165+
'com.google.protobuf.TypeProto',
166+
'com.google.protobuf.UInt32Value$Builder',
167+
'com.google.protobuf.UInt32Value',
168+
'com.google.protobuf.UInt32ValueOrBuilder',
169+
'com.google.protobuf.UnknownFieldSet$Builder',
170+
'com.google.protobuf.UnknownFieldSet',
171+
'com.google.protobuf.WireFormat$FieldType',
172+
'com.google.protobuf.WrappersProto',
173+
'com.google.protobuf.util.Timestamps',
174+
'org.apache.http.ConnectionReuseStrategy',
175+
'org.apache.http.Header',
176+
'org.apache.http.HttpEntity',
177+
'org.apache.http.HttpEntityEnclosingRequest',
178+
'org.apache.http.HttpHost',
179+
'org.apache.http.HttpRequest',
180+
'org.apache.http.HttpResponse',
181+
'org.apache.http.HttpVersion',
182+
'org.apache.http.RequestLine',
183+
'org.apache.http.StatusLine',
184+
'org.apache.http.client.AuthenticationHandler',
185+
'org.apache.http.client.HttpClient',
186+
'org.apache.http.client.HttpRequestRetryHandler',
187+
'org.apache.http.client.RedirectHandler',
188+
'org.apache.http.client.RequestDirector',
189+
'org.apache.http.client.UserTokenHandler',
190+
'org.apache.http.client.methods.HttpDelete',
191+
'org.apache.http.client.methods.HttpEntityEnclosingRequestBase',
192+
'org.apache.http.client.methods.HttpGet',
193+
'org.apache.http.client.methods.HttpHead',
194+
'org.apache.http.client.methods.HttpOptions',
195+
'org.apache.http.client.methods.HttpPost',
196+
'org.apache.http.client.methods.HttpPut',
197+
'org.apache.http.client.methods.HttpRequestBase',
198+
'org.apache.http.client.methods.HttpTrace',
199+
'org.apache.http.conn.ClientConnectionManager',
200+
'org.apache.http.conn.ConnectionKeepAliveStrategy',
201+
'org.apache.http.conn.params.ConnManagerParams',
202+
'org.apache.http.conn.params.ConnPerRouteBean',
203+
'org.apache.http.conn.params.ConnRouteParams',
204+
'org.apache.http.conn.routing.HttpRoutePlanner',
205+
'org.apache.http.conn.scheme.PlainSocketFactory',
206+
'org.apache.http.conn.scheme.Scheme',
207+
'org.apache.http.conn.scheme.SchemeRegistry',
208+
'org.apache.http.conn.ssl.SSLSocketFactory',
209+
'org.apache.http.conn.ssl.X509HostnameVerifier',
210+
'org.apache.http.entity.AbstractHttpEntity',
211+
'org.apache.http.impl.client.DefaultHttpClient',
212+
'org.apache.http.impl.client.DefaultHttpRequestRetryHandler',
213+
'org.apache.http.impl.conn.ProxySelectorRoutePlanner',
214+
'org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager',
215+
'org.apache.http.message.BasicHttpResponse',
216+
'org.apache.http.params.BasicHttpParams',
217+
'org.apache.http.params.HttpConnectionParams',
218+
'org.apache.http.params.HttpParams',
219+
'org.apache.http.params.HttpProtocolParams',
220+
'org.apache.http.protocol.HttpContext',
221+
'org.apache.http.protocol.HttpProcessor',
222+
'org.apache.http.protocol.HttpRequestExecutor'
54223
]
55224

56225
check {
57226
// also execute the QA tests when testing the plugin
58227
dependsOn 'qa:google-cloud-storage:check'
59-
}
228+
}

‎plugins/repository-gcs/licenses/api-common-1.5.0.jar.sha1‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
7e537338d40a57ad469239acb6d828fa544fb52b

‎plugins/repository-gcs/licenses/api-common-LICENSE.txt‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
Copyright 2016, Google Inc.
2+
All rights reserved.
3+
Redistribution and use in source and binary forms, with or without
4+
modification, are permitted provided that the following conditions are
5+
met:
6+
7+
* Redistributions of source code must retain the above copyright
8+
notice, this list of conditions and the following disclaimer.
9+
* Redistributions in binary form must reproduce the above
10+
copyright notice, this list of conditions and the following disclaimer
11+
in the documentation and/or other materials provided with the
12+
distribution.
13+
* Neither the name of Google Inc. nor the names of its
14+
contributors may be used to endorse or promote products derived from
15+
this software without specific prior written permission.
16+
17+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
18+
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
19+
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
20+
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
21+
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
22+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
23+
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
24+
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
25+
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
26+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎plugins/repository-gcs/licenses/api-common-NOTICE.txt‎

Whitespace-only changes.

‎plugins/repository-gcs/licenses/commons-codec-1.10.jar.sha1‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎plugins/repository-gcs/licenses/commons-logging-1.1.3.jar.sha1‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎plugins/repository-gcs/licenses/gax-1.25.0.jar.sha1‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
36ab73c0b5d4a67447eb89a3174cc76ced150bd1

‎plugins/repository-gcs/licenses/gax-LICENSE.txt‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
Copyright 2016, Google Inc. All rights reserved.
2+
3+
Redistribution and use in source and binary forms, with or without
4+
modification, are permitted provided that the following conditions are
5+
met:
6+
7+
* Redistributions of source code must retain the above copyright
8+
notice, this list of conditions and the following disclaimer.
9+
* Redistributions in binary form must reproduce the above
10+
copyright notice, this list of conditions and the following disclaimer
11+
in the documentation and/or other materials provided with the
12+
distribution.
13+
* Neither the name of Google Inc. nor the names of its
14+
contributors may be used to endorse or promote products derived from
15+
this software without specific prior written permission.
16+
17+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
18+
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
19+
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
20+
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
21+
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
22+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
23+
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
24+
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
25+
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
26+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎plugins/repository-gcs/licenses/gax-NOTICE.txt‎

Whitespace-only changes.

0 commit comments

Comments
 (0)