Skip to content

[enterprise-4.8] BZ-1915563 Update metering MySQL Hive metastore configuration #29855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vikram-redhat merged 1 commit into from
Feb 25, 2021
Merged

[enterprise-4.8] BZ-1915563 Update metering MySQL Hive metastore configuration #29855

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 33 additions & 18 deletions modules/metering-use-mysql-or-postgresql-for-hive.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,23 @@

The default installation of metering configures Hive to use an embedded Java database called Derby. This is unsuited for larger environments and can be replaced with either a MySQL or PostgreSQL database. Use the following example configuration files if your deployment requires a MySQL or PostgreSQL database for Hive.

There are three configuration options you can use to control the database used by Hive metastore: `url`, `driver`, and `secretName`.
There are three configuration options you can use to control the database that is used by Hive metastore: `url`, `driver`, and `secretName`.

Create your MySQL or Postgres instance with a username and password. Then create a secret by using the OpenShift CLI or a YAML file. The secretName you create for this secret must map to the spec.hive.spec.config.db.secretName field in the MeteringConfig resource.
Create your MySQL or Postgres instance with a user name and password. Then create a secret by using the OpenShift CLI (`oc`) or a YAML file. The `secretName` you create for this secret must map to the `spec.hive.spec.config.db.secretName` field in the `MeteringConfig` object resource.

To create a secret in OpenShift CLI you can use the following command:
.Procedure

. Create a secret using the OpenShift CLI (`oc`) or by using a YAML file:
+
* Create a secret by using the following command:
+
[source,terminal]
----
$ oc --namespace openshift-metering create secret generic <YOUR_SECRETNAME> --from-literal=username=<YOUR_DATABASE_USERNAME> --from-literal=password=<YOUR_DATABASE_PASSWORD>
----

To create a secret by using a YAML file, use the following example file:
+
* Create a secret by using a YAML file. For example:
+
[source,yaml]
----
apiVersion: v1
Expand All @@ -30,11 +35,14 @@ data:
password: <BASE64_ENCODED_DATABASE_PASSWORD> <3>
----
<1> The name of the secret.
<2> Base64 encoded database username.
<2> Base64 encoded database user name.
<3> Base64 encoded database password.

Use the example configuration file below to use a MySQL database for Hive:

. Create a configuration file to use a MySQL or PostgreSQL database for Hive:
+
* To use a MySQL database for Hive, use the example configuration file below. Metering supports configuring the internal Hive metastore to use the MySQL server versions 5.6, 5.7, and 8.0.
+
--
[source,yaml]
----
spec:
Expand All @@ -45,16 +53,22 @@ spec:
create: false
config:
db:
url: "jdbc:mysql://mysql.example.com:3306/hive_metastore"
driver: "com.mysql.jdbc.Driver"
secretName: "REPLACEME" <1>
url: "jdbc:mysql://mysql.example.com:3306/hive_metastore" <1>
driver: "com.mysql.cj.jdbc.Driver"
secretName: "REPLACEME" <2>
----
<1> The name of the secret containing the base64-encrypted username and password database credentials.

You can pass additional JDBC parameters using the `spec.hive.config.url`. For more details see the https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html[MySQL Connector/J documentation].

Use the example configuration file below to use a PostgreSQL database for Hive:

[NOTE]
====
When configuring Metering to work with older MySQL server versions, such as 5.6 or 5.7, you might need to add the link:https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-usagenotes-known-issues-limitations.html[`enabledTLSProtocols` JDBC URL parameter] when configuring the internal Hive metastore.
====
<1> To use the TLS v1.2 cipher suite, set `url` to `"jdbc:mysql://<hostname>:<port>/<schema>?enabledTLSProtocols=TLSv1.2"`.
<2> The name of the secret containing the base64-encrypted user name and password database credentials.
--
+
You can pass additional JDBC parameters using the `spec.hive.config.url`. For more details, see the link:https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-configuration-properties.html[MySQL Connector/J 8.0 documentation].
+
* To use a PostgreSQL database for Hive, use the example configuration file below:
+
[source,yaml]
----
spec:
Expand All @@ -70,4 +84,5 @@ spec:
username: "REPLACEME"
password: "REPLACEME"
----
You can pass additional JDBC parameters using the URL. For more details see the https://jdbc.postgresql.org/documentation/head/connect.html#connection-parameters[PostgreSQL JDBC driver documentation].
+
You can pass additional JDBC parameters using the `spec.hive.config.url`. For more details, see the link:https://jdbc.postgresql.org/documentation/head/connect.html#connection-parameters[PostgreSQL JDBC driver documentation].