Add documentation for db resource group manager by phd3 · Pull Request #4523 · trinodb/trino

phd3 · 2020-07-21T17:38:19Z

Lewuathe · 2020-07-22T06:46:33Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+* ``resource-groups.config-file``: a JSON file storing the configurations, which can be an absolute
+  path, or a path relative to the Presto data directory.
+
+Db Resource Group Manager


Suggested change

Db Resource Group Manager

Database Resource Group Manager

Might Database instead of Db be more descriptive?

Database resource group manager

Lewuathe · 2020-07-22T07:05:48Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+* ``resource-groups.max-refresh-interval``: Time period for which the cluster will continue to accept
+  queries after refresh failures cause configuration to become stale. The default is ``1h``.
+* ``resource-groups.exact-match-selector-enabled``: Boolean flag to enable usage of a selector requiring
+  an exact match of environment, source and query type.


As the default value for DbResourceGroupConfig.exactMatchSelectorEnabled implicitly set false, should we put the default value here?

yes ... also this needs to be explained more.. maybe with an example.

mosabua · 2020-07-23T19:25:29Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

-Add an ``etc/resource-groups.properties`` file with the following contents to enable
-the built-in manager that reads a JSON config file:
+
+There are two built-in managers: file-based and database-based. Add an ``etc/resource-groups.properties``


You can use a file-based or a database-based resource group manager:

Add a file etc/resource-groups.properties

Set the resource-groups.configuration-manager property to file or db

Add further configuration properties for the the desired manager.

mosabua · 2020-07-23T19:25:43Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+There are two built-in managers: file-based and database-based. Add an ``etc/resource-groups.properties``
+file with the appropriate configurations to enable one of the two managers.
+
+File Resource Group Manager


File resource group manager

mosabua · 2020-07-23T19:25:57Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+File Resource Group Manager
+---------------------------
+
+This manager reads the resource group configuration from a JSON config file.


The file resource group manager reads a JSON configuration file, specified with resource-groups.config-file

mosabua · 2020-07-23T19:32:21Z

presto-docs/src/main/sphinx/admin/resource-groups.rst


-Change the value of ``resource-groups.config-file`` to point to a JSON config file,
-which can be an absolute path, or a path relative to the Presto data directory.
+* ``resource-groups.config-file``: a JSON file storing the configurations, which can be an absolute


The path to the JSON file can be an absolute path, or a path relative to the Presto data directory.

mosabua · 2020-07-23T19:32:38Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

-which can be an absolute path, or a path relative to the Presto data directory.
+* ``resource-groups.config-file``: a JSON file storing the configurations, which can be an absolute
+  path, or a path relative to the Presto data directory.
+


Also does this file need to be on the coordinator and all workers?

mosabua · 2020-07-23T19:34:32Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

 .. literalinclude:: resource-groups-example.json
    :language: json
+
+Providing Selector Properties


Providing selector properties

mosabua · 2020-07-23T19:35:13Z

presto-docs/src/main/sphinx/admin/resource-groups.rst


+.. _global-properties:
+
 Global Properties


Global propertie

mosabua · 2020-07-23T19:35:19Z

presto-docs/src/main/sphinx/admin/resource-groups.rst


+.. _selector-rules:
+
 Selector Rules


Selector rules

mosabua · 2020-07-23T19:36:03Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+Db Resource Group Manager
+-------------------------
+
+This manager periodically loads the resource group configuration from a database.


The database resource group manager loads the configuration from a relational database.

Need to doc which ones are supported.

Only MySQL is supported at the moment. It does mean it should work with MariaDB too but I'm not sure.

mosabua · 2020-07-23T19:36:50Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+.. code-block:: none
+
+    resource-groups.configuration-manager=db
+    resource-groups.config-db-url=jdbc:mysql://localhost:3306/config_db?user=presto_admin


Which databases are supported? How do users get JDBC driver in place?

mosabua · 2020-07-23T19:38:48Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+    resource-groups.max-refresh-interval=1m
+    resource-groups.exact-match-selector-enabled=true
+
+* ``resource-groups.config-db-url`` (required): Database URL to load configuration from.


specify more details about this JDBC URL

mosabua · 2020-07-23T19:40:06Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+    resource-groups.exact-match-selector-enabled=true
+
+* ``resource-groups.config-db-url`` (required): Database URL to load configuration from.
+* ``resource-groups.max-refresh-interval``: Time period for which the cluster will continue to accept


Thats a odd name for what it does. Anyway .. remove future usage . otherwise ok

mosabua · 2020-07-23T19:41:52Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+    resource-groups.max-refresh-interval=1m
+    resource-groups.exact-match-selector-enabled=true
+
+* ``resource-groups.config-db-url`` (required): Database URL to load configuration from.


All properties in the following section require quite a bit more docs so I would suggest to remove usage of a bullet lest and just document them in multiple paragraphs one after the other.

mosabua · 2020-07-23T19:42:35Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+is used for loading exact-match based selectors. If any of the tables do not exist, Presto creates them
+on server startup.
+
+---------------------+-------------------------------------------------------+---------------------------------+


This is way too much info for a table. I suggest we break this up into a separate section with proper code blocks and paragraphs that details more info.

mosabua · 2020-07-23T19:50:03Z

presto-docs/src/main/sphinx/admin/resource-groups.rst

+* ``resource-groups.exact-match-selector-enabled``: Boolean flag to enable usage of a selector requiring
+  an exact match of environment, source and query type.
+
+The configuration is loaded every second from the following tables under the specified db url. If the


The paragraph and following table needs to be broken up and reordered

First startup and automatic table creation

Document each table maybe with example row of data (including additional table for exact-match)

Document how we suggest users add data to the table (maybe example insert statement, update statement), also should they do that from Presto via catalog, or some other SQL UI directly on the db (probably that)

How it works (refresh interval...)

mosabua

Various updates and clarifications needed.

posulliv · 2022-01-21T21:30:45Z

@phd3 do you mind if I create a new PR to update these docs by rebasing from your branch? I've made some updates in this area and want to document them.

phd3 · 2022-01-24T20:14:25Z

please go ahead @posulliv , thanks for picking this up.

posulliv · 2022-01-24T21:58:55Z

thanks @phd3!

I opened a new PR with a branch off your commit and it has some of the feedback from here applied.

posulliv · 2022-01-24T21:59:10Z

This is the PR I opened - #10771

Add documentation for db resource group manager

7ab26f7

cla-bot bot added the cla-signed label Jul 21, 2020

phd3 requested a review from raghavsethi July 21, 2020 17:38

martint requested a review from mosabua July 22, 2020 02:48

Lewuathe reviewed Jul 22, 2020

View reviewed changes

mosabua reviewed Jul 23, 2020

View reviewed changes

presto-docs/src/main/sphinx/admin/resource-groups.rst

.. _global-properties:

Global Properties

Copy link
Copy Markdown

Member

mosabua Jul 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global propertie

mosabua reviewed Jul 23, 2020

View reviewed changes

presto-docs/src/main/sphinx/admin/resource-groups.rst

.. _selector-rules:

Selector Rules

Copy link
Copy Markdown

Member

mosabua Jul 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Selector rules

mosabua reviewed Jul 23, 2020

View reviewed changes

mosabua requested changes Jul 23, 2020

View reviewed changes

findepi force-pushed the master branch from 8538e49 to 1f896ea Compare July 30, 2021 22:13

posulliv mentioned this pull request Jan 20, 2022

Add PostgreSQL and Oracle migration files for DB backed resource group manager #9812

Merged

posulliv closed this Jan 24, 2022

Conversation

phd3 commented Jul 21, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mosabua Jul 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mosabua Jul 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mosabua left a comment

Choose a reason for hiding this comment

Uh oh!

posulliv commented Jan 21, 2022

Uh oh!

phd3 commented Jan 24, 2022

Uh oh!

posulliv commented Jan 24, 2022

Uh oh!

posulliv commented Jan 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants

mosabua Jul 23, 2020 •

edited

Loading

mosabua Jul 23, 2020 •

edited

Loading