From 1624b097c244f875372921fe115871d451fcd3b0 Mon Sep 17 00:00:00 2001 From: Steve Burnett Date: Thu, 6 Feb 2025 16:42:13 -0500 Subject: [PATCH] Add documentation for file-based metastore --- presto-docs/src/main/sphinx/connector.rst | 1 + .../sphinx/connector/file-based-metastore.rst | 71 +++++++++++++++++++ .../main/sphinx/installation/deployment.rst | 2 + 3 files changed, 74 insertions(+) create mode 100644 presto-docs/src/main/sphinx/connector/file-based-metastore.rst diff --git a/presto-docs/src/main/sphinx/connector.rst b/presto-docs/src/main/sphinx/connector.rst index d337fe4ed12d1..0c49baf1db363 100644 --- a/presto-docs/src/main/sphinx/connector.rst +++ b/presto-docs/src/main/sphinx/connector.rst @@ -17,6 +17,7 @@ from different data sources. connector/deltalake connector/druid connector/elasticsearch + connector/file-based-metastore connector/googlesheets connector/hana connector/hive diff --git a/presto-docs/src/main/sphinx/connector/file-based-metastore.rst b/presto-docs/src/main/sphinx/connector/file-based-metastore.rst new file mode 100644 index 0000000000000..46c7df09a2d6b --- /dev/null +++ b/presto-docs/src/main/sphinx/connector/file-based-metastore.rst @@ -0,0 +1,71 @@ +==================== +File-Based Metastore +==================== + +.. contents:: + :local: + :backlinks: none + :depth: 1 + +Overview +^^^^^^^^ + +For testing or developing purposes, Presto can be configured to use a local +filesystem directory as a Hive Metastore. + +The file-based metastore works only with the following connectors: + +* :doc:`/connector/deltalake` +* :doc:`/connector/hive` +* :doc:`/connector/hudi` +* :doc:`/connector/iceberg` + +Configuring a File-Based Metastore +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. In ``etc/catalog/``, find the catalog properties file for the supported + connector. + +2. In the catalog properties file, set the following properties: + +.. code-block:: none + + hive.metastore=file + hive.metastore.catalog.dir=file:/// + +Replace ```` in the example with the path to a directory on an +accessible filesystem. + +Using a File-Based Warehouse +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For this example, assume the Hive connector is being used, and the properties +in the Hive connector catalog file are set to the following: + +.. code-block:: none + + connector.name=hive + hive.metastore=file + hive.metastore.catalog.dir=file:///data/hive_data/ + +Create a schema + +.. code-block:: none + + CREATE SCHEMA hive.warehouse; + +This query creates a directory ``warehouse`` in the directory set for +``hive.metastore.catalog.dir``, so the path to the new directory is +``/data/hive_data/warehouse``. + +Create a table with any connector-supported file formats. For example, if the +Hive connector is being configured: + +.. code-block:: none + + CREATE TABLE hive.warehouse.orders_csv("order_name" varchar, "quantity" varchar) WITH (format = 'CSV'); + CREATE TABLE hive.warehouse.orders_parquet("order_name" varchar, "quantity" int) WITH (format = 'PARQUET'); + +These queries create folders as ``/data/hive_data/warehouse/orders_csv`` and +``/data/hive_data/warehouse/orders_parquet``. Users can insert and query +from these tables. diff --git a/presto-docs/src/main/sphinx/installation/deployment.rst b/presto-docs/src/main/sphinx/installation/deployment.rst index b7f4af7311004..5f11e4fd03c8d 100644 --- a/presto-docs/src/main/sphinx/installation/deployment.rst +++ b/presto-docs/src/main/sphinx/installation/deployment.rst @@ -7,6 +7,8 @@ Deploying Presto :backlinks: none :depth: 1 +.. _Installing Presto: + Installing Presto -----------------