Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: catalog datatype mappings #13410

Merged
merged 5 commits into from
Oct 24, 2023
Merged

docs: catalog datatype mappings #13410

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
748c8d6
Update 50-file-format-options.md
soyeric128 Oct 24, 2023
b18ee12
Update 50-file-format-options.md
soyeric128 Oct 24, 2023
68173c8
Datatype Mapping to Databend
soyeric128 Oct 24, 2023
83fd969
fix comments
soyeric128 Oct 24, 2023
d83a6e0
Merge branch 'main' into catalog_datatype
soyeric128 Oct 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/doc/13-sql-reference/50-file-format-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Input & Output File Formats
---

Databend accepts a variety of file formats both as a source and as a target for data loading or unloading. For example, you can load data into Databend from a file with the [COPY INTO table command](../14-sql-commands/10-dml/dml-copy-into-table.md) or the Streaming Load API. You can also unload data from Databend into a file with the [COPY INTO location command](../14-sql-commands/10-dml/dml-copy-into-location.md) command.
Databend accepts a variety of file formats both as a source and as a target for data loading or unloading. This page explains the supported file formats and their available options.

## Syntax

Expand All @@ -17,7 +17,7 @@ To specify a file format in a statement, use the following syntax:
```

- Databend currently supports XML as a source ONLY. Unloading data into an XML file is not supported yet.
- If you don't specify the FILE_FORMAT when performing a COPY INTO or SELECT operation from a stage, Databend will use the file format that you initially defined for the stage when you created it. In cases where you didn't explicitly specify a file format during the stage creation, Databend defaults to using the PARQUET format.
- If you don't specify the FILE_FORMAT when performing a COPY INTO or SELECT operation from a stage, Databend will use the file format that you initially defined for the stage when you created it. In cases where you didn't explicitly specify a file format during the stage creation, Databend defaults to using the PARQUET format. If you specify a different FILE_FORMAT from the one you defined when creating the stage, Databend will prioritize the FILE_FORMAT specified during the operation.
- For managing custom file formats in Databend, see [File Format](../14-sql-commands/00-ddl/100-file-format/index.md).

### formatTypeOptions
Expand Down
20 changes: 20 additions & 0 deletions docs/doc/14-sql-commands/00-ddl/91-catalog/hive.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,26 @@ import FunctionDescription from '@site/src/components/FunctionDescription';

Databend supports the integration of an [Apache Hive](https://hive.apache.org/) catalog, enhancing its compatibility and versatility for data management and analytics. This extends Databend's capabilities by seamlessly incorporating the powerful metadata and storage management capabilities of Apache Hive into the platform.

## Datatype Mapping to Databend

This table maps data types between Apache Hive and Databend. Please note that Databend does not currently support Hive data types that are not listed in the table.

| Apache Hive | Databend |
| ------------------- | -------------------- |
| BOOLEAN | [BOOLEAN](../../../13-sql-reference/10-data-types/00-data-type-logical-types.md) |
| TINYINT | [TINYINT (INT8)](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#integer-data-types) |
| SMALLINT | [SMALLINT (INT16)](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#integer-data-types) |
| INT | [INT (INT32)](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#integer-data-types) |
| BIGINT | [BIGINT (INT64)](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#integer-data-types) |
| DATE | [DATE](../../../13-sql-reference/10-data-types/20-data-type-time-date-types.md) |
| TIMESTAMP | [TIMESTAMP](../../../13-sql-reference/10-data-types/20-data-type-time-date-types.md) |
| FLOAT | [FLOAT (FLOAT32)](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#floating-point-data-types) |
| DOUBLE | [DOUBLE (FLOAT64)](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#floating-point-data-types) |
| VARCHAR | [VARCHAR (STRING)](../../../13-sql-reference/10-data-types/30-data-type-string-types.md) |
| DECIMAL | [DECIMAL](../../../13-sql-reference/10-data-types/11-data-type-decimal-types.md) |
| ARRAY<TYPE> | [ARRAY](../../../13-sql-reference/10-data-types/40-data-type-array-types.md), supports nesting |
| MAP<KEYTYPE, VALUETYPE> | [MAP](../../../13-sql-reference/10-data-types/42-data-type-map.md) |

## Managing Apache Hive Catalogs

Databend provides you the following commands to manage Apache Hive catalogs:
Expand Down
20 changes: 20 additions & 0 deletions docs/doc/14-sql-commands/00-ddl/91-catalog/iceberg.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,26 @@ import FunctionDescription from '@site/src/components/FunctionDescription';

Databend supports the integration of an [Apache Iceberg](https://iceberg.apache.org/) catalog, enhancing its compatibility and versatility for data management and analytics. This extends Databend's capabilities by seamlessly incorporating the powerful metadata and storage management capabilities of Apache Iceberg into the platform.

## Datatype Mapping to Databend

This table maps data types between Apache Iceberg and Databend. Please note that Databend does not currently support Iceberg data types that are not listed in the table.

| Apache Iceberg | Databend |
| ------------------------------- | ----------------------- |
| BOOLEAN | [BOOLEAN](../../../13-sql-reference/10-data-types/00-data-type-logical-types.md) |
| INT | [INT32](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#integer-data-types) |
| LONG | [INT64](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#integer-data-types) |
| DATE | [DATE](../../../13-sql-reference/10-data-types/20-data-type-time-date-types.md) |
| TIMESTAMP/TIMESTAMPZ | [TIMESTAMP](../../../13-sql-reference/10-data-types/20-data-type-time-date-types.md) |
| FLOAT | [FLOAT](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#floating-point-data-types) |
| DOUBLE | [DOUBLE](../../../13-sql-reference/10-data-types/10-data-type-numeric-types.md#floating-point-data-types) |
| STRING/BINARY | [STRING](../../../13-sql-reference/10-data-types/30-data-type-string-types.md) |
| DECIMAL | [DECIMAL](../../../13-sql-reference/10-data-types/11-data-type-decimal-types.md) |
| ARRAY<TYPE> | [ARRAY](../../../13-sql-reference/10-data-types/40-data-type-array-types.md), supports nesting |
| MAP<KEYTYPE, VALUETYPE> | [MAP](../../../13-sql-reference/10-data-types/42-data-type-map.md) |
| STRUCT<COL1: TYPE1, COL2: TYPE2, ...> | [TUPLE](../../../13-sql-reference/10-data-types/41-data-type-tuple-types.md) |
| LIST | [ARRAY](../../../13-sql-reference/10-data-types/40-data-type-array-types.md) |

## Managing Apache Iceberg Catalogs

Databend provides you the following commands to manage Apache Iceberg catalogs:
Expand Down