Skip to content

Error Registering Hadoop Based Table Into Hive Metastore based table. #4794

@vamen

Description

@vamen

This current registerTable function is not working for converting hadoop catalog based tables to glue based tables. because tables written using hadoop catalog writes metadata.json file in the form v<V+1>.metadata.json

And tables written using glue and hive catalog have format <V+1>-.metadata.json

Hence when we call registerTable with HadoopCatalog based table parseVersion function would fail while parsing.

after renaming the latest metadata.json file I was able to register and query the data.

we can handle this in registerTable function as follows.

1.) We shall call parseVersion() in registerTable. If parsing gets failed in BaseMetastoreTableOperations then the file can be in File-System-Tables spec and we can reparse the file using File-System-Tables spec.

2.) We can then rename the metadata file in required spec and use the renamed file path as metadataFileLocation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions