Skip to content

Add documentation for COMET_PARQUET_PARALLEL_IO_ENABLED #876

@andygrove

Description

@andygrove

What is the problem the feature request solves?

We have some undocumented configs COMET_PARQUET_PARALLEL_IO_ENABLED and COMET_PARQUET_PARALLEL_IO_THREADS in ReadOptions.

They seem to be specific to the Comet native Parquet reader.

It would be nice to have some documenation. Can these be added to CometConf?

public class ReadOptions {
  private static final Logger LOG = LoggerFactory.getLogger(ReadOptions.class);
  public static final String COMET_PARQUET_PARALLEL_IO_ENABLED =
      "comet.parquet.read.parallel.io.enabled";
  public static final boolean COMET_PARQUET_PARALLEL_IO_ENABLED_DEFAULT = true;

  public static final String COMET_PARQUET_PARALLEL_IO_THREADS =
      "comet.parquet.read.parallel.io.thread-pool.size";
  public static final int COMET_PARQUET_PARALLEL_IO_THREADS_DEFAULT = 32;

Describe the potential solution

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions