Skip to content

Commit

Permalink
Docs: clarify when the reader will read from object store when using …
Browse files Browse the repository at this point in the history
…cached metadata (#10909)
  • Loading branch information
alamb committed Jun 14, 2024
1 parent cc60278 commit 8f76ac5
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions datafusion/core/src/datasource/physical_plan/parquet/reader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
// under the License.

//! [`ParquetFileReaderFactory`] and [`DefaultParquetFileReaderFactory`] for
//! creating parquet file readers
//! low level control of parquet file readers

use crate::datasource::physical_plan::{FileMeta, ParquetFileMetrics};
use bytes::Bytes;
Expand All @@ -33,12 +33,19 @@ use std::sync::Arc;
///
/// The combined implementations of [`ParquetFileReaderFactory`] and
/// [`AsyncFileReader`] can be used to provide custom data access operations
/// such as pre-cached data, I/O coalescing, etc.
/// such as pre-cached metadata, I/O coalescing, etc.
///
/// See [`DefaultParquetFileReaderFactory`] for a simple implementation.
pub trait ParquetFileReaderFactory: Debug + Send + Sync + 'static {
/// Provides an `AsyncFileReader` for reading data from a parquet file specified
///
/// # Notes
///
/// If the resulting [`AsyncFileReader`] returns `ParquetMetaData` without
/// page index information, the reader will load it on demand. Thus it is important
/// to ensure that the returned `ParquetMetaData` has the necessary information
/// if you wish to avoid a subsequent I/O
///
/// # Arguments
/// * partition_index - Index of the partition (for reporting metrics)
/// * file_meta - The file to be read
Expand Down

0 comments on commit 8f76ac5

Please sign in to comment.