Set default metadata prefetch size to 512k

[alamb]

Which issue does this PR close?

• Closes #.

Rationale for this change

Inspired by looking at some profiling results I think the metadata prefetch hint will reduce the number of required IOs by 1/2 for most parquet queries at minimal additional overhead. Let's give this a try.

I will write this idea up in more detail if it turns out to make a difference

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[alamb]

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.8.0-1016-gcp #18-Ubuntu SMP Fri Oct 4 22:16:29 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/prefetch (099499f) to 5ab5a03 diff
Benchmarks: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

[alamb]

🤖: Benchmark completed

Details

Comparing HEAD and alamb_prefetch
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_prefetch ┃    Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0     │  1944.05ms │      1940.66ms │ no change │
│ QQuery 1     │   748.05ms │       750.69ms │ no change │
│ QQuery 2     │  1471.52ms │      1445.58ms │ no change │
│ QQuery 3     │   703.03ms │       691.35ms │ no change │
│ QQuery 4     │  1536.24ms │      1485.23ms │ no change │
│ QQuery 5     │ 16937.70ms │     16804.20ms │ no change │
│ QQuery 6     │  2016.33ms │      2030.05ms │ no change │
└──────────────┴────────────┴────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary             ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)             │ 25356.92ms │
│ Total Time (alamb_prefetch)   │ 25147.75ms │
│ Average Time (HEAD)           │  3622.42ms │
│ Average Time (alamb_prefetch) │  3592.54ms │
│ Queries Faster                │          0 │
│ Queries Slower                │          0 │
│ Queries with No Change        │          7 │
└───────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_prefetch ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     1.88ms │         1.89ms │     no change │
│ QQuery 1     │    36.69ms │        40.84ms │  1.11x slower │
│ QQuery 2     │    92.32ms │        89.68ms │     no change │
│ QQuery 3     │    98.54ms │        97.22ms │     no change │
│ QQuery 4     │   782.75ms │       760.18ms │     no change │
│ QQuery 5     │   833.26ms │       881.13ms │  1.06x slower │
│ QQuery 6     │     1.91ms │         1.90ms │     no change │
│ QQuery 7     │    43.39ms │        43.62ms │     no change │
│ QQuery 8     │   925.29ms │       926.41ms │     no change │
│ QQuery 9     │  1205.14ms │      1245.17ms │     no change │
│ QQuery 10    │   262.10ms │       265.58ms │     no change │
│ QQuery 11    │   306.40ms │       302.72ms │     no change │
│ QQuery 12    │   923.85ms │       936.69ms │     no change │
│ QQuery 13    │  1375.17ms │      1275.04ms │ +1.08x faster │
│ QQuery 14    │   862.83ms │       872.23ms │     no change │
│ QQuery 15    │  1062.95ms │      1051.33ms │     no change │
│ QQuery 16    │  1746.84ms │      1786.07ms │     no change │
│ QQuery 17    │  1614.93ms │      1623.43ms │     no change │
│ QQuery 18    │  3142.17ms │      3279.39ms │     no change │
│ QQuery 19    │    98.79ms │        95.73ms │     no change │
│ QQuery 20    │  1171.47ms │      1147.32ms │     no change │
│ QQuery 21    │  1337.25ms │      1325.19ms │     no change │
│ QQuery 22    │  2328.64ms │      2380.56ms │     no change │
│ QQuery 23    │  8359.96ms │      8405.68ms │     no change │
│ QQuery 24    │   460.85ms │       471.21ms │     no change │
│ QQuery 25    │   381.20ms │       400.42ms │  1.05x slower │
│ QQuery 26    │   534.33ms │       536.65ms │     no change │
│ QQuery 27    │  1695.23ms │      1667.68ms │     no change │
│ QQuery 28    │ 12759.54ms │     12754.58ms │     no change │
│ QQuery 29    │   526.58ms │       536.29ms │     no change │
│ QQuery 30    │   824.73ms │       829.15ms │     no change │
│ QQuery 31    │   862.08ms │       884.26ms │     no change │
│ QQuery 32    │  2741.33ms │      2756.61ms │     no change │
│ QQuery 33    │  3341.94ms │      3425.23ms │     no change │
│ QQuery 34    │  3528.85ms │      3444.66ms │     no change │
│ QQuery 35    │  1340.19ms │      1314.81ms │     no change │
│ QQuery 36    │   133.41ms │       133.65ms │     no change │
│ QQuery 37    │    58.96ms │        59.00ms │     no change │
│ QQuery 38    │   127.73ms │       128.35ms │     no change │
│ QQuery 39    │   200.21ms │       203.42ms │     no change │
│ QQuery 40    │    49.07ms │        51.30ms │     no change │
│ QQuery 41    │    46.07ms │        45.58ms │     no change │
│ QQuery 42    │    40.51ms │        40.22ms │     no change │
└──────────────┴────────────┴────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary             ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)             │ 58267.33ms │
│ Total Time (alamb_prefetch)   │ 58518.06ms │
│ Average Time (HEAD)           │  1355.05ms │
│ Average Time (alamb_prefetch) │  1360.89ms │
│ Queries Faster                │          1 │
│ Queries Slower                │          3 │
│ Queries with No Change        │         39 │
└───────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃     HEAD ┃ alamb_prefetch ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │ 127.84ms │       124.31ms │    no change │
│ QQuery 2     │  24.74ms │        24.25ms │    no change │
│ QQuery 3     │  35.56ms │        36.24ms │    no change │
│ QQuery 4     │  21.25ms │        20.97ms │    no change │
│ QQuery 5     │  57.34ms │        58.09ms │    no change │
│ QQuery 6     │   8.24ms │         8.05ms │    no change │
│ QQuery 7     │ 102.61ms │       104.99ms │    no change │
│ QQuery 8     │  26.98ms │        26.29ms │    no change │
│ QQuery 9     │  63.37ms │        62.92ms │    no change │
│ QQuery 10    │  60.83ms │        58.13ms │    no change │
│ QQuery 11    │  12.89ms │        13.04ms │    no change │
│ QQuery 12    │  37.63ms │        39.22ms │    no change │
│ QQuery 13    │  28.82ms │        30.62ms │ 1.06x slower │
│ QQuery 14    │   9.92ms │         9.96ms │    no change │
│ QQuery 15    │  24.36ms │        25.93ms │ 1.06x slower │
│ QQuery 16    │  23.69ms │        23.80ms │    no change │
│ QQuery 17    │  96.84ms │        95.51ms │    no change │
│ QQuery 18    │ 247.24ms │       246.83ms │    no change │
│ QQuery 19    │  28.74ms │        29.20ms │    no change │
│ QQuery 20    │  39.39ms │        38.88ms │    no change │
│ QQuery 21    │ 172.44ms │       175.64ms │    no change │
│ QQuery 22    │  16.64ms │        20.19ms │ 1.21x slower │
└──────────────┴──────────┴────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary             ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)             │ 1267.37ms │
│ Total Time (alamb_prefetch)   │ 1273.06ms │
│ Average Time (HEAD)           │   57.61ms │
│ Average Time (alamb_prefetch) │   57.87ms │
│ Queries Faster                │         0 │
│ Queries Slower                │         3 │
│ Queries with No Change        │        19 │
└───────────────────────────────┴───────────┘

[alamb]

TLDR is that this didn't seem to have much/any effect on performance so closing