Improve reading for File-like table engine with column oriented format

[keen-wolf]

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

• Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Make the File-like table engine to read only needed columns which will reduce the IO, Memory, also the compution cost. This close #issue:20129.

Detailed description / Documentation draft:
When call executeFetchColumns to construct the read pipeline for StorageFile engines, previously all the columns in this table(storage metadata) are passed to the subsequent InputFormat which does the real IO. Now only the columns needed for latter reading are passed through: StorageFileSource -> SourceWithProgress ->InputFormat->(parquet, arrow, native...)BlockInputFormat. If the underlying InputFormat is column-oriented, then it can only read these required columns so to avoid unnecessary cost, and for those non-column-oriented InputFormats, this will have no side-effects on them. This update mainly modify codes within the "StorageFile.cpp".

[keen-wolf]

Sorry for update so late. It's a long new year holiday~

Three situations for File-like related methods have been carefully analyzed here: the File-Engine for Creating Table, the TableFunction.File and the TableFunction.Input. The first two have been tested to be improved by this feature, It has no effects on the last one, for "Input" is handled in client-side, also the reading is done by outside PIPE CMD.

I've used parquet files to verify the performance improvement. also checked with the previous parquet and arrow format testcases to testify the correctness. Here is the comparison result before and after this optimization, for both File-Storage and TableFunction.File methods.

FileFunction-before:

FileFunction-after:

FileStorage-after:

But no thoroughly tests have been done for all other File-like Formats, it's going to be a big job. The automated performance comparison testcase can be made and integrated to the CI, if needed.

[keen-wolf]

After test and code analysis, I found the “Native” format will read all columns from the raw data file, as this patch only fix the column filter path from the upper layer of StorageFile, so it will do no favor to the “Native” format. If we want to improve it, some internal code changes may be required.
And below is another test for 3 valid formats(Parquet, Arrow, ORC). As we can see, the most obvious improvement comes from the Parquet format

Btw. the random String was created via:
dd if=/dev/urandom of=/var/lib/clickhouse/user_files/200KB_rand bs=200K count=1

[nikitamikhaylov]

This PR is merged in neighboring PR. All commits are saved.

[keen-wolf]

This PR is merged in neighboring PR. All commits are saved.

great job!