Tracking: Reading iceberg tables.

[liurenjie1024]

Now we have finished rest catalog to load table, reading/writing table metadata. The next step is to read iceberg tables using these apis.

• Implement basic file scan planning. #66
• Add runtime module to enable concurrent load of manifest files. #124
• Enable reading to arrow record batch stream. #125
• Pruning partitions using filter when reading file. #126
• Pruning data files according to index, statistics when reading. #127
• Plan file scan task according scan file size. #128

[sdd]

If you are aiming just to have table reads working first, and optimizing them afterwards, then #124 is not completely necessary to do at this stage?

[liurenjie1024]

Hi, @sdd

If you are aiming just to have table reads working first, and optimizing them afterwards, then #124 is not completely necessary to do at this stage?

Yes, I think currently it's a relatively low priority task compared with others. But it's not that far away since this doesn't require huge effort, but careful design.

[Xuanwo]

Is this tracking issue still relevant? Should we start a new one to better reflect the current status?

[liurenjie1024]

Is this tracking issue still relevant? Should we start a new one to better reflect the current status?

Most of them are closed, but size based planning still not finished.

[Fokko]

I would be in favor of closing this one, and just keep #128 :)