Which part is this question about
How to use the API of arrow_ipc and arrow_array for delta dictionaries when nested builders are involved.
Describe your question
I have implemented delta dictionaries based on the example at https://docs.rs/arrow-ipc/latest/arrow_ipc/writer/struct.StreamWriter.html#example---efficient-delta-dictionaries and this works for top-level columns in my schema by calling column.finish_preserve_values() instead of column.finish() when collecting the arrays for the next RecordBatch.
However, I also have nested builders of lists of structs which contain dictionary fields. The type of the top-level build is then ListBuilder<Box<dyn ArrayBuilder>> which does not have a finish_preserve_values method.
How can I produce the top-level array while instructing the StringDictionaryBuilder hidden inside the StructBuilder inside the ListBuilder to preserve its values?
Which part is this question about
How to use the API of
arrow_ipcandarrow_arrayfor delta dictionaries when nested builders are involved.Describe your question
I have implemented delta dictionaries based on the example at https://docs.rs/arrow-ipc/latest/arrow_ipc/writer/struct.StreamWriter.html#example---efficient-delta-dictionaries and this works for top-level columns in my schema by calling
column.finish_preserve_values()instead ofcolumn.finish()when collecting the arrays for the nextRecordBatch.However, I also have nested builders of lists of structs which contain dictionary fields. The type of the top-level build is then
ListBuilder<Box<dyn ArrayBuilder>>which does not have afinish_preserve_valuesmethod.How can I produce the top-level array while instructing the
StringDictionaryBuilderhidden inside theStructBuilderinside theListBuilderto preserve its values?