Is your feature request related to a problem or challenge? Please describe what you are trying to do.
RecordBatchStream represents an async stream of record batches, but currently it's only possible to iterate synchronously through each batch.
Describe the solution you'd like
Implement __anext__ as well as __next__ here, so it can be used as an asynchronous iterator from Python:
|
fn next(&mut self, py: Python) -> PyResult<Option<PyRecordBatch>> { |
|
let result = self.stream.next(); |
|
match wait_for_future(py, result) { |
|
None => Ok(None), |
|
Some(Ok(b)) => Ok(Some(b.into())), |
|
Some(Err(e)) => Err(e.into()), |
|
} |
|
} |
Describe alternatives you've considered
Additional context
This is similar to what I already have here wrapping ObjectStore::get
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
RecordBatchStreamrepresents an async stream of record batches, but currently it's only possible to iterate synchronously through each batch.Describe the solution you'd like
Implement
__anext__as well as__next__here, so it can be used as an asynchronous iterator from Python:datafusion-python/src/record_batch.rs
Lines 57 to 64 in 5e97701
Describe alternatives you've considered
Additional context
This is similar to what I already have here wrapping
ObjectStore::get