csv.writer.writerow() does not accept generator (must be coerced to list)

[jdufresne]

BPO	23171
Nosy	@merwok, @bitdancer, @vadmium, @serhiy-storchaka, @jdufresne
Files	csv-gen.patch: patch csv_writerow_iterable.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-03-30.07:11:02.953>
created_at = <Date 2015-01-05.17:31:42.641>
labels = ['easy', 'type-feature', 'library']
title = 'csv.writer.writerow() does not accept generator (must be coerced to list)'
updated_at = <Date 2015-06-02.19:45:10.088>
user = 'https://github.com/jdufresne'

bugs.python.org fields:

activity = <Date 2015-06-02.19:45:10.088>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2015-03-30.07:11:02.953>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2015-01-05.17:31:42.641>
creator = 'jdufresne'
dependencies = []
files = ['37609', '37612']
hgrepos = []
issue_num = 23171
keywords = ['patch', 'easy']
message_count = 9.0
messages = ['233470', '233475', '233501', '233508', '233509', '239192', '239570', '244689', '244690']
nosy_count = 6.0
nosy_names = ['eric.araujo', 'r.david.murray', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'jdufresne']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23171'
versions = ['Python 3.5']

[jdufresne]

The csv.writer.writerow() does not accept a generator as input. I find this counter-intuitive and against the spirit of similar APIs. If the generator is coerced to a list, everything works as expected. See the following test script which fails on the line "w.writerow(g)". In my opinion, this line should work identically to the line "w.writerow(list(g))".

---

import csv

f = open('foo.csv', 'w')
w = csv.writer(f)
g = (i for i in ['a', 'b', 'c'])
w.writerow(list(g))
g = (i for i in ['a', 'b', 'c'])
w.writerow(g)

---

[bitdancer]

This seems like a sensible enhancement request to me.

It is possible it could even be considered a bug, the docs aren't exactly clear on what 'row' is expected to be.

[jdufresne]

I have created an initial patch such that writerow() now allows generators. I have also added a unit test to demonstrate the fix.

The code now coerces iterators (and generators) to a list, then operates on the result. I would have preferred to simply iterate over the argument, however, there is a special case where the length of the argument is exactly 1. So coercing to a list makes checking the length simpler.

All feedback welcome.

[bitdancer]

Hmm. That could be an issue. If someone passes a generator they will generally expect it to be consumed as a generator, not turned into a list implicitly. So it may be better to turn this into a doc bug and require the explicit "list" call :(.

[serhiy-storchaka]

The docs mention that "row" should be a sequence, so there is no a bug.

Here is a patch which makes writerow() accept an iterable without converting it to a list. It also adds tests for few corner cases and fixes the docs.

[vadmium]

Left a question about handling of the unquoted empty field exception on Rietveld.

[python-dev]

New changeset cf5b62036445 by Serhiy Storchaka in branch 'default':
Issue bpo-23171: csv.Writer.writerow() now supports arbitrary iterables.
https://hg.python.org/cpython/rev/cf5b62036445

[bitdancer]

Looks like Serhiy forgot to close this, so closing it.