shutil.copytree failing on xattr-less filesystems (like NFS)

[Yhg1s]

BPO	17076
Nosy	@Yhg1s, @hynek
Files	shutil.patch: shutil._coypxattr patch with test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/hynek'
closed_at = <Date 2013-02-05.08:16:53.761>
created_at = <Date 2013-01-29.18:18:12.534>
labels = ['type-bug']
title = 'shutil.copytree failing on xattr-less filesystems (like NFS)'
updated_at = <Date 2013-02-05.08:16:53.760>
user = 'https://github.com/Yhg1s'

bugs.python.org fields:

activity = <Date 2013-02-05.08:16:53.760>
actor = 'hynek'
assignee = 'hynek'
closed = True
closed_date = <Date 2013-02-05.08:16:53.761>
closer = 'hynek'
components = []
creation = <Date 2013-01-29.18:18:12.534>
creator = 'twouters'
dependencies = []
files = ['28927']
hgrepos = []
issue_num = 17076
keywords = ['patch', 'needs review']
message_count = 6.0
messages = ['180920', '180947', '181046', '181047', '181423', '181424']
nosy_count = 3.0
nosy_names = ['twouters', 'python-dev', 'hynek']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17076'
versions = ['Python 3.3', 'Python 3.4']

[Yhg1s]

The new xattr support in shutil causes shutil.copytree and shutil.copy2 to fail inelegantly on (source) filesystems that do not support xattrs (like NFS):

# /home/twouters does not support xattrs
>>> os.listxattr("/home/twouters/foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 95] Operation not supported: '/home/twouters/foo'

>>> shutil.copytree("/home/twouters/spam", "/tmp/spam")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/twouters/gvs-pristine/python/Python-3.3.0/Lib/shutil.py", line 345, in copytree
    raise Error(errors)
shutil.Error: [('/home/twouters/spam/ham', '/tmp/spam/ham', "[Errno 95] Operation not supported: '/home/twouters/spam/ham'"), ('/home/twouters/spam/eggs', '/tmp/spam/eggs', "[Errno 95] Operation not supported: '/home/twouters/spam/eggs'"), ('/home/twouters/spam', '/tmp/spam', "[Errno 95] Operation not supported: '/home/twouters/spam'")]

(The actual files will have been copied, since xattr copies are done after everything else.) Interestingly shutil._copyxattr does try to cope with unsupported xattrs on the *target* filesystem (which seems like it might be a mistake to do, since it loses data), just not the original filesystem (which seems like a sensible thing instead):

# /tmp does support xattrs
>>> os.listxattr("/tmp/spam")
[]
>>> shutil.copytree("/tmp/spam", "/home/twouters/spam-new")
'/home/twouters/spam-new'

The attached patch fixes shutil._copyxattr to also ignore unsupported/empty xattrs (but not permission errors) on the source. (I'm not certain if errno.ENODATA can be expected from os.listxattr(), but internet searches suggest that some people think so, and I don't know what other meaning it could have.)

[hynek]

Could you add regression tests to your patch please?

[Yhg1s]

Updated patch with test.

[Yhg1s]

Now really updated the patch.

[python-dev]

New changeset 47c65639390d by Hynek Schlawack in branch '3.3':
bpo-17076: Make copying of xattrs more permissive of missing FS support
http://hg.python.org/cpython/rev/47c65639390d

New changeset 7ccdbd1cd213 by Hynek Schlawack in branch 'default':
bpo-17076: Make copying of xattrs more permissive of missing FS support
http://hg.python.org/cpython/rev/7ccdbd1cd213

[hynek]

The buildbots look happy, thank you for spotting & the patch Thomas!