Repository navigation
os.statvfs() not working well with unicode paths #62895
Description
| BPO | 18695 |
|---|---|
| Nosy | @vstinner, @giampaolo, @benjaminp, @ezio-melotti, @serhiy-storchaka |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/giampaolo'
closed_at = <Date 2020-05-31.14:42:33.481>
created_at = <Date 2013-08-09.12:06:43.536>
labels = ['type-bug', 'expert-unicode']
title = 'os.statvfs() not working well with unicode paths'
updated_at = <Date 2020-05-31.14:42:33.480>
user = 'https://github.com/giampaolo'bugs.python.org fields:
activity = <Date 2020-05-31.14:42:33.480>
actor = 'serhiy.storchaka'
assignee = 'giampaolo.rodola'
closed = True
closed_date = <Date 2020-05-31.14:42:33.481>
closer = 'serhiy.storchaka'
components = ['Unicode']
creation = <Date 2013-08-09.12:06:43.536>
creator = 'giampaolo.rodola'
dependencies = []
files = ['31207', '33633', '33634']
hgrepos = []
issue_num = 18695
keywords = ['patch', 'needs review']
message_count = 16.0
messages = ['194726', '194764', '194774', '194783', '194788', '208114', '208844', '208847', '208849', '208853', '208857', '208959', '219038', '219048', '219054', '370467']
nosy_count = 6.0
nosy_names = ['vstinner', 'giampaolo.rodola', 'benjamin.peterson', 'ezio.melotti', 'Arfrever', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue18695'
versions = ['Python 2.7']
Activity
giampaolo commentedon Aug 9, 2013
From: https://code.google.com/p/psutil/issues/detail?id=416
The script above works fine on Python 3.3 but on 2.7 you'll get:
Patch in attachment fixes the issue.
serhiy-storchaka commentedon Aug 9, 2013
Functions such as rename(), popen(), mkfifo(), mknod(), etc have the same issue.
vstinner commentedon Aug 9, 2013
Cool, you now have a good reason to upgrade to Python 3 ;-)
I'm not sure that it's a good idea to invest time on fixing Unicode issues in Python 2, especially in a minor version (Python 2.7.x).
giampaolo commentedon Aug 10, 2013
I admit I sort of share the same doubts, but considering 2.7 a "minor python version", especially at this point, would be a mistake.
serhiy-storchaka commentedon Aug 10, 2013
2.7.x is a minor version, not 2.7. We fixed Unicode issues in Python 2 bugfixes many times.
serhiy-storchaka commentedon Jan 14, 2014
Giampaolo, do you want to provide a test?
giampaolo commentedon Jan 22, 2014
Attached patch includes tests. I took test_sax.py as an example.
giampaolo commentedon Jan 22, 2014
While I'm at it I'm going to fix also mkfifo(), mknod() and others. Hold on a bit more.
giampaolo commentedon Jan 22, 2014
Ok, patch in attachment fixes mkfifo(), mknod() and statvfs() and also includes Unicode tests for all os module's path-related functions.
serhiy-storchaka commentedon Jan 22, 2014
You have eaten "return NULL;" in posix_mkfifo.
"from test.test_support import TESTFN_UNICODE, TESTFN_ENCODING" can fail. The simplest solution is just initialize them to None by default in test_support.
If TESTFN_UNICODE.encode(TESTFN_ENCODING) fails (on POSIX locale), it will be better to run tests with unicode(TESTFN, 'ascii') than skip them.
Tests should check that results for unicode filename is same as for str filename.
As far as Victor have doubts, we should ask Benjamin.
vstinner commentedon Jan 22, 2014
Well, if you begin to patch some os functions, we will find much functions which don't support Unicode path.
I prefer to consider that Python 2 doesn't support Unicode filenames to avoid bugs.
If you want to support Unicode filename, we will have to modify a lot of code. What's the point since Python 3 has a very good support of Unicode? Much better than Python 2?
giampaolo commentedon Jan 23, 2014
Either way it's fine with me.
Regardless I think these tests have some value because all those os functions are not currently tested so it might makes sense to apply Serhiy's suggestions and port them to Python 3.4.
serhiy-storchaka commentedon May 24, 2014
BDFL says (http://permalink.gmane.org/gmane.comp.python.devel/146074):
"""Given that the claim "Python 2 doesn't support Unicode filenames" is factually incorrect (in Python 2.7, most filesystem calls in fact do support Unicode, at least on some platforms), I think individual functions in the os module that are found lacking should be considered bugs, and if someone goes through the effort to supply an otherwise acceptable fix, we shouldn't reject it on the basis that we don't want to consider supporting Unicode filenames."""
giampaolo commentedon May 24, 2014
Ok, I will go on then.
What do you mean?
Agreed.
What do you mean? Can you provide an example?
serhiy-storchaka commentedon May 24, 2014
You deleted "return NULL;" after "if (!PyArg_ParseTuple(...))" in the
posix_mkfifo() function.
For example test_statvfs should check that os.statvfs(TESTFN_UNICODE) ==
os.statvfs(TESTFN_UNICODE_ENCODED) (where TESTFN_UNICODE_ENCODED is relevant
8-bit str).
serhiy-storchaka commentedon May 31, 2020
Python 2.7 is no longer supported.