Skip to content

Unicode problem with TZ #45381

Closed
Closed

Description

@theller
BPO 1040
Nosy @loewis, @theller, @amauryfa

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/loewis'
closed_at = <Date 2007-08-30.14:40:27.467>
created_at = <Date 2007-08-28.06:05:53.228>
labels = ['OS-windows']
title = 'Unicode problem with TZ'
updated_at = <Date 2007-08-30.14:40:27.450>
user = 'https://github.com/theller'

bugs.python.org fields:

activity = <Date 2007-08-30.14:40:27.450>
actor = 'loewis'
assignee = 'loewis'
closed = True
closed_date = <Date 2007-08-30.14:40:27.467>
closer = 'loewis'
components = ['Windows']
creation = <Date 2007-08-28.06:05:53.228>
creator = 'theller'
dependencies = []
files = []
hgrepos = []
issue_num = 1040
keywords = []
message_count = 5.0
messages = ['55351', '55352', '55426', '55427', '55481']
nosy_count = 3.0
nosy_names = ['loewis', 'theller', 'amaury.forgeotdarc']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1040'
versions = ['Python 3.0']

Activity

  1. theller commented on Aug 28, 2007

    @theller
    Author

    In my german version of winXP SP2, python3 cannot import the time module:

    c:\svn\py3k\PCbuild>python_d
    Python 3.0x (py3k:57600M, Aug 28 2007, 07:58:23) [MSC v.1310 32 bit
    (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import time
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    UnicodeDecodeError: 'utf8' codec can't decode bytes in position 9-11:
    invalid data
    [36719 refs]
    >>> ^Z

    The problem is that the libc '_tzname' variable contains umlauts. For
    comparison, here is what Python2.5 does:

    c:\svn\py3k\PCbuild>\python25\python
    Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
    (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import time
    >>> time.tzname
    ('Westeurop\xe4ische Normalzeit', 'Westeurop\xe4ische Normalzeit')
    >>>
  2. theller commented on Aug 28, 2007

    @theller
    Author

    BTW, setting the environment variable TZ to, say, 'GMT' makes the
    problem go away.

  3. self-assigned this
    on Aug 29, 2007
  4. amauryfa commented on Aug 29, 2007

    @amauryfa
    Contributor

    I have a patch for this, which uses MBCS conversion instead of relying
    on the default utf-8 (here and several other places). Tested on a French
    version of winXP.

    Which leads me to the question: should Windows use MBCS encoding by
    default when converting between char* and PyUnicode, and not utf-8?
    There are some other tracker items which would benefit from this.

    After all, C strings can only come from 1) python code, 2) system i/o
    and messages, and 3) constants in source code.
    IMO, 1) can use the representation it prefers, 2) would clearly lead to
    less error if handled as MBCS and 3) only uses 7bit ascii.
    There is very little need for utf-8 here.

  5. theller commented on Aug 29, 2007

    @theller
    Author

    IMO the very best would be to avoid as many conversions as possible by
    using the wide apis on Windows. Not for _tzname maybe, but for env
    vars, sys.argv, sys.path, and so on. Not that I would have time to work
    on that...

  6. loewis commented on Aug 30, 2007

    loewismannequin
    Mannequin

    This is now fixed in r57720.

    Using wide APIs would be possible through GetTimeZoneInformation,
    however, then TZ won't be supported anymore (unless the CRT code to
    parse TZ is duplicated).

  7. transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions