unicode('foo', '.utf99') does not raise LookupError

[osvenskan]

BPO	1446043
Nosy	@akuchling, @doerwalter, @birkenfeld, @osvenskan
Files	UnexpectedValueError.txt: demo of the ValueError encodings-search.diff: patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/akuchling'
closed_at = <Date 2006-10-04.05:58:13.000>
created_at = <Date 2006-03-09.00:55:01.000>
labels = ['expert-unicode', 'release-blocker']
title = "unicode('foo', '.utf99') does not raise LookupError "
updated_at = <Date 2006-10-04.05:58:13.000>
user = 'https://github.com/osvenskan'

bugs.python.org fields:

activity = <Date 2006-10-04.05:58:13.000>
actor = 'nnorwitz'
assignee = 'akuchling'
closed = True
closed_date = None
closer = None
components = ['Unicode']
creation = <Date 2006-03-09.00:55:01.000>
creator = 'osvenskan'
dependencies = []
files = ['1916', '1917']
hgrepos = []
issue_num = 1446043
keywords = []
message_count = 8.0
messages = ['27716', '27717', '27718', '27719', '27720', '27721', '27722', '27723']
nosy_count = 5.0
nosy_names = ['akuchling', 'doerwalter', 'nnorwitz', 'georg.brandl', 'osvenskan']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1446043'
versions = ['Python 2.4']

[osvenskan]

A very minor inconsistency -- when I call unicode()
with an encoding that Python doesn't know about, it
usually returns a lookup error (e.g LookupError:
unknown encoding: utf99). But when the encoding begins
with a dot (ASCII 0x2e), Python instead gives a
ValueError: Empty module name. It is certainly correct
in raising an error, but it should raise a lookup
error, not a value error.

I've recreated this under Python 2.4.1/FreeBSD 6.0 and
2.3/OS X. See attachment for recreation steps.

[doerwalter]

Logged In: YES
user_id=89016

The problem is that after normalizing the encoding name a
module with this name is imported. Maybe
encodings/init.py:search_function should do:

if ".".join(filter(None, modname.split("."))) != modname:
   return None

[birkenfeld]

Logged In: YES
user_id=849994

Is it possible for an encoding name to contain dots at all?

If not, this would do too:
if '.' in modname: continue

[osvenskan]

Logged In: YES
user_id=1119995

There are encoding names that contain dots, such as
ANSI_X3.4-1968, ANSI_X3.4-1986 and ISO_646.IRV:1991 (as
reported by iconv). There are none in iconv's list that
begin with a dot.

Please note that the behavior of this function has been
discussed before in Python bugs 513666 and 960874. Apologies
for not referencing them in my original report.

Having stepped through the code, I understand how the
ValueError is getting generated. My frustration with this as
a programmer is that I want to write specific except clauses
for each possible exception that a method can raise, but
that's impractical if any exception is fair game on any
method. So I'm forced to use a catch-all except clause about
which the Python documentation says (wisely, IMHO), "Use
this with extreme caution, since it is easy to mask a real
programming error in this way!" While it is helpful to
document errors that a method is *likely* to raise, my code
needs to handle all possibilities, not just likely ones.

Perhaps the answer is just, "This is how Python works" and
if I feel it is a weakness in the language I need to take it
up on a different level.

[osvenskan]

Logged In: YES
user_id=1119995

I noticed that the documentation for unicode() says, "if the
encoding is not known, LookupError is raised". Regarding the
3rd parameter ("errors") to unicode(), the docs say, "Error
handling is done according to errors; this specifies the
treatment of characters which are invalid in the input
encoding. If errors is 'strict' (the default), a ValueError
is raised on errors..."
ref: http://docs.python.org/lib/built-in-funcs.html

That makes the code's current behavior doubly confusing
because a the documentation says that a ValueError is
reserved for indicating an undecodable byte sequence, not an
unknown encoding name.

[birkenfeld]

Logged In: YES
user_id=849994

I'd say that this should be fixed before 2.5 final.

Attached patch (the modname that's used for import may not
contain a dot anymore...)

[birkenfeld]

Logged In: YES
user_id=849994

Fixed in rev. 52075, 52076 (2.4), 52077 (2.5).