Enhancement request : parametrize internal decode of VARCHAR2 data in Python 3

[shoyu63]

When there are corrupted data in a VARCHAR2 (for example \x90 in a WINDOWS-1252 database), we can't fetch the corrupted record because of a cx_Oracle internal decode error.
For example:

(<class 'UnicodeDecodeError'>, UnicodeDecodeError('charmap', b'ELLE REVIENDRA VE
RS NOUS - BIEN PLAC\x90\xc9S - ELLE ATTEND SON CLIENT', 36, 37, 'character maps
to <undefined>'), <traceback object at 0x0000000003C6B5C8>)

With Python 3, there is no way in cx_Oracle (v6.2) to specify the "errors" parameter to the decode() that takes place internally.

It might be nice to be able to

1. set this parameter.
   and/or
2. bypass the internal "decode" to have a direct access to the binary data of the STRING and FIXED_CHAR fields.

Thank you.
Stéphane

[anthony-tuininga]

Using code like the following you can now work around bad data in your database. Let me know if that works for you!

def OutputTypeHandler(cursor, name, defaultType, size, precision, scale):
    if defaultType == cx_Oracle.STRING:
        return cursor.var(defaultType, size, arraysize = cursor.arraysize,
                encodingErrors = "replace")

cursor.outputtypehandler = OutputTypeHandler

cursor.execute("select column1, column2 from SomeTableWithBadData")

[shoyu63]

Hello,
It looks great for me. I will use it as soon as the new version is available.
Thank you.
Stéphane.

[cjbj]

@anthony-tuininga is there any performance impact by having this always enabled?

[anthony-tuininga]

The default error handler is "strict" but there are other error handlers besides "replace" which can be seen here: https://docs.python.org/3/library/codecs.html#error-handlers. There is also the possibility of implementing your own error handler. So I'm not sure what "always enabled" would look like!

[anthony-tuininga]

cx_Oracle 6.4 has been released with this enhancement included in it.