init_oracle_client lib dir with chinese

[13033266796]

1. What versions are you using?

platform.platform: Windows-10-10.0.17763-SP0
sys.maxsize > 2**32: True
platform.python_version: 3.8.10

oracledb.version: 1.3.2

2. Is it an error or a hang or a crash?
   oracledb.init_oracle_client(lib_dir="D:\哈哈哈哈哈哈哈哈哈哈或或\instantclient_11_2")
   Traceback (most recent call last):
   File "D:\Python38\lib\code.py", line 90, in runcode
   exec(code, self.locals)
   File "", line 1, in
   File "src\oracledb\impl/thick/utils.pyx", line 458, in oracledb.thick_impl.init_oracle_client
   File "src\oracledb\impl/thick/utils.pyx", line 482, in oracledb.thick_impl.init_oracle_client
   File "src\oracledb\impl/thick/utils.pyx", line 403, in oracledb.thick_impl._raise_from_info
   oracledb.exceptions.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "The specified module could not be found". See https://python-oracledb.readthedocs.io/en//latest/user_guide/initialization.html#setting-the-oracle-client-library-directory for help

3. What error(s) or behavior you are seeing?

init_oracle_client lib dir with chinese

How to solve the problem？

[cjbj]

Does it work if you use non-Chinese characters?

Does it work if you set the PATH environment variable (and call init_oracle_client() without any parameters)?

[13033266796]

Does it work if you use non-Chinese characters?
It work if use non-Chinese characters

Does it work if you set the PATH environment variable (and call init_oracle_client() without any parameters)?
I try to set PATH:
D:\111\instantclient_11_2
D:\哈哈哈哈哈哈哈哈哈哈或或\instantclient_11_2
None of them will take effect

[cjbj]

Can you set DPI_DEBUG_LEVEL to 64 (see the doc) and share the log?

[13033266796]

Is it like this ?

((venv) D:\A_projects\dmp_tools>set DPI_DEBUG_LEVEL=64

(venv) D:\A_projects\dmp_tools>echo %DPI_DEBUG_LEVEL%
64

(venv) D:\A_projects\dmp_tools>python
Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import oracledb
oracledb.init_oracle_client(lib_dir=r"D:\哈哈哈哈哈哈哈哈哈哈哈哈哈合伙\instantclient_11_2")
ODPI [15820] 2023-11-24 10:53:34.077: ODPI-C 5.0.1
ODPI [15820] 2023-11-24 10:53:34.078: debugging messages initialized at level 64
ODPI [15820] 2023-11-24 10:53:34.081: Context Parameters:
ODPI [15820] 2023-11-24 10:53:34.082: Oracle Client Lib Dir: D:?????????????????????????????????????????????\instantclient_11_2
ODPI [15820] 2023-11-24 10:53:34.085: Environment Variables:
ODPI [15820] 2023-11-24 10:53:34.087: PATH => "D:\A_projects\dmp_tools\venv\Scripts;D:\111\instantclient_11_2;D:\Python310\Scripts;D:\Python310;D:\Python36\Scripts;D:\Python36;D:\Python38\Scripts;D:\Python38;D:\SecureCrt_FX;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\Git\cmd;C:\Program Files\TortoiseSVN\bin;D:\Goang\bin;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit;D:\NodeJs;C:\Users\User\AppData\Local\Microsoft\WindowsApps;;D:\PyCharm 2022.1\bin;;D:\DataGrip 2022.2.1\bin;;D:\GoLand 2022.2.1\bin;;C:\Users\User\go\bin;D:\Microsoft VS Code\bin;C:\Users\User\AppData\Roaming\npm"
ODPI [15820] 2023-11-24 10:53:34.104: load in parameter directory
ODPI [15820] 2023-11-24 10:53:34.105: load in dir D:?????????????????????????????????????????????\instantclient_11_2
ODPI [15820] 2023-11-24 10:53:34.107: load with name D:?????????????????????????????????????????????\instantclient_11_2/oci.dll
ODPI [15820] 2023-11-24 10:53:34.111: load by OS failure: The specified module could not be found
Traceback (most recent call last):
File "", line 1, in
File "src\oracledb\impl/thick/utils.pyx", line 476, in oracledb.thick_impl.init_oracle_client
File "src\oracledb\impl/thick/utils.pyx", line 500, in oracledb.thick_impl.init_oracle_client
File "src\oracledb\impl/thick/utils.pyx", line 421, in oracledb.thick_impl._raise_from_info
oracledb.exceptions.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "The specified module could not be found". See https://python-oracledb.readthedocs.io/en/latest/user_guide/initialization.html for help

[anthony-tuininga]

Yes, that is what @cjbj meant. I took a quick peek at this. It seems that if you use LoadLibraryA(), even with a simple search name, that the underlying code is incapable of handling paths containing Chinese characters. Doing the same on Linux works as expected. Since this appears to be a Windows limitation it is not a bug in python-oracledb or ODPI-C itself. We may wish to document this limitation but it is unlikely that we can eliminate it. I am not a Windows expert, however, so if you have any suggestions I'd be happy to hear them!

[PaleNeutron]

@anthony-tuininga I found the problem here:

python-oracledb/src/oracledb/impl/thick/utils.pyx

Lines 501 to 503 in c513952

	if lib_dir is not None:
	lib_dir_bytes = lib_dir.encode()
	params.oracleClientLibDir = lib_dir_bytes

lib_dir.encode() should use operating system encoding.

For chinese it should be gb2312 or cp936:

> [System.Text.Encoding]::Default


BodyName          : gb2312
EncodingName      : 简体中文(GB2312)
HeaderName        : gb2312
WebName           : gb2312
WindowsCodePage   : 936
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
IsSingleByte      : False
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 936

I add a system_encoding parameter to init function and tested on my machine, it works.

def init_oracle_client(lib_dir=None, config_dir=None, error_url=None,
                       driver_name=None, system_encoding=None):

But I think this is not the best solution. ctypes.CDLL support load dll from path contains Chinese character without passing encoding and it seems it use windows new api, refer https://peps.python.org/pep-0529/ .

May be we should use it instead of current LoadLibraryA.

[anthony-tuininga]

What is the output of sys.getfilesystemencoding() in your case? Assuming that encoding is one of the ones you mentioned, that might work well. Another option is to allow the specification of encoded bytes instead of requiring a string and performing the encode.

[anthony-tuininga]

I changed the code page on my Windows VM to UTF-8 and then it worked just fine. So that brings up a number of options, I think.

• simply allow encoded bytes to be specified in a call to init_oracle_client() which puts the onus on the user to know the encoding and perform the encode prior to the call
• let me know the "right" way to find out what encoding to use (I am not a Windows user so my knowledge of such things is very limited)

Note that the underlying library (ODPI-C) is not Python-specific at all, so it can't make use of Python features!

[PaleNeutron]

Both sys.getfilesystemencoding() or sys.getdefaultencoding() will return 'utf-8' on Chinese windows machine.

See the test below

Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, locale
>>> sys.getfilesystemencoding()
'utf-8'
>>> sys.getdefaultencoding()
'utf-8'
>>> locale.getencoding()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'locale' has no attribute 'getencoding'
>>> # above function is only for python>=3.11
>>> import ctypes
>>> from ctypes import windll
>>> windll.kernel32.GetConsoleOutputCP()
936
>>> exit()

C:\Users\palen>python
Python 3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getencoding()
'cp936'
>>> exit()

I recommend checking python version to choose locale.getencoding() or "cp" + str(windll.kernel32.GetConsoleOutputCP()) .

[anthony-tuininga]

Ok. So if you make thise change to python-oracledb:

diff --git a/src/oracledb/impl/thick/utils.pyx b/src/oracledb/impl/thick/utils.pyx
index dc4f69ae..39e065b9 100644
--- a/src/oracledb/impl/thick/utils.pyx
+++ b/src/oracledb/impl/thick/utils.pyx
@@ -499,7 +499,10 @@ def init_oracle_client(lib_dir=None, config_dir=None, error_url=None,
         if config_dir is None:
             config_dir = C_DEFAULTS.config_dir
         if lib_dir is not None:
-            lib_dir_bytes = lib_dir.encode()
+            if isinstance(lib_dir, bytes):
+                lib_dir_bytes = lib_dir
+            else:
+                lib_dir_bytes = lib_dir.encode()
             params.oracleClientLibDir = lib_dir_bytes
         if config_dir is not None:
             config_dir_bytes = config_dir.encode()

And then in your code add this:

import locale
import oracledb

lib_dir = r"D:\哈哈哈哈哈哈哈哈哈哈或或\instantclient_11_2"
oracledb.init_oracle_client(lib_dir=lib_dir.encode(locale.getencoding()))

Does that work for you?

[PaleNeutron]

The code works.