Unicode to PHP exec

Question

I have a Python file I'm calling with PHP's exec function. Python then outputs a string (apparently Unicode, based on using isinstance), which is echoed by PHP. The problem I'm running into is that if my string has any special characters in it (like the degree symbol), it won't output. I'm sure I need to do something to fiddle with the encoding, but I'm not really sure what to do, and why.

EDIT: To get an idea of how I am calling exec, please see the following code snippet:

$tables = shell_exec('/s/python-2.6.2/bin/python2.6 getWikitables.py '.$title);

Python properly outputs the string when I call getWikitables.py by itself.

EDIT: It definitely seems to be something either on the Python end, or in transmitting the results. When I run strlen on the returned values in PHP, I get 0. Can exec only accept a certain type of encoding?

Can you post a code snippet illustrating how you are calling exec to get the string. — defines
– defines, Commented May 19, 2011 at 23:13

score 12 · Accepted Answer · 2012-02-15 02:41:56Z

12

Try setting the LANG environment variable immediately before executing the Python script per http://php.net/shell-exec#85095:

shell_exec(sprintf(
  'LANG=en_US.utf-8; /s/python-2.6.2/bin/python2.6 getWikitables.py %s',
    escapeshellarg($title)
));

(use of sprintf() to (hopefully) make it a little easier to follow the lengthy string)

You might also/instead need to do this before calling shell_exec(), per http://php.net/shell-exec#78279:

$locale = 'en_US.utf-8';
setlocale(LC_ALL, $locale);
putenv('LC_ALL='.$locale);

edited Feb 15, 2012 at 2:41

answered May 20, 2011 at 1:54

user212218

Sign up to request clarification or add additional context in comments.

1 Comment

anno Over a year ago

Oh god, I just spent 6 hours on this ! The second solution fixes the problem.

robsch · Accepted Answer · 2019-02-05 13:50:01Z

I have had a similar issue and solved it with the following. I don't understand why it is necessary, since I though all is already processed with UTF-8. Calling my Python script on the command line worked, but not with exec (shell_exec) via PHP and Apache.

According to a php forum entry this one is needed when you want to use escapeshellarg():

setlocale(LC_CTYPE, "en_US.UTF-8");

It needs to be called before escapeshellarg() is executed. Also, it was necessary to set a certain Python environment variable before the exec command (found an unrelated hint here):

putenv("PYTHONIOENCODING=utf-8");

My Python script evaluated the arguments like this:

sys.argv[1].decode("utf-8")

(Hint: That was required because I use a library to convert some arabic texts.)

So finally, I could imagine that the original question could be solved this way:

setlocale(LC_CTYPE, "en_US.UTF-8");
putenv("PYTHONIOENCODING=utf-8");
$tables = shell_exec('/s/python-2.6.2/bin/python2.6 getWikitables.py ' .
          escapeshellarg($title));

But I cannot tell anything regarding the return value. In my case I could output it to the browser directly without any problems.

Spent many, many hours to find that out... One of the situations when I hate my job ;-)

Thanh Trúc Thái · Accepted Answer · 2021-09-02 01:59:50Z

0

This worked for me

setlocale(LC_CTYPE, "en_US.UTF-8");
putenv("PYTHONIOENCODING=utf-8");
$tables = shell_exec('/s/python-2.6.2/bin/python2.6 getWikitables.py ' .
      escapeshellarg($title));

answered Sep 2, 2021 at 1:59

Thanh Trúc Thái

193 bronze badges

1 Comment

StevenSiebert Over a year ago

This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question.

David Rodrigues · Accepted Answer · 2011-11-13 18:09:25Z

-1

On php you can use methods like utf8_encode() or utf8_decode() to solve your problem.

edited Nov 13, 2011 at 18:09

user212218

answered May 19, 2011 at 22:35

David Rodrigues

12.6k17 gold badges68 silver badges98 bronze badges

3 Comments

cryptic_star Over a year ago

Neither worked - is it possible the problem is in transmitting from Python to php and I need to do something on the Python side?

David Rodrigues Over a year ago

Maybe, or can be the php output charset. Try it header('Content-type: text/html; charset=utf-8'); and tell me if you get a different result.

David Rodrigues Over a year ago

You need take a look on Multibyte library of PHP. This can tell you what is the returned encoding, and you can turn it to utf-8 and make visible.

Collectives™ on Stack Overflow

Unicode to PHP exec

4 Answers 4

1 Comment

Comments

1 Comment

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

1 Comment

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related