Concise way to capture stderr output in an exception when calling a process from Python

I’d like to call an external executable from a Python script. If it exits with a nonzero code, I’d like an exception containing the return code and the stderr output. The exception will be caught and logged by the UncaughtExceptionHook. The subprocess.check_call(…) would be nice and concise, but it didn’t add the stderr output to the subprocess.CalledProcessError exception.

subprocess.check_call(
    ["wget.exe", 
        "--page-requisites", "--span-hosts", "--convert-links",
        sp_url.url],
    text=True)``

Here’s a snippet which explicitly generates subprocess.CalledProcessError exception with stderr. It works, but it’s more verbose.

with subprocess.Popen(
    ["wget.exe", "--page-requisites", "--span-hosts", "--convert-links",
        sp_url.url],
    stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True
) as proc:
    try:
        (stdout, stderr) = proc.communicate(timeout=60)
    except (subprocess.TimeoutExpired, subprocess.CalledProcessError) as exc:
        proc.kill()
        raise   # Re-throw the exception.  The unhandled exception hook will log it, then sys.exit() the script.

if proc.returncode != 0:
    raise subprocess.CalledProcessError(proc.returncode, proc.args, output=stdout, stderr=stderr)     # the unhandled exception hook (catch-all) will log the exception and exit

Here’s my exception logging code.

def excepthook(self, exc_type, exc_value, exc_traceback):

    exc_attributes = ""
    for name, value in vars(exc_value).items():
        exc_attributes += f"{name}: {value}, "

    self._logger.exception(f"Uncaught exception hook.  {repr(exc_value)},  {exc_attributes} ", 
                           exc_info=(exc_type, exc_value, exc_traceback))
    sys.exit(1)

edit:

Come to think of it, I could create a helper function out of the verbose variant. It would be concise from the calling code's point of view.

In the event of success, do you need the stdout captured, or are the side effects of the process sufficient? [from comments]

Someday I may want to call a process and capture its stdout. That's a general case. For the processes which I'm actually calling at this time, if the process completes successfully, then I don't need the stdout. I only need files (side effects) created by the process.

Is it necessary from your perspective to impose a timeout, or is that just incidental to the fact that your current solution uses .communicate ? [from comments]

The timemout argument is just incidental because I'm calling .communicate.

posted 1 day ago

CC BY-NC-SA 4.0

7h ago by trichoplax‭

Nick Alexeev‭

51 reputation 1 1 6 3