tempfile.TemporaryDirectory() context manager can fail to propagate exceptions generated within its context #85034

Activity

1. granchester commented on Jun 3, 2020

   

   granchestermannequin

   on Jun 3, 2020

   MannequinAuthor

   More actions

   When an exception occurs within a tempfile.TemporaryDirectory() context
   and the directory cleanup fails, the _cleanup exception_ is propagated,
   not the original one. This effectively 'masks' the original exception,
   and makes it impossible to catch using a simple 'try'/'except' construct.

   ----------------------------------------------------------------------------
   Code like this:

     import tempfile
     import os
     import sys

   try:
   with tempfile.TemporaryDirectory() as tempdir:
   print(tempdir)
   # some code happens here

     except ArithmeticError as exc:
         print('An arithmetic error occurred: {}'.format(exc))
   
     print('Continuing...')

   is effective at catching any ArithmeticError which occurs in the
   code fragment but is not otherwise handled. However if, in addition,
   an error occues in cleaning up the temporary directory, the exception
   which occurred in the code is replaced by the cleanup exception, and is
   not be propagated to be caught by the 'except' clause.

   For example:

     import tempfile
     import os
     import sys

   try:
   with tempfile.TemporaryDirectory() as tempdir:
   print(tempdir)
   n = 1 / 0

     except ArithmeticError as exc:
         print('An arithmetic error occurred: {}'.format(exc))
   
     print('Continuing...')

   produces this:

   /tmp/tmp_r2sxqgb
   An arithmetic error occurred: division by zero
   Continuing...

   but this:

     import tempfile
     import os
     import sys

   try:
   with tempfile.TemporaryDirectory() as tempdir:
   print(tempdir)
   os.rmdir(tempdir) # this new line is the only difference
   n = 1 / 0

     except ArithmeticError as exc:
         print('An arithmetic error occurred: {}'.format(exc))
   
     print('Continuing...')

   produces this:

     /tmp/tmp_yz6zyfs
     Traceback (most recent call last):
       File "tempfilebug.py", line 9, in <module>
         n = 1 / 0
     ZeroDivisionError: division by zero
   
     During handling of the above exception, another exception occurred:
   
     Traceback (most recent call last):
       File "tempfilebug.py", line 9, in <module>
         n = 1 / 0
       File "/usr/lib/python3.6/tempfile.py", line 948, in __exit__
         self.cleanup()
       File "/usr/lib/python3.6/tempfile.py", line 952, in cleanup
         _rmtree(self.name)
       File "/usr/lib/python3.6/shutil.py", line 477, in rmtree
         onerror(os.lstat, path, sys.exc_info())
       File "/usr/lib/python3.6/shutil.py", line 475, in rmtree
         orig_st = os.lstat(path)
     FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp_yz6zyfs'

   and the program exits with the top-level code having no chance to catch
   the ZeroDivisionError and continue execution. (To catch this exception,
   the top-level code would need to know to catch FileNotFoundError.)

   My view is that if an exception happens within a TemporaryDirectory
   context, *and* there is an exception generated as a result of the cleanup
   process, the original exception is likely to be more significant, and
   should be the exception which is propagated, not the one generated by
   the cleanup.

   ----------------------------------------------------------------------------
   System info:

   $ python3 --version
   Python 3.6.9
   
   $ lsb_release -a
   No LSB modules are available.
   Distributor ID: Ubuntu
   Description:    Ubuntu 18.04.4 LTS
   Release:        18.04
   Codename:       bionic
2. granchester

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jun 3, 2020
3. remilapeyre commented on Jun 4, 2020

   

   remilapeyremannequin

   on Jun 4, 2020

   Mannequin

   More actions

   This was fixed in e9b51c0 so it's fixed in Python 3.8 and 3.9. Maybe it should have been backported to 3.7.

   Python 3.6 only receives security fixes now.

   See also bpo-26660 and bpo-35144.
4. remilapeyre

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   3.7 (EOL)end of lifeend of life

   and removed

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   on Jun 4, 2020
5. iritkatriel commented on Nov 25, 2020

   

   iritkatriel

   on Nov 25, 2020

   Member

   More actions

   Too late for 3.7 now, closing.
6. iritkatriel

   closed this as completedon Nov 25, 2020
7. iritkatriel

   closed this as completedon Nov 25, 2020
8. ezio-melotti

   transferred this issue fromon Apr 10, 2022