shutil rmtree fails on readonly files in Windows

[ivanradic]

BPO	19643
Nosy	@pfmoore, @tjguk, @bitdancer, @zware
Files	issue19643-doc.patch issue19643-doc.2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tjguk'
closed_at = <Date 2014-05-07.17:34:58.251>
created_at = <Date 2013-11-18.12:11:11.635>
labels = ['type-feature', 'library', 'OS-windows']
title = 'shutil rmtree fails on readonly files in Windows'
updated_at = <Date 2014-10-16.17:37:23.941>
user = 'https://bugs.python.org/ivanradic'

bugs.python.org fields:

activity = <Date 2014-10-16.17:37:23.941>
actor = 'paul.moore'
assignee = 'tim.golden'
closed = True
closed_date = <Date 2014-05-07.17:34:58.251>
closer = 'tim.golden'
components = ['Library (Lib)', 'Windows']
creation = <Date 2013-11-18.12:11:11.635>
creator = 'ivan.radic'
dependencies = []
files = ['35168', '35170']
hgrepos = []
issue_num = 19643
keywords = ['patch']
message_count = 27.0
messages = ['203285', '203297', '203298', '203301', '203308', '203313', '203314', '203315', '203316', '208647', '208651', '208654', '208657', '208662', '213867', '217997', '218021', '218023', '218044', '218051', '218052', '218053', '218054', '218063', '229538', '229541', '229542']
nosy_count = 8.0
nosy_names = ['paul.moore', 'tim.golden', 'r.david.murray', 'python-dev', 'zach.ware', 'Jovik', 'ivan.radic', 'sushisource']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19643'
versions = ['Python 3.5']

[ivanradic]

shutil.rmtree works nice on Windows until it hits file with read only attribute set. Workaround is to provide a onerror parameter as a function that checks and removes file attribute before attempting to delete it. Can option to delete read_only files be integrated in shutil.rmtree?

Example output in In Python 2.7:
shutil.rmtree("C:\\2")

Traceback (most recent call last):
  File "<pyshell#60>", line 1, in <module>
    shutil.rmtree("C:\\2")
  File "C:\Program Files (x86)\Python.2.7.3\lib\shutil.py", line 250, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Program Files (x86)\Python.2.7.3\lib\shutil.py", line 248, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: 'C:\\2\\read_only_file.txt'

Example output in In Python 3.3:
shutil.rmtree("C:\\2")
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    shutil.rmtree("C:\\2")
  File "C:\Program Files (x86)\Python.3.3.0\lib\shutil.py", line 460, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files (x86)\Python.3.3.0\lib\shutil.py", line 367, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Program Files (x86)\Python.3.3.0\lib\shutil.py", line 365, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\2\\read_only_file.txt'

[bitdancer]

You are essentially asking that we have an option to make the windows behavior mirror the posix behavior? (A read only file in a writable directory can be deleted in posix, since only the directory entry, not the file, is being deleted.)

That makes some sense to me. I wonder what the windows devs think?

[tjguk]

This, unfortunately, is the classic edge-case where intra-platform consistency and inter-platform consistency clash.

I (on Windows) would certainly be surprised if a tree delete removed read-only files without my specifying some kind of override. I understand why cross-platform behaviour might be preferred, and I'm open to being convinced, but I start at -0.

[bitdancer]

Well, it would *definitely* need to be a new explicit option whose default value was the current behavior.

[ivanradic]

You are essentially asking that we have an option to make the windows behavior mirror the posix behavior? (A read only file in a writable directory can be deleted in posix, since only the directory entry, not the file, is being deleted.)

Actually, your explanation is perfect.
I want to be able to remove some directory after I am done using it. When similar operation is done through file manager, dialog pops up asking for confirmation, I would like to have function parameter equivalent of "yes to all" dialog that file manager gives me.

The thing is, anyone working with files is used to think in "rm -rf" kind of way, and on Windows read_only files break this workflow. I discovered this problem few days ago when I was working on custom backup script that needs to work both on Linux (at home) and Windows (at work). Currently, I need to have some extra *windows only* code just to be able to successfully remove a directory.

Quick Google search discovered the workaround (http://stackoverflow.com/questions/1889597/deleting-directory-in-python), so I am set, but the original
question: "Why oh why is this such a pain?"
and the comment: "Maybe nobody has taken the five minutes to file a bug at bugs.python.org"
resonated in my head long enough to give it a try.

For me it makes sense to have this option configurable. And it make a ton of sense to support one line equivalent of "rm -rf".

[zware]

I like the idea of a remove_readonly flag. I was going to say that I'm a bit worried about the fact that shutil.rmtree already has a couple of keyword arguments, but it's nowhere near what, say, copytree has. Call me +0.75.

[bitdancer]

That's not a good name for the flag. The problem is that 'read-only' means different things on Windows than it does on Unix. Which is why I suggested that the flag control whether or not it acts "posix like" on Windows. So perhaps 'posix_compat'? That feels a bit odd, though, since on posix it does nothing...so it's really about behavioral consistency across platforms....

Hmm. It's really hard to think of a name that conveys succinctly what we are talking about here.

A more radical notion would be something like 'delete_control' and have it be tri-valued: 'unixlike', 'windowslike', and 'native', with the default being native. Bad names, most likely, but you get the idea. The disadvantage is that it would be even more code to implement ;)

[tjguk]

TBH I'm still fairly -0 and edging towards -0.5. If we didn't already
have two keyword args I might be convinced towards a jfdi=True flag.
But, as the OP described, the current params already allow for a
workaround of sorts and another param of the semantics we're discussing
would surely just be confusing?

(Or you could just "preprocess" on Windows, eg attrib -r /s)