Remove old-deprecated ElementTree features (part 2)

[serhiy-storchaka]

BPO	36543
Nosy	@scoder, @tiran, @encukou, @serhiy-storchaka, @hroncok, @miss-islington, @rahul-kumi
PRs	[3.9] bpo-36543: Remove old-deprecated ElementTree features. #12707 bpo-39937: modify iter() suggestion to Element.iter() #18937 bpo-36543: Remove the xml.etree.cElementTree module. #19108 bpo-36543: What's new: Document how to replace xml.etree.cElementTree #19188 Revert "bpo-36543: Remove the xml.etree.cElementTree module." #19920 bpo-36543: Restore cElementTree and mark it for removal in 3.10 #19921 bpo-36543: Revert "bpo-36543: Remove the xml.etree.cElementTree module." #20117 [3.9] bpo-36543: Revert "bpo-36543: Remove the xml.etree.cElementTree module." (GH-20117) #20780

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2020-06-10.16:04:24.719>
created_at = <Date 2019-04-06.16:06:22.348>
labels = ['expert-XML', 'type-feature', '3.9']
title = 'Remove old-deprecated ElementTree features (part 2)'
updated_at = <Date 2021-01-05.17:46:41.989>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-01-05.17:46:41.989>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2020-06-10.16:04:24.719>
closer = 'scoder'
components = ['XML']
creation = <Date 2019-04-06.16:06:22.348>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 36543
keywords = ['patch']
message_count = 27.0
messages = ['339533', '339535', '350938', '364802', '365190', '368083', '368105', '368108', '368111', '368113', '368118', '368120', '368121', '368122', '368123', '368127', '368135', '371210', '371212', '371214', '384390', '384399', '384400', '384401', '384412', '384415', '384419']
nosy_count = 8.0
nosy_names = ['scoder', 'christian.heimes', 'eli.bendersky', 'petr.viktorin', 'serhiy.storchaka', 'hroncok', 'miss-islington', 'rahul-kumi']
pr_nums = ['12707', '18937', '19108', '19188', '19920', '19921', '20117', '20780']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue36543'
versions = ['Python 3.9']

[serhiy-storchaka]

The proposed PR removes old-deprecated ElementTree features.

• Methods Element.getchildren(), Element.getiterator() and ElementTree.getiterator() deprecated in 2.7 and 3.2. They were deprecated in the documentation only, and started to emit a warning in 3.8. Use list(elem) or iteration instead of getchildren(), methods iter() instead of getiterator().

• The xml.etree.cElementTree module deprecated in 3.3. It was deprecated documentation only because adding a runtime warning will cause more harm than removing it because of the common idiom of using it since Python 2:

  try:
  import xml.etree.cElementTree as ET
  except ImportError:
  import xml.etree.ElementTree as ET

TODO: Add a What's New entry after the start of developing 3.9.

[serhiy-storchaka]

See also bpo-29209.

[serhiy-storchaka]

New changeset f02ea62 by Serhiy Storchaka in branch 'master':
bpo-36543: Remove old-deprecated ElementTree features. (GH-12707)
f02ea62

[serhiy-storchaka]

New changeset b33e525 by Serhiy Storchaka in branch 'master':
bpo-36543: Remove the xml.etree.cElementTree module. (GH-19108)
b33e525

[serhiy-storchaka]

New changeset 6467134 by Miro Hrončok in branch 'master':
bpo-36543: What's new: Document how to replace xml.etree.cElementTree (GH-19188)
6467134

[tiran]

For the record, I'm quite unhappy that xml.etree.cElementTree was removed without going through a proper active deprecation cycle with plenty of head start. The removal came as a surprise to me -- and I'm a core dev, author of PEP-594 and owner of the defusedxml package. xml.etree.cElementTree module was also never added to PEP-4 and is still not mentioned as removed in PEP-4.

This changeset broke defusedxml, a package with over 10M downloads a month and 57k dependencies on Github. I'm sure that the removal is going to cause more trouble for other packages, too.

In PEP-594 I deliberately included a long deprecation phase for all packages with at least one release deprecation warnings. I gave users plenty of time to plan for removal.

[scoder]

Christian, I understand your complaint, but I've actually never seen code in the wild that didn't provide a fallback for the import, usually something like what Serhiy spelled out and explained above:

try:
    import xml.etree.cElementTree as ET
except ImportError:
    import xml.etree.ElementTree as ET

This makes it inconvenient for users to emit a deprecation warning for the import, because it would impact perfectly future proof code like the above, without a good way for users to work around it by adapting their code, because the above import order is actually what they want. And there's a lot of such code.

I personally believe that the breakage will be quite limited. But that's a belief, not a fact. I don't have numbers to back it.

[serhiy-storchaka]

xml.etree.cElementTree should be treated in the same way as cStringIO and cPickle -- they where separate modules in the past, but now the acceleration is used by default in io.StringIO and pickle. It looks an oversign that it was not removed in 3.0.

xml.etree.cElementTree was deprecated since 3.3. I think 6 releases (3.3-3.8) is long enough period. It was not possible to emit a deprecation warning at runtime for reasons mentioned above. And we did not have deprecation warnings for cStringIO and cPickle for the same reasons.

defusedxml is the only known to me package which does not fall back to xml.etree.ElementTree. I hope it is easy to fix it. In Python xml.etree.cElementTree is the same as xml.etree.ElementTree, so you can just import xml.etree.ElementTree. And you can use the above mentioned idiom if you still support Python 2.

[hroncok]

Some stats: See https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON39&hide_resolved=0 and Ctrl+F for cElementTree.

[scoder]

Thanks Miro, that's beautiful. Good to know that I'm not the only one who doesn't read documentation. ;-)

So, how do we deal with this? We can't warn about the deprecation without annoying everyone. We can't just remove the empty module without breaking lots of code. We also can't really let the module stick around forever and let people write more code that uselessly imports it.

Any ideas?