bpo-37261: Document sys.unraisablehook corner cases

[vstinner]

Document reference cycle and resurrected objects issues in
sys.unraisablehook and threading.excepthook documentations.

https://bugs.python.org/issue37261

[pablogsal]

Shouldn't it make more sense to not pass the object if it happens in a delicate moment to avoid resurrection? Resurrection can be tricky especially if it can happen in a very delicate way like what we are documenting here. We are already avoiding passing the object in places where is complicated or close to finalization like:

cpython/Modules/gcmodule.c

Lines 927 to 935 in 9549203

	if ((clear = Py_TYPE(op)->tp_clear) != NULL) {
	Py_INCREF(op);
	(void) clear(op);
	if (PyErr_Occurred()) {
	_PyErr_WriteUnraisableMsg("in tp_clear of",
	(PyObject*)Py_TYPE(op));
	}
	Py_DECREF(op);
	}

[vstinner]

@pablogsal: I was wrong, the resurrected object is not finalized twice. Finalization only occurs once. First, I didn't understand why test_writer_close_error_on_close() of test_io logs 2 unraisable exceptions. In fact, the first comes from BufferedRWPair. catch_unraisable_exception() resurrects BufferedRWPair which prevents its deallocation. Once BufferedRWPair is deallocated (at catch_unraisable_exception() exit which does "del self.unraisable"), BufferedWriter logs a second unraisable exception. Again, it's only deallocated when the second catch_unraisable_exception() exit.

I modified the test to use two (nested) catch_unraisable_exception().

[vstinner]

@pablogsal:

Resurrection can be tricky especially if it can happen in a very delicate way like what we are documenting here.

I don't see how resurrection is "tricky". Since PEP 442 -- Safe object finalization, Python 3.4 finalization is now clearly separated from deallocation. When an object is resurected, it may become "unusable", but using it should not crash Python. You must not use an object after it's deallocated, but... you cannot: the last strong reference to the object is gone, the object is no longer accessible.

[vstinner]

For the very specific case of delete_garbage() in gcmodule.c, I agree that the object must not be passed to sys.unraisablehook, since tp_clear() can make an object inconsistent or "broken". Using it can crash Python.

But for the general case of an exception occurred during an object finalization, IMHO it's fine to resurrect it in a hook.

Well, this PR advices to avoid storing an object to avoid object resurrection :-)

[vstinner]

cc @serhiy-storchaka @tirkarthi

[miss-islington]

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

[bedevere-bot]

GH-14089 is a backport of this pull request to the 3.8 branch.