Idle: improve idle_test.htest

[terryjreedy]

BPO	21477
Nosy	@terryjreedy, @ned-deily
Superseder	bpo-21624: Idle: Improve htests
Files	21477-htest.txt: renamed and copied from 18104 week1.diff run-runall.diff run-runall-34.diff run-runall-27.diff htest-25052014.diff htest-25052014-34.diff htest-25052014-27.diff: Corresponding patch for 2.7 for test-25052014-34.diff htest-26052014-34.diff htest-26052014-27.diff htest-28052014-34.diff htest-28052014-27.diff htest-docstring-34.diff: Docstring change to reflect change in code. htest-docstring-27.diff: Docstring change to reflect change in code.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2014-06-01.04:44:08.199>
created_at = <Date 2014-05-12.04:08:37.972>
labels = ['expert-IDLE', 'type-feature']
title = 'Idle: improve idle_test.htest'
updated_at = <Date 2019-03-23.20:08:32.581>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2019-03-23.20:08:32.581>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2014-06-01.04:44:08.199>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2014-05-12.04:08:37.972>
creator = 'terry.reedy'
dependencies = []
files = ['35221', '35334', '35335', '35342', '35343', '35352', '35356', '35364', '35368', '35369', '35386', '35387', '35411', '35412']
hgrepos = []
issue_num = 21477
keywords = ['patch']
message_count = 24.0
messages = ['218312', '218830', '218839', '219021', '219055', '219060', '219061', '219066', '219096', '219107', '219108', '219109', '219161', '219190', '219205', '219207', '219209', '219210', '219292', '219335', '219336', '219484', '219487', '219535']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'ned.deily', 'jesstess', 'python-dev', 'Saimadhav.Heblikar']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = '21624'
type = 'enhancement'
url = 'https://bugs.python.org/issue21477'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[terryjreedy]

bpo-18104 created idle_test/htest.py. Besides adding more tests in other issues, some possible improvements to htest itself follow. Many depend on not reusing run, as it now it, in the runall loop. What is left in common to both might be factored out.

• For runall, reuse the same master box so the user does not have to click away each master box to see another (which also appears in a different position). Instead, rewrite the message and if still used, the button.

• Make sure tested widgets do not cover the master box. For configSectionNameDialog, an htest parameter was added to the class to change the usual positioning (which was intentionally centered over the parent).

• For most tests, eliminate the button and simply display the widget. The button is needed for testing message boxes when one wants to test different entries (the section name dialog). 'button=True' could be added to the spec where needed.

• If there is no button to restart a test, then closing the widget should, if possible, close the master. The editor window does this, but that would have to be changed if runall ran in one root.

• The pre-existing editor window test hides (withdraws) the master (root) window. This is not necessary and should be changed.

[python-dev]

New changeset 33a39dfc239e by Terry Jan Reedy in branch '2.7':
Issue bpo-21477: idle htests - lower case function names, other cleanups.
http://hg.python.org/cpython/rev/33a39dfc239e

New changeset 7c70198ec48e by Terry Jan Reedy in branch '3.4':
Issue bpo-21477: idle htests - lower case function names, other cleanups.
http://hg.python.org/cpython/rev/7c70198ec48e

[SaimadhavHeblikar]

Adds spec dicts for aboutDialog, ClassBrowser, PathBrowser, textView and configHelpSourceEdit and modifies the related modules appropriately.
The spec dicts(and the tests relating to) for editorwindow and help dialog have not been modified in this patch.

The tests work OK when run individually from respective modules. The widgets are placed below parent. Some widgets close the parent when they are closed, some do not.(I feel the parent should be manually closed by the user, because the user might want to re-run a test. This would also help in reusing root, when running all the tests together).

The runall() seems shaky at the moment. For eg, as mentioned above, some widgets close the parent when they are closed, some do not. This is causes build-up of parent dialogs. I did not try to fix it, because things will change once we agree upon factoring out run() and runall(), about reusing the same root window for runall(). If things dont change, I will work on fixing the buildup of dialogs.

If this patch seems "incomplete" because of the above issue, please provide feedback about the tests itself, when run individually.

[SaimadhavHeblikar]

Posting a cumulative patch of all htest written so far.

They include IOBinding, Tooltips, MultiStatusbar, tabbedpages, objectbrowser,
scrolledlist, dynOptionWidget, treewidget, widgetredirector,
colordelegator, calltip and multicall, besides aboutDialog, ClassBrowser, PathBrowser, textView and configHelpSourceEdit already posted before.
This patch is named week1.diff

*Improving run/runall - (run-runall.diff)
This patch concerns more with the way run/runall are executed. The only modification for this occurs in the file htest.py
I have streamlined run/runall into a single function. The behavior is same as the current htest.py, except for the reuse of a single root window and the addition of a "next" button.

The logic behind it is contained in line 206 to line 223 of htest.py when run-runall.diff is applied.
In short, a list of tuple containing(spec, kwds and callable object) is used to store the required information. Global variables maintain the above information only for the current test. Once a user finishes a test, the values of the global variables are updated to reflect the values for the next test. The "next" button is disabled for the case when a) it is the last test 2) when run() is called from a module

If this approach is desired, please say so. I will work on improving the responsiveness of the root window(Constant size of root dialog, ensuring buttons, labels etc stay in the "same place" and dont "jump" around).

[terryjreedy]

run-runall.diff import cleanly to 3.4. Running all the tests works well enough that I am applying this, with minor changes, to be a base for further patches. Good job.

The few problems I fixed:

1. AutoCompleteWindow.py had \n added at the beginning. I just deleted it.
2. 2 files had an unused 'import tkinter'. I deleted those lines. (Any new imports, that are used, should be 'import tkinter as tk' so that is the only place where the Tkinter/tkinter change shows up in the Idle file.)
3. patchcheck found 7 files with extra whitespace to be deleted.

Things to do starting from this.

1. See review comments on htest.run().

2. See review comments on text changes.

3. See review comments on individual tests. There will be more of these, but run changes and 4) below are the most important now.

4. Many wrappers create a separate root=Tk() (and mainloop) ins4tead of reusing the parent arg. The result is that clicking [Next] does not delete the test window, as it is not a child of the main window. The test window has to be closed separately. So unless the test *requires* a new root and mainloop, please delete them. (Modal dialogs have to be closed explicitly - message should say so.)

5. When I changed root = tk.Tk() to root = parent in _editor_window, the test seemed to continue to work fine. Why did you comment it out? Please restore it and make any change necessary to _editor_window to work with the revised run().

[terryjreedy]

Since I wanted to get this large patch applied as a basis for further work, I went ahead with the 2.7 backport. I am listing the issues in converting run-runall-34.diff to run-runall-27.diff to help Saimadhav do this in the future.

1. /tkinter/Tkinter/ 7 places (easy).

2. /print(xxx)/print xxx/ 3 places in 2 idlelib files,
   but leave print(widget.result) alone in htest

3. /importlib.machinery/imp/ in Pathbrowser.py

4. In textView.py, the 2.7 line
   text = file(filename, 'r').read()
   was changed in 3.x to two lines:
   with open(filename, 'r') as f:
   text = f.read()
   Since inserting or deleting a line in a diff file (as opposed to the within-line edits above) is tricky, and since this line is part of a chunk being deleted, I left this alone in the -27.diff resulting from the changes above. The diff uploaded included the handpatch changes.

(Note: some time ago, I converted all open statements to 'with open' in 2.7 and 3.x. I did not think of 'file(' as a synonym for 'open('. Any other uses of 'file(' should be converted also.)

5. The surprise I had forgotten about. 'nonlocal', used in the new version of run(), is new in 3.0! Fortunately, I remember the ugly hack we used in 2.x. It does mean, however, that the run code is different in 2.7. Future patches to run should only patch run().

6. ColorDelegator: the code in the 3.x test is displayed for me as
   �ó
   ��ﾁSc
   I replaced source with a string literal. I think we should use a revised version of this, with everything that should be colored, in 3.x also and skip the file read.

[python-dev]

New changeset ece24bcd1a6f by Terry Jan Reedy in branch '2.7':
Issue bpo-21477: Idle htest: merge and modify run and runall; add many tests.
http://hg.python.org/cpython/rev/ece24bcd1a6f

New changeset 038cbbef4539 by Terry Jan Reedy in branch '3.4':
Issue bpo-21477: Idle htest: merge and modify run and runall; add many tests.
http://hg.python.org/cpython/rev/038cbbef4539

[terryjreedy]

I just discovered today that Rietveld can do diffs between diffs, so to speak. For example,
http://bugs.python.org/review/21477/diff2/11941:11942/Lib/idlelib/idle_test/htest.py
showed the changes between patched htest in 3.4 and 2.7.

[SaimadhavHeblikar]

Modifications in htest-25052014.diff

1. ClassBrowser, PathBrowser, EditorWindow no longer close parent when closed

2. Sample code in _color_delegator changed to string, instead of reading from file.

3. String text change for Tooltip.

4. Adds htest for FormatParagraph, Percolator, EditorWindow(rather uncomment's spec), StackViewer, KeyBinding

5. Modification to run based on review comments at http://bugs.python.org/review/21477/diff/11937/Lib/idlelib/idle_test/htest.py

6. Other cosmetic changes to spec string 'msg' text.

When this diff(subject to passing review and feedback) is pushed, I will make a corresponding patch for 2.7

[python-dev]

New changeset d7eea8f608c2 by Terry Jan Reedy in branch '3.4':
Issue bpo-21477: Idle htest: modify run; add more tests.
http://hg.python.org/cpython/rev/d7eea8f608c2

[terryjreedy]

0. I pushed a slight revision of the patch. See 2) for the main change. In htest, I also did 3), corrected some line spacing issues, and changed 'focussing' to the US spelling 'focusing'. A tested 2.7 version of the attached, htest-25052014-34.py, is top priority.

1. parent.mainloop() - ClassBrower, EditorWindow, PathBrowser: since mainloop is already running, this should have no effect. I had no problem with closing of child window closing parent. Perhaps there is an OS difference, perhaps related to the focus difference. Or this might even be a tk or tkinter bug in not masking OS differences. If the line is necessary for #nix, it seems harmless on Windows.

2. Let us remember that the purpose of htest is to test what cannot be sensibly tested with automated unittests. That includes the visual look of widgets and some behaviors that either cannot be automatically tested or that we currently do not know how to test.

The format paragragh methods and functions are well tested, better than a human will, in the current test_formatgraph unittest. I removed this part of the patch. (But see 5. below about moving it.)

When I wrote the notes that became htest.txt, a year ago, I had not written any tests other than for calltips. I listed almost all the files that had existig tests. So don't add new tests not mentioned in htest.txt without discussion. And question the existing and proposed tests listed there.

For instance, I presume we can use the Text.tag_methods to write an automated ColorDelegator test that checks that tagged words are as expected and that colors match the configured values. We can leave the current test (with patch), until replaced.

3. Class/PathBrowser: not only does double clicking 'not work', it causes an exception to be printed to console or shell, but which seems to be ignored as test continues. The exception is either like this

Exception in Tkinter callback
Traceback (most recent call last):
  File "F:\Python\dev\4\py34\lib\idlelib\run.py", line 121, in main
    seq, request = rpc.request_queue.get(block=True, timeout=0.05)
  File "F:\Python\dev\4\py34\lib\queue.py", line 175, in get
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "F:\Python\dev\4\py34\lib\tkinter\__init__.py", line 1487, in __call__
    return self.func(*args)
  File "F:\Python\dev\4\py34\lib\idlelib\TreeWidget.py", line 122, in flip
    self.item.OnDoubleClick()
  File "F:\Python\dev\4\py34\Lib\idlelib\ClassBrowser.py", line 174, in OnDoubleClick
    edit = PyShell.flist.open(self.file)
AttributeError: 'module' object has no attribute 'flist'

or with just the AttributeError. Unless and until we can suppress the traceback, the message should be changed from 'not work' to 'will print a traceback for an exception that is ignored'. [done]

4. Please comment on point 4 in msg219055.

5. Possible followup idea: By reviewing and running tests for classes I have not previously paid attention to, I learned something about them. After htest is done, polish the test explanations as appropriate to explain each class to someone not familiar with it. Revise run() to start as follows:

def run(*tests, *, test_list=None):
    if test_list is None: test_list = []

Add htour.py (for instance) with a callable and spec for non-htested classes (and behaviors). It could begin like this
---

from idlelib.idle_test.htest import run

def _format_paragraph(parent): ...(such as in current patch)

_format_paragraph_spec = ...(such as in current patch)

tour_list = [(_format_paragraph, _format_paragraph_spec),]
# automate by scanning globals for callables and grabbing matching specs

run(test_list=tour_list)

---

This should be a separate issue that follows on this one. If you feel like expanding the above to a preliminary patch, go ahead.

6. For me, many of the htest geometry increments are too small. Example:
   root.geometry("+%d+%d"%(x, y + 150) # percolator

[terryjreedy]

Yes, remove unscrollable (test2) tree widget (review response).