bpo-19382: Adding test cases for module tabnanny.

[ultimatecoder]

• Description: Unit tests for standard module tabnanny. Tests are added for almost all the functionality except tabnanny.Whitespace class.

• Reason leaving Whitespace:? I found the module contains mathematical calculations. I will try to understand it first and then write tests for it. I think it will be good to do that in another branch.

• Testing strategy: Whitebox

• BPOs

  • https://bugs.python.org/issue19382
  • https://bugs.python.org/issue19102

https://bugs.python.org/issue19382

[DimitrisJim]

Hi @ultimatecoder, you'll also need to remove tabnanny from the untested tuple in test_sundry. This is one of the failures AppVeyor (and soon Travis, I'd guess), reports.

[vadmium]

Just write “captures exceptions” (or change arised → raised).

What is “tabsize”? Did you mean “tabnanny.check”?

[vadmium]

feeded → fed
dipicting → depicting

[vadmium]

feeded → fed
Expacted → Expected

[vadmium]

verifying → verify the
existance → existence

[vadmium]

few → a few [otherwise, the implication is that most of the files have errors: few are error-free]

[vadmium]

giving

[vadmium]

quiet mode?
“less” looks wrong, unless you meant “Should display less”

[vadmium]

Don’t you get CRLF on Windows?

[vadmium]

In Python 3, b"%r" is equivalent to calling the “ascii” function. I expect the test will fail if the file path has non-ASCII characters. It may be better to only test that an error is being reported, but tolerate variations in the message.

[vadmium]

What if the path is non-ASCII? Again, just check the general form of the output and don’t be too specific.

[ultimatecoder]

I don't think this https://github.com/python/cpython/blob/master/Lib/tempfile.py#L144 generates anything Non-ASCII.

[vadmium]

No, but the OS can provide a parent directory for temporary files that has non-ASCII path components. It seems relatively common on Windows that the temporary directory is within a user’s profile, and the profile directory can be a localized non-ASCII name. Top Google result: https://bugzilla.mozilla.org/show_bug.cgi?id=260034.

[ultimatecoder]

Many thanks for affording time for reviewing this. I will agree with you on this point. I have updated code. Will you please review again?

[vadmium]

Check out the existing patches and reviews you linked. You may be repeating some of the same problems, e.g. CRLF on Windows.

[ultimatecoder]

@vadmium Many thanks for your comments. I am not native English speaker. I have fixed the mistakes. I have added comments to some comments. Please review my fixes. Thanks!

@DimitrisJim Thanks for your comment. I have removed tabnanny from untested modules list. Thanks!

[vadmium]

Sorry but I don’t have an easy way to see your fixes relative to the old version I reviewed. I’m not really set up to properly review and push stuff with the new Git Hub setup, so I will leave this to someone else, or for another time in the future.

[vstinner]

remove one empty line

[ultimatecoder]

Removed this. This is Github glitch.

[vstinner]

Hum, I prefer to write mock.patch, please remove this import

[vstinner]

Please import tempfile and write tempfile.NamedTemporaryFile.

[vstinner]

IMHO this docstring is useless. Usually, we don't comment much tests, they are not intented to end users.

[vstinner]

If you only care of creating a temporary filename, use tempfile.mktemp() and then open() manually the file to create it. You can use open(tmpname, "x") to prevent race conditions.

[vstinner]

Instead of a dict, you can simply use a tuple (args, expected)

[ultimatecoder]

Don't think we should kill the readability by converting it to tuple. How about collections.namedtuple?

[ultimatecoder]

Or How about this way?

        tests = [
         #  (Function arguments , Expected output)
            (['first', 'second'], 'first second\n'),
            (['first']          , 'first\n'),
            ([1, 2, 3]          , '1 2 3\n'),
            ([]                 , '\n')
        ]

[merwok]

Keep tests simple:

for args, expected in [
    ([1], '1'),
  ]:

[brettcannon]

Yes, keep them simple. And don't use namedtuples except for keeping backwards-compatibility with an interface that used to take an indexed object and now takes an object with attributes.

[vstinner]

Not sure that this docstring is super useful.

[vstinner]

I don't think that it's worth it to declare one test case to test a single class/function. Try to put most or all tests into a single test case called "TabnannyTestCase".

TestCheck deserves its own test case since it has a setUp() method.

[vstinner]

Please move this list into the method directly.

[vstinner]

Unit tests must have zero side effect: you must save the old verbose value and restore it. Example:

self.addCleanup(setattr, tabnanny, 'verbose', tabnanny.verbose)
tabnanny.verbose = 0

[ultimatecoder]

Aha! a geek way 👍

[vstinner]

No need to make this method private, remove "_" prefix.

[vstinner]

Again, I'm not sure that a long comment is needed for a simple unit test.

[ultimatecoder]

I think it is explaining why the method is here. Suggest a way for improving the docstring.

[brettcannon]

At least ditch the whole Arguments section; we don't use that style in the stdlib.

[vstinner]

Who removes the temporary file? I suggest to put create_tmp_python_file() in the test case (or in a base class if it's needed by multiple test cases) and use self.addCleanup(support.unlink, tmpfile) to make sure that the temporary file is removed.

[ultimatecoder]

Adding it to a method will be a poor idea. If I will add it to base class then we will have a base class with one method. Will you accept that?

[brettcannon]

Yes, if the method needs to access information that will be on the instance then a class with a single method is fine. But if you aren't reusing it then just inline the function.

[vstinner]

I like the overall change, but I have a long list of requests :-)

[vstinner]

IMHO the docstring is useless.

[vstinner]

Move this assert out of the with block.

[ultimatecoder]

We can't move this out.

[brettcannon]

Why not? Is tokenize.generate_tokens() lazy (which isn't documented)?

[ultimatecoder]

@brettcannon The tokenize.generate_tokens() is a lazy function (generator). Can you help to identify a way which allows putting assertRaises outside the with? Thanks!

[vstinner]

Use universal_newlines=True to normalize newlines. I also prefer to use text rather than bytes for stdout/stderr.

[ultimatecoder]

I don't think these chain of methods, assert_python_ok -> _assert_python -> run_python_until_end is allowing to pass universal_newlines keyword argument!

Reference: https://github.com/python/cpython/blob/master/Lib/test/support/script_helper.py#L94

Are you suggesting to replace script_helper.assert_python_ok() with subprocess.Popen() here?

[vstinner]

assert_python_ok() already tests that returncode==0.

[vstinner]

make this method public

[vstinner]

For better readability, you can use textwrap.dedent() and a multiline """...""" indented string. See test_faulthandler for examples.

[ultimatecoder]

This is a nice suggestion 👍 Are you expecting these multiline strings to convert into a regular expression? I think this way just looks fine.

[JelleZijlstra]

witnesses, not whitenesses

[brettcannon]

To try and help move older pull requests forward, we are going through and backfilling 'awaiting' labels on pull requests that are lacking the label. Based on the current reviews, the best we can tell in an automated fashion is that a core developer requested changes to be made to this pull request.

If/when the requested changes have been made, please leave a comment that says, I have made the requested changes; please review again. That will trigger a bot to flag this pull request as ready for a follow-up review.

[ultimatecoder]

@brettcannon Many thanks for your comment. I had tried to comment back at confusing points. I think @vstinner lost his interest in my solution. What efforts should I do to make it mergeable? Have a great day!

[brettcannon]

@ultimatecoder if you think you have addressed all of the comments left by @vstinner then just leave a comment that says I have made the requested changes; please review again.

[ultimatecoder]

@brettcannon I have done the requested changes. There were few which I would like to discuss with @vstinner according to his availability. He has pointed out few changes on which I am confused about. I have tried to write back as a part of reply comment in response to improvement comment. I hope I made a clear statement about the situation. Thanks!

[ultimatecoder]

@brettcannon Many thanks for your comments. I am under little work pressure. Please expect answers/improvements at the end of this week. Thanks!

[ultimatecoder]

@brettcannon Reminding you for this PR. All the best for sprints 👍

[brettcannon]

@ultimatecoder no need as this is the top of the list for when I actually get time to start reviewing PRs 😉

[brettcannon]

Nothing fundamentally wrong, just some things to help make the tests easier to manage in future years.

[brettcannon]

I would use subtests here: https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests

[vstinner]

Done

[brettcannon]

Use f-strings everywhere. 😄

[vstinner]

Done

[brettcannon]

To make this easier to read, I would create the context managers outside the with statement:

file1 = TemporaryPyFile(...)
file2 = TemporaryPyFile(...)
with file1 as file1_path, file2 as file2_path:
    ...

[vstinner]

For this specific case, IMHO "file1 = ..." is better.

[brettcannon]

Move the TemporaryPyFile instantiation to its own line so you don't have to spread over multiple lines. (Same of the other lines where you were forced to spread a with statement over multiple lines.)

[vstinner]

I looked at the new code and it's fine.

[brettcannon]

Pull this up to the next line up (it's okay for it to spill over 80 columns).

[brettcannon]

Don't worry too much about the exact output being formatted as it is. For instance, if someone tweaked the grammar of the sentence the test should still pass. Do still check for semantically important info, though, like that the offending line is in the output.

[vstinner]

Maybe the test is too strict, but IMHO we have been too pendantic on this nice PR which has been proposed 3 months ago. tabnanny has currently no test. If the test fails tomorrow, we will just fix the test. No need to write the perfect test here.

[brettcannon]

You don't want to tie yourself to exception messages as they do change between versions. Just care about the right exception being raised.

[ultimatecoder]

I agree, but the code does not raise an exception reference: https://github.com/python/cpython/blob/master/Lib/tabnanny.py#L99
What way will be better to assert the working of mentioned function?

[brettcannon]

Sub tests would be great here.

[vstinner]

Done

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[ultimatecoder]

@brettcannon I am a bit busy with present work. I am sure, I will not be able to complete during this sprints. But I will be able to improve according to your comments in few days. Thanks for reviewing :)

[brettcannon]

@ultimatecoder no rush! I'm not expecting to take quite so long to do future reviews as I only need to review relative future changes.

[vstinner]

Please remove Lib/test/.test_tabnanny.py.swo from your PR.

[vstinner]

Oh, I am sorry, I forgot about your old PR :-( Here is a new review. Your PR is very close to be ready to be merged. Sorry again about the slow review :-(

[vstinner]

Sorry but this style is not compliant with PEP 8, please strip spaces before ",".

[vstinner]

You should always remove the created file.

[vstinner]

You may use:

proc = script_helper.spawn_python('-m', 'tabnanny', *args, text=True)
out, err = proc.communicate()
self.assertEqual(out, stdout)
self.assertEqual(err, stderr)
self.assertEqual(proc.returncode, 0)

text=True normalizes newlines to \n (Unix EOL).

[ultimatecoder]

@vstinner Thanks for re-observing this. I am working on the improvements suggested by you and @brettcannon 😃

[ultimatecoder]

@brettcannon I have updated the changes according to your comments. There is one comment in which you mentioned to assert the exception and not the output. I have added a reply to that comment with a reference. I request to provide your input on that. Thanks.

[ultimatecoder]

@vstinner I have improved the code according to your suggestion. I request to re-review.

[vstinner]

I'm not sure that returning True is correct here. I just suggest to remove the "return" line.

[ultimatecoder]

I agree, returning True isn't making sense here. Thanks for commenting.

[ultimatecoder]

@vstinner @brettcannon I have updated the code according to your comments. I request to proceed for further review. Thanks.

I checked all latest Brett's comments: all fixed

[ultimatecoder]

@vstinner Thanks for taking time and reviewing this PR.

@vstinner Thank you for reviewing your PR and inspiring for continuing.

I have learned many this at the time of creating, maintaining this Patch.