Marking test as skipped will prevent execution of all subsequent tests

[trebi]

• bug report? yes
• feature request? no
• version: 2.0.0

Description

If test is marked as skipped, all subsequent tests will not execute.

Steps To Reproduce

1. Create file

<?php declare(strict_types=1);

namespace Tester\BugReport;

use Tester\Assert;

require __DIR__ . '/../bootstrap.php';

/** @TestCase */
final class BugReportTest extends \Tester\TestCase
{
	public function testSkip(): void
	{
		\Tester\Environment::skip();
	}


	public function testFail(): void
	{
		Assert::true(false);
	}
}

(new BugReportTest)->run();

2. Run the file, the output will be:

Skipped:


Process finished with exit code 177

giving no indication that second test is failing. Such behaviour causes that I prefer commenting out the test rather than makring it as skipped.

BTW if you swap the order of the tests, it will execute the failing test and fail as expected, which is inconsistent behaviour. Order of methods should never matter.

[milo]

If you run this test by Tester, you'll get two results: sF That's correct. One skipped, one failed. Every test method runs in own separated process.

But if you run it as php test.phpt, the @TestCase is ignored (as mentioned in doc), and all test methods run one by one in one process, so you get only one result.

You can try:

php test.phpt --method testSkip
php test.phpt --method testFail

[trebi]

Thanks for reply, but I think it would be handy that even in "serial mode" it would run all the tests until the first real failure (skip is not really a failure), or you will be sure that all tests have passed.

[milo]

I hope solved by #380