wp-test.php does not exit with proper error code when tests fail

[benbalter]

If running wp-test.php as part of an automated test (e.g., using Travis CI), even when a test fails, the script simply ends (what's interpreted as a passing return code 0).

Instead, it should exit(1) on failure, and even better explicitly exit(0) on success so that the proper exit code is passed to other programs.

( The attached patch also removed the closing PHP tag )

Only exit with error code when -x flag is present

Per nacin, updated ticket with 2nd patch attachment:37-1.diff​ that only exits with the proper error code when a -x flag is passed.

Error code now returns the combined number of errors/warnings each time (0 = success, >0 = failure).

I don't think this is going to behave like you want:

exit ( intval( $result->failureCount() ) + intval( $result->errorCount() ) ); 

Exit codes are an unsigned byte. They're limited to a range of 0-255. Anything outside that range would just roll over.

For example, if the combined error and failure count was 355, the above code would set the exit code to 355, but that's outside the range, so any process that checked the exit code would see 99.

So perhaps it should just exit 0 or 1 for success/fail?

Replying to benbalter:

So perhaps it should just exit 0 or 1 for success/fail?

Yes.

This will work fine when using the TextUI runner.