id	summary	reporter	owner	description	type	status	priority	milestone	component	resolution	keywords	cc
14	Improve test separation	duck_	westi	"Quite a few tests work perfectly well on their own (i.e. via the -t option), however, they fail as soon as they're run as part of the whole test suite. This is because of poor separation of test environments between successive tests, e.g. not removing inserted posts or relying on a user being in the DB which gets removed in a test run earlier as part of the whole suite.

----

Example:

Was looking at fixing the failures in WPTestUserCapabilities, specifically test_page_meta_caps. This was easy enough on its own, _insert_quick_posts doesn't actually use the $type argument to set the post_type, but still other failures popped up when run with the rest of the tests. This turned out out to be because TestWPQueryVars and TestWPQueryShortPageRules, which run before WPTestUserCapabilities, extend _WPDataSet1 which destroys all data, apart from options, including the default user that _WPEmptyBlog assumes exists and sets to `$this->author` which is subsequently used in `_insert_quick_posts`.

Fix I came up with in this case was to create a new user exclusively for the failing test and use that ID to create a new post, see WPTestUserCapabilities.diff (this is only a partial patch of what I have for that whole file). New users will be cleaned up in tearDown, though currently I don't think the new post will be.

----

One thought for this is to ensure that _WPEmptyBlog and other sample sites delete all residual data at setUp, forcing tests to create new users/posts/etc. as required instead of relying on anything being present already. This would need to be done all together so that tests extending _WPEmptyBlog aren't broken because of the change"	task	closed	normal		Test Framework	fixed