WordPress function gives different results outside vs inside PHPUnit test

I’ve been working very hard to get PHPUnit to play nice with WordPress (with the help of this, this, this, this, and this), and I think I’ve gotten pretty close. The code I run directly before my tests looks like this (basically lifted verbatim from this article):

( PHP_SAPI === 'cli' ) || die( 'Access Denied' );

define( 'PHPUNIT_DB_PREFIX', 'phpunit_' );

global $wp_rewrite, $wpdb;

define( 'WP_MEMORY_LIMIT', '100M' );

require_once( dirname( __FILE__ ) . '/../../../../wp-load.php' );
require_once( ABSPATH . 'wp-admin/includes/admin.php' );

wp_set_current_user( 1 );

Seems to work fine, except that functions return different things based on where they are in the file. For instance, outside of a test class, shortcode_exists returns true, whereas within a test case the exact same shortcode_exists statement returns false. What am I doing wrong?

Here are the contents of the entire test file:

[The file has changed since I originally posted this…]

EDIT: Ok, so I’ve followed J.D.’s tutorial as closely as I could. It feels really close. I’m now getting this error:

PHPUnit_Framework_Error_Warning : copy(data/not-gettexted-0.php): failed to open stream: No such file or directory
#0 [internal function]: PHPUnit_Util_ErrorHandler::handleError(2, 'copy(data/not-g...', '/Applications/w...', 41, Array)
#1 /Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/wordpress-dev/trunk/tools/i18n/t/NotGettextedTest.php(41): copy('data/not-gettex...', 'data/not-gettex...')
#2 [internal function]: NotGettextedTest->test_replace()
#3 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestCase.php(988): ReflectionMethod->invokeArgs(Object(NotGettextedTest), Array)
#4 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestCase.php(838): PHPUnit_Framework_TestCase->runTest()
#5 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestResult.php(648): PHPUnit_Framework_TestCase->runBare()
#6 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestCase.php(783): PHPUnit_Framework_TestResult->run(Object(NotGettextedTest))
#7 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestSuite.php(779): PHPUnit_Framework_TestCase->run(Object(PHPUnit_Framework_TestResult))
#8 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestSuite.php(749): PHPUnit_Framework_TestSuite->runTest(Object(NotGettextedTest), Object(PHPUnit_Framework_TestResult))
#9 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/Framework/TestSuite.php(709): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult), false, Array, Array, false)
#10 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/TextUI/TestRunner.php(350): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult), false, Array, Array, false)
#11 phar:///Applications/wordpress-3.8.3-0/apps/wordpress/htdocs/wp-content/plugins/hf-accountability/tests/phpunit-lts.phar/phpunit/TextUI/Command.php(176): PHPUnit_TextUI_TestRunner->doRun(Object(PHPUnit_Framework_TestSuite), Array)
#12 /private/var/folders/r5/0w5bp7b13pndfymz_pgf_zz00000gn/T/ide-phpunit.php(268): PHPUnit_TextUI_Command->run(Array, true)
#13 /private/var/folders/r5/0w5bp7b13pndfymz_pgf_zz00000gn/T/ide-phpunit.php(506): IDE_Base_PHPUnit_TextUI_Command::main()
#14 {main}

Any idea how I can fix that? In the mean time, I’m going to keep checking all the file paths I defined in the several config files.

EDIT EDIT: That last error was resolved by modifying the PhpStorm configuration I was using. You have to both set it to “Use alternative configuration file” and set the test scope to “Defined in the configuration file.”

The reason this is happening is actually not because of anything in the code you’ve posted, per se, but because of how you have PHPUnit configured. The shortcode callbacks are stored in a global ($shortcode_tags). When you load WP, the shortcode is registered and added to the global. You call shortcode_exists(), it checks $shortcode_tags, and returns true. Then PHPUnit starts running your tests. And it clears out the global variable scope. So, when your test is run, $shortcode_tags no longer exists, so shortcode_exists() reports that your shortcode isn’t registered. (PHPUnit will then restore the global scope to what it was before the test, so that tests don’t affect the global variables. Running shortcode_exists() after the tests are finished would report true.)

PHPUnit does have the backupGlobals setting, and setting it to false will prevent PHPUnit from touching the global scope. And that should fix your issues. However….

A more excellent way

As the author of this tutorial, I would like to recommend that you set up your tests more like I show there. The reason being that you won’t have to write all this bootstrap to get PHPUnit to play with WordPress. You should instead just use WordPress’s PHPUnit bootstrap, which, obviously, is already engineered to play nice with WordPress. 🙂