Maarten’s post at Tillate finally brought the motivation to document the PHP testing approach we use at local.ch.
First let me give you a short introduction to our architecture at local.ch. We have a clear separation of frontend (presentation, user-visible parts) and backend (search logic and database accesses). The frontend is written in PHP and XSLT. The PHP-part basically only orchestrates queries to our Java-based backend and passes the XML responses to XSLT. The bigger parts of the system are the XSLT stylesheet. All this means, that traditional unit tests don’t have a big value for the frontend as there isn’t much traditional logic. But we need to do functional/integration testing.
Only since a short time we actually have a nice PHP-based testing infrastructure. Before that, we almost exclusively used Selenium Core - see for example my presentation of last year. Now we use SimpleTest slightly extended and with a helper class for the Selenium testing (to be documented in a separate blog post).
This is the basic test.php file which we use to execute the tests:
<code><?php
require_once("common.php");
// "Configuration"
$GLOBALS['TLD'] = 'local.ch';
$GLOBALS['SELENIUM_SERVER'] = 'localhost';
if (file_exists('config_developer.php')) {
include_once('config_developer.php');
}
if (getenv('SELENIUM_SERVER')) {
$GLOBALS['SELENIUM_SERVER'] = getenv('SELENIUM_SERVER');
}
if (getenv('TLD')) {
$GLOBALS['TLD'] = getenv('TLD');
}
/**
* $case: Only run this test case
* $test: Only run this test within the case
*/
function runAllTests($onlyCase = false, $onlyTest = false) {
$test = &new TestSuite('All tests');
$dirs = array("unit", "selenium", "selenium/*");
foreach ($dirs as $dir) {
foreach (glob($dir . '/*.php') as $file) {
$test->addTestFile($file);
}
}
if (!empty($onlyCase))
$result = $test->run(new SelectiveReporter(new TextReporter(), $onlyCase, $onlyTest));
else
$result = $test->run(new XMLReporter());
return ($result ? 0 : 1);
}
return runAllTests(@$argv[1], @$argv[2]);
?></code>
The top part sets up some configuration values we use for Selenium. There are two global variables, the TLD which defines the host name to test against and SELENIUM_SERVER which is the Selenium server to connect to. There are two ways to configure. Either with the “config-developer.php” file which is excluded from version control and can be created by the developer. And then by setting environment variables when calling the test script.
After that the tests are run. Basically it includes tests from a set of directories. Then it either uses the SelectiveReporter or our own XMLReporter to execute tests. The SelectiveReporter will only execute a given test class or even only a given method (the first and second parameter from the command line respectively). The XMLReport gives a JUnit-style parseable output that we use for the continuous integration tool (Bamboo in our case).
The included common.php file contains this:
<code><?php
error_reporting(E_ALL);
ini_set('log_errors', '0');
if (! defined('SIMPLE_TEST')) {
define('SIMPLE_TEST', BX_PROJECT_DIR . '/inc/vendor/simpletest/');
}
require_once(SIMPLE_TEST . 'reporter.php');
require_once(SIMPLE_TEST . 'unit_tester.php');
class XMLReporter extends SimpleReporter {
function XMLReporter() {
$this->SimpleReporter();
$this->doc = new DOMDocument();
$this->doc->loadXML('<testsuite/>');
$this->root = $this->doc->documentElement;
}
function paintHeader($test_name) {
$this->testsStart = microtime(true);
$this->root->setAttribute('name', $test_name);
$this->root->setAttribute('timestamp', date('c'));
$this->root->setAttribute('hostname', 'localhost');
echo "<?xml version=\"1.0\"?>\n";
echo "<!-- starting test $test_name\n";
echo "Executing on *." . $GLOBALS['TLD'] . " with selenium server " . $GLOBALS['SELENIUM_SERVER'] . "\n";
}
/**
* Paints the end of the test with a summary of
* the passes and failures.
* @param string $test_name Name class of test.
* @access public
*/
function paintFooter($test_name) {
echo "-->\n";
$duration = microtime(true) - $this->testsStart;
$this->root->setAttribute('tests', $this->getPassCount() + $this->getFailCount() + $this->getExceptionCount());
$this->root->setAttribute('failures', $this->getFailCount());
$this->root->setAttribute('errors', $this->getExceptionCount());
$this->root->setAttribute('time', $duration);
$this->doc->formatOutput = true;
$xml = $this->doc->saveXML();
// Cut out XML declaration
echo preg_replace('/<\?[^>]*\?>/', "", $xml);
echo "\n";
}
function paintCaseStart($case) {
echo "- case start $case\n";
$this->currentCaseName = $case;
}
function paintCaseEnd($case) {
// No output here
}
function paintMethodStart($test) {
echo " - test start: $test\n";
$this->methodStart = microtime(true);
$this->currCase = $this->doc->createElement('testcase');
}
function paintMethodEnd($test) {
$duration = microtime(true) - $this->methodStart;
$this->currCase->setAttribute('name', $test);
$this->currCase->setAttribute('classname', $this->currentCaseName);
$this->currCase->setAttribute('time', $duration);
$this->root->appendChild($this->currCase);
}
function paintFail($message) {
parent::paintFail($message);
if (!$this->currCase) {
error_log("!! currCase was not set.");
return;
}
error_log("Failure: " . $message);
$ch = $this->doc->createElement('failure');
$breadcrumb = $this->getTestList();
$ch->setAttribute('message', $breadcrumb[count($breadcrumb)-1]);
$ch->setAttribute('type', $breadcrumb[count($breadcrumb)-1]);
$message = implode(' -> ', $breadcrumb) . "\n\n\n" . $message;
$content = $this->doc->createTextNode($message);
$ch->appendChild($content);
$this->currCase->appendChild($ch);
}
}
?></code>
This file sets up SimpleTest by including the necessary file. Then follows the definition of the XMLReporter. It will print out some debugging so we know where it’s at. That’s necessary for us because our Selenium tests take about 15 to 20 minutes. At the end follows the XML-result which can be parsed by Bamboo. It should also work for other tools that expect JUnit XML output but I haven’t tested that.