Automatic Testing for TDD with PHP

Traditional test-driven development can, at times, be cumbersome. You have to stop writing code in order to run your tests. Luckily, there are solutions, which provide the ability to automatically run your tests as you code. In this tutorial, you will learn how to use a Ruby gem, called watchr, to monitor your code and automatically run the appropriate tests whenever you save your work.

Step 1: Software Requirements

Any tool that helps you obtain quicker feedback is a valuable asset.

This tutorial uses PHP for the code example, however, the techniques are applicable for any language, which offers a CLI utility for unit testing. Ruby is required because we will use the watchr gem. So, make sure you have a working installation of Ruby and PHP with PHPUnit.

Next, ensure that you have libnotify installed, if you’re on Linux; Windows and Mac OSX users need “Growl.” This tutorial is directly applicable on Linux, but I will suggest alternative commands and settings where possible.

Now, it’s time to install the watchr gem. Open a console, and make sure you are in the folder where you can directly run gem. Type the following command:

gem install watchr

Step 2: Technical Background

When a file or folder is modified, watchr can trigger a callback function.

The watchr gem is an executable program written in Ruby, and it wraps around features found in an operating system’s file system to provide the ability to watch for changes made to a specific file or folder. Naturally, these file system features differ for each operating system and file system.

watchr provides a unified application programming interface (API) for all operating systems. On Linux, it uses inotify, the kernel’s file system event library; on other operating systems, it uses the appropriate alternative. If, for some reason, the operating system does not have an available event service, watchr periodically polls the watched file or folder.

When a file or folder is modified, watchr can trigger a callback function. We will use this function to run our tests.

Step 3: Create a PHP Project

Our project is rather simple. Replicate the simple directory structure shown in the following image:

In the Nettuts.php file, add the following code:

<?php

class Nettuts {

}

?>

Next, add the following code to NettutsTest.php:

<?php

require_once dirname(__FILE__) . '/../Classes/Nettuts.php';

class NettutsTest extends PHPUnit_Framework_TestCase {

  protected $object;

  protected function setUp() {
    $this->object = new Nettuts;
  }

  protected function tearDown() {

  }
}

?>

At this point, the test file is simply a skeleton, and, as you can see in the image above, the tests pass.

Step 4: Create the First watchr Script

Now, we need to create a Ruby file in our project’s folder; let’s call it autotest_watchr.rb. Next, add the following code to the file:

watch("Classes/(.*).php") do |match|
  run_test %{Tests/#{match[1]}Test.php}
end

Automated tests are IDE independent – a big plus in my book.

This code uses the watch method to watch all the .php files in our project’s Classes folder. When a .php file changes, the operating system issues an event, and our watch method will be triggered. The name of the .php file is returned (minus the extension) in a match array’s position of 1. As with any regular expression, parentheses are used to specify a match variable, and in this code, we use them in the matching condition to fetch the file name. Then, we call the run_test method with the path of the composed test file name.

We should also watch our test files; so, add the following code to the Ruby file:

watch("Tests/.*Test.php") do |match|
  run_test match[0]
end

Note that the match array contains the full file name at position 0, and we pass it directly to the run_test method.

Step 5: Make the Script Run the Tests

The Ruby script is set up to watch our .php files, and now we need to implement the run_test method. In our case, we want to run PHPUnit for the specific file.

def run_test(file)
  unless File.exist?(file)
    puts "#{file} does not exist"
    return
  end

  puts "Running #{file}"
  result = `phpunit #{file}`
  puts result
end

We first ensure that the file exists, and simply return if it doesn’t. Next, we run the test with PHPUnit and send the result to the console. Let’s run our watchr script. Open your console, navigate to your project’s directory, and then run:

watchr ./autotest_watchr.rb

Windows users should omit “./” from the above command.

Now modify one of the .php files (just add an empty line at the end of the file), save it, and observe the output in the console. You should see something similar to what’s shown below:

Running Tests/NettutsTest.php
PHPUnit 3.6.0 by Sebastian Bergmann.

F

Time: 0 seconds, Memory: 3.75Mb

There was 1 failure:

1) Warning
No tests found in class "NettutsTest".

/usr/bin/phpunit:46

FAILURES!
Tests: 1, Assertions: 0, Failures: 1.

Yep, we don’t yet have a test to run; so let’s put in a dummy test. Add the following code to the test PHP file:

function testDummyPassingTest() {
  $this->assertTrue(true);
}

Run the Ruby script again, and you should see:

Running Tests/NettutsTest.php
PHPUnit 3.6.0 by Sebastian Bergmann.

.

Time: 0 seconds, Memory: 3.75Mb

OK (1 test, 1 assertion)

Step 6: Parse the Test Output

Let’s notify the user, via the system’s notification mechanismm about the test results. We’ll modify the run_tests method to trigger a method, called notify. Below is the modified run_tests:

def run_tests(file)
  unless File.exist?(file)
    puts "#{file} does not exist"
    return
  end

  puts "Running #{file}"
  result = `phpunit #{file}`
  puts result

  if result.match(/OK/)
    notify "#{file}", "Tests Passed Successfuly", "success.png", 2000
  end
end

The name of the image file, success.png, points to the image you want to display in the notification area. This image is not provided in this tutorial; so you will need to find your own. Now, let’s write the notify method:

def notify title, msg, img, show_time
  images_dir='~/.autotest/images'
  system "notify-send '#{title}' '#{msg}' -i #{images_dir}/#{img} -t #{show_time}"
end

Mac OSX and Windows users: replace the notify-send command with the appropriate Growl alternative. Modify something in either your test or code file so that the test still passes. Save the modified PHP file, and watch the magic happen. Below is an image of the result on my system:

Next, we need to catch the failures. The following code adds a couple of lines to run_tests:

def run_tests(file)
  unless File.exist?(file)
    puts "#{file} does not exist"
    return
  end

  puts "Running #{file}"
  result = `phpunit #{file}`
  puts result

  if result.match(/OK/)
    notify "#{file}", "Tests Passed Successfuly", "success.png", 2000
  elsif result.match(/FAILURES\!/)
    notify_failed file, result
  end
end

Also, let’s add the notify_failed method to the file:

def notify_failed cmd, result
  failed_examples = result.scan(/failure:\n\n(.*)\n/)
  notify "#{cmd}", failed_examples[0], "failure.png", 6000
end

Modify either of your PHP files to make the test fail; save the modified file. Observe the notification message. It contains the name of the first failing test. This name is selected by the regular expression in the method notify_failed, which parses the PHPUnit output.

Step 7: Clear the Console Before Each Test Run

Add the following method to your Ruby script, and be sure to call it in the run_test method. The code should work in Linux and Mac OSX, though you might need to do some research for Windows.

def clear_console
  puts "\e[H\e[2J"  #clear console
end

Conclusion

Whenever you program using TDD, any tool that helps you obtain quicker feedback is a valuable asset. My coworkers use similar scripts with watchr or alternatives (some are written around fs_event on MacOS). Needless to say, we’re spoiled now, and can’t imagine developing anything without automatically running tests.

Automated tests are IDE independent – a big plus in my book. Too many IDEs force you to use a specific testing framework, and don’t get me started on remote testing. I prefer to use scripts like this daily, and surely recommend them to any agile software developer.