Scratching a JavaScript Testing Itch
I've been using jsunit for unit and some integration testing of my javascript, and it is extremely slick -- but I wanted a few things it doesn't have, so in traditional open source fashion, I started scratching my itch :-)
I've used lots of XUnits, but not written one before, so it was a fun experiment. As this kind of library is very API driven I started with what I though it should look like, it started pretty traditional, something like:
function testSomething() {
assert(1 == 1);
}
function testSomethingElse() {
assertNotNull("hello world");
}
But this really bothered me -- it was filling up the global scope with tests, and required the assertion functions be global as well. I thought about taking a step from JUnit, et. al, and putting them in a class, but... this is JavaScript, it has no class.
Okay, what does it have. It has a nice prototype and open object system. Hmm, that lends itself to builders, maybe something like:
xn.test(function() {
this.testSomething = function() {
this.assert("Math doesn't work", 1==1);
}
this.testSomethingElse = function() {
this.assertNotNull("Strings are null", "hello world");
}
})
That looked interesting, so I wrote some code to make it work. Woot! It wasn't quite right though, so I poked around some more and eventually wound up veering away from the XUnit canonical form and making stuff that looks like this:
dojo.provide("test.test");
dojo.require("xn.test");
xn.test.suite("A Test Suite Named Bob", function(s) {
s.setUp = function(t) {
// it is good to store test-specific state on the test object
// but don't collide with the publicly exposed methods =)
t.setupCalled = true;
}
s.tearDown = function(t) {
t.log("tearDown invoked on %s", t.name);
}
/* actual tests */
s.test("Verify that setup was called", function(t) {
t.assert("Setup was not called", t.setupCalled);
})
s.test("assertNotNull passes for empty object", function(t) {
// t.log("first test invoked");
t.assertNotNull("Empty object is null!", {});
});
s.test("Test which fails on purpose", function(t) {
// t.log("second test invoked");
t.assertNotNull("This failure is on purpose", null);
});
});
Along the way I went ahead and introduced a dependency on Dojo as I use it in almost all my javascript anyway. I also introduced suites (I am dearly tempted to rename them groups instead of suites as they are not really suites). All fun and good, and works well. Had some spiffy output going to report results. Yea!.
Now that I have something that I like the feel of the API for... tim to tackle one of the itches that drove me to write it -- handling arbitrary async calls. The first stab was to have explicit asynchronous test functions, something like this:
s.test.async("Asynch test which succeeds", function(t) {
dojo.io.bind({
url: "/Give-Me-A-404", // 404
error: function(type, thing) {
t.succeed();
}
});
});
s.test.async("Async test which should fail", function(t) {
dojo.io.bind({
url: "/Give-Me-A-404", // 404
error: function(type, thing) {
t.fail("Failure is on purpose");
}
});
});
This, while kind of pretty API-wise, was annoying to implement as the async function wan't actually on the Suite object, so it got ickers. Using Suite#testAsync looked awful (I believe strongly in aesthetic APIs), so, much muckiting about and I wound up with:
s.test("Asynch test which succeeds", function(t) {
t.async();
dojo.io.bind({
url: "/Give-Me-A-404", // 404
error: function(type, thing) {
t.succeed();
}
});
});
s.test("Async test which should fail", function(t) {
t,async();
dojo.io.bind({
url: "/Give-Me-A-404", // 404
error: function(type, thing) {
t.fail("Failure is on purpose");
}
});
});
Where it is the same test() function but adds an async() function to the test object which changes its behavior. The change of behavior is important because idiomatic XUnit says that if a test completes without failing an assertion then it passes. With an async test you need to tell it when the test completes via the succeed() function. Having to call this in normal tests would get old, fast.
In the first real use of this I realized I needed a timeout, no problem, have the async call take a timeout (I made the default pretty short, this is a unit test, after all):
s.test("Async test which fails via a timeout", function(t) {
t.async(100);
});
Woot!
A lot of where the design veers from standard XUnit form it is to accommodate JavaScript idiosyncrasies. For instance, if we look at the test() function we see a variable being passed in! This variable is the instance of the test case, which is also, as it so happens, the value of this in the test function.
JavaScript's scoping bizarrity makes it much more practical to pass the test case in as an argument, that way you don't have to capture the correct value of this, or pass around this as a context to other calls when using anonymous functions. Handy. The same thing applies to suite.
Defining it as a Dojo module is also handy, it makes building a real suite pretty easy, you just require each of the "suites" -- which is why I think I'll rename it to group or even testCase, but I hate the camel casing, so will think a bit :-)
Reporting success and failure is easy to override, the default just creates a definition list and plugs results in, like so. Not pretty, but easy to make pretty as time goes on.
The API exploration test cases are all online, but not especially polished. We'll see where this goes. I kind of like the feel of how it is coming together. A couple things I definitely need to do though are changing failure to raise an exception rather than allow the rest of the test to continue, which will make stack trace generation for tracking down failures more useful; and add some more assertion helper functions (oh, and add the one-argument form, always including an explanation is annoying once you have stack traces to see where it came from).
All in all, a fun scratch!