Brian's Waste of Time

DataFlow Style Sequencing for Concurrent Ajax Calls

There is a somewhat misleading notion that you don't need to worry about concurrency in JavaScript because JavaScript itself is strictly single threaded. When you have concurrent ajax requests you do need to deal with concurrency. You can (for RFC compliant browsers) have two sockets open to the server at a time. Making use of them for concurrent requests, or having some operation dependent on UI input and a server response forces you to deal with not knowing what will happen when.

The most useful mechanism I have found is based on dataflow concurrency. It is far from a full dataflow machine, but it basically takes the part where functions can (and will) execute when all their arguments are available, but not until then. Tim and I recently extracted the techniques we had been using into a nice general purpose tool, the DataFlowSequence. Setting it up works like this (note I am using a different implementation then what we extracted, I did it to explore a different algorithm):

    var dfs = new skife.ajax.DataFlowSequence();

    dfs.when("firstName", function(args) {
      alert(args.firstName);
    });

    dfs.when("lastName", function(args) {
      alert(args.lastName);
    });

    /**
     * Note that we explicitly return the full name
     */
    var fullname_key = dfs.when("firstName", "lastName", function(args) {
      var full_name = args.firstName + " " + args.lastName;
      alert(full_name);
      return full_name;
    });

    /**
     * Note that this depends on the return value of the previous
     * function.
     */
    dfs.when("salutation", fullname_key, function(args) {
      alert(args.salutation + " " + args[fullname_key]);
    });

All that does is create a managed set of functions which need some data to execute. The common use case is where these are things which need to be fetched via ajax, and might come back in any order. The dfs.when("name", "anotherName", function(args) { ... }) calls say that the function passed in depends on name and anotherName. When these variables have been bound, this function will be executed. We found it was more convenient to bind them onto a single args object than to match them positionally, but you could pass them in positionally instead of binding them by name. Your choice.

In a number of cases we need the output of one function to be input to another so we have the DataFlowSequence return a key which can be used by other functions to depend on output from each other. It's worked well.

Now, to supply some things to these so they do some actual work...

dojo.io.bind({
  url: "/xn/rest/1.0/first-name",
  mimetype: "text/json",
  load: function(type, name, evt) {
    dfs.supply("firstName", name);
  }
});

dojo.io.bind({
  url: "/xn/rest/1.0/salutations",
  mimetype: "text/json",
  load: function(type, greets, evt) {
    dfs.supply("salutation", greets);
  }
});

dojo.io.bind({
  url: "/xn/rest/1.0/last-name",
  mimetype: "text/json",
  load: function(type, name, evt) {
    dfs.supply("lastName", name);
  }
});

We don't know what order these will come back in, but regardless, the functions dependent on a particular piece of data won't be called until that data is in fact available, and ones not dependent on it can be called as soon as what they need is there =)

To do a better demo, I wired the above with a some visual gewgaws and did a mechanical blogreader style web service (faked them out) with forms you can submit to have the service "respond" right here.

Given Name

Family Name

Full Name

Greeting

Source and unit tests are available of course =)

1 writebacks [/src/javascript] permanent link