You are currently browsing the tag archive for the ‘sequential’ tag.

NPM Module

So that we can reuse the simple serialq code from the previous blog post, I have tidied things up and packaged into an NPM module.Apparently its now installable using “npm install serialq”.

Creating the module was a breeze –

  • within your directory run ‘npm install’ and fill in the questions
  • npm adduser, npm publish
  • test with npm install

After publishing, the module magically shows up on the extremely handy npm.mape.me module search site, under keyword ‘serial’.  See isaacs article ‘How to Module‘ for overview.

Code

The code is a bit simpler to read, as you can see its a very short implementation :

exports.SerialQueue = function()
{
    var sq = 
    {
        funcs : [],
        next : function()
        {
            var Q = this;
            var f = Q.funcs.shift();
            if (f)
                f(function() {Q.next();});    
        },
        add : function(f)
        {
            this.funcs.push(f);
        },
        run : function()
        {
            this.next();
        }
    };
    return sq;
}

Usage :

    var Q = SerialQueue();
    Q.add(fn_first);
    Q.add(fn_second);
    Q.run();</code>

[ For a more readable version of the code snippets above, see this github Gist. Would be nice if wordpress.com supported Gists, Posterous do..ahh maybe time to move my blog. ]

Thoughts

I found this module handy for serializing access to a mysql database. Breaking out this boilerplate made the rest of the code clearer.  Code is up on github

Surprisingly Javascript + Node.js is a real workhorse.  I actually prefer it to Perl/PHP and even Ruby/Python for data plumbing tasks.  You have hashmaps and regex handling built in, garbage collection, and a superb general purpose data format in JSON.  Perhaps Javascript is the hundred-year-language?

Be aware this is ‘cooperative’ sequencing.. each function gets passed a done or next argument, and will have to invoke that to signal completion [ causing the next function to be run ].

Node is Async by Default

The whole crux of Node.js is that everything is done async by default – you fire off something now and get a callback at some later time.  Its a beautiful paradigm and means that you can get great performance, because it fits so closely with the underlying operating system calls [ libevent, completion ports, sockets etc. ]

However, there are times you do need to enforce serial processing… for example checking for valid user/password must return a result before getting sensitive data and displaying it on the web page.

There are now sophisticated serial modules for Node, which Id recommend you look at for real work.  For example Conductor can mark sequential dependencies and will allow the most async processing to happen, while honoring those sequential constraints – the best of both worlds.  Another nice approach is this fork() primitive via stackoverflow.com.

Lets have a look at the simplest case, to see whats under the hood…

Demo Code

I made a test program to compare sync versus async,  This illustrates a very direct approach for serial processing using a queue of work functions. Code on github, here : async_vs_serial.js

Read the rest of this entry »