You are currently browsing the monthly archive for March 2011.

Been using git more and more for public hacking and private consulting work.

Some impressions / notes –

  • is superb!  Radically better/simpler/easier/nicer than sourceforge or google code
  • I lurrve code snippets, aka GISTs hosted on github. Sane blogs support inline gists [ not, yet ]
  • Found a clear readable tutorial on setting up a remote git repo (with git and gitosis) for private or public use
  • Github is so nice, its really tempting to just pay them money for some private repos rather than step thru the above
  • A handy git meta- cheat sheet here

NPM Module

So that we can reuse the simple serialq code from the previous blog post, I have tidied things up and packaged into an NPM module.Apparently its now installable using “npm install serialq”.

Creating the module was a breeze –

  • within your directory run ‘npm install’ and fill in the questions
  • npm adduser, npm publish
  • test with npm install

After publishing, the module magically shows up on the extremely handy module search site, under keyword ‘serial’.  See isaacs article ‘How to Module‘ for overview.


The code is a bit simpler to read, as you can see its a very short implementation :

exports.SerialQueue = function()
    var sq = 
        funcs : [],
        next : function()
            var Q = this;
            var f = Q.funcs.shift();
            if (f)
                f(function() {;});    
        add : function(f)
        run : function()
    return sq;

Usage :

    var Q = SerialQueue();

[ For a more readable version of the code snippets above, see this github Gist. Would be nice if supported Gists, Posterous do..ahh maybe time to move my blog. ]


I found this module handy for serializing access to a mysql database. Breaking out this boilerplate made the rest of the code clearer.  Code is up on github

Surprisingly Javascript + Node.js is a real workhorse.  I actually prefer it to Perl/PHP and even Ruby/Python for data plumbing tasks.  You have hashmaps and regex handling built in, garbage collection, and a superb general purpose data format in JSON.  Perhaps Javascript is the hundred-year-language?

Be aware this is ‘cooperative’ sequencing.. each function gets passed a done or next argument, and will have to invoke that to signal completion [ causing the next function to be run ].

Node is Async by Default

The whole crux of Node.js is that everything is done async by default – you fire off something now and get a callback at some later time.  Its a beautiful paradigm and means that you can get great performance, because it fits so closely with the underlying operating system calls [ libevent, completion ports, sockets etc. ]

However, there are times you do need to enforce serial processing… for example checking for valid user/password must return a result before getting sensitive data and displaying it on the web page.

There are now sophisticated serial modules for Node, which Id recommend you look at for real work.  For example Conductor can mark sequential dependencies and will allow the most async processing to happen, while honoring those sequential constraints – the best of both worlds.  Another nice approach is this fork() primitive via

Lets have a look at the simplest case, to see whats under the hood…

Demo Code

I made a test program to compare sync versus async,  This illustrates a very direct approach for serial processing using a queue of work functions. Code on github, here : async_vs_serial.js

Read the rest of this entry »

Been hacking in Node.js and am really enjoying the saneness of this dev environment.

Some handy links before I forget  :

  • articles – especially Part I, II and III of ‘Learning Javascript with Object Graphs’
  • Joyeur blogs – people working for Joyent on Node / DTrace / Solaris
  • Joyent Node hosting [ built atop – ‘open’ solaris, ZFS, DTrace ]
  • NodeJitsu blog
  • – searchable Node.js Modules list
  • connect-it guide – web framework with chained middleware layers
  • express guide – article on express web framework for Node
  • eventserver – Tom Lee’s internet tee piping for notifications

In other news.. Im hacking over ssh via a long thin pipe to my linode server – using a very erratic mobile broadband connection, arrgh!

Cant wait for ADSL to _finally_ be connected here, so I can watch Bryan Cantrill talk about Cloud Analytics :]

As an aside.. why Javascript?  Consider

  • Javascript is a totally distinct language from Java
  • Javascript deserves its bum rap.. to mis-quote Dame Judi Dench, its bad parts are “arse-clench-ingly” bad :]
  • The good parts of Javascript feel very nice, like a modern lisp inspired language, fairly concise, many valid idioms
  • V8 js engine is fast
  • Javascript callback mechanism fits async event IO really well
  • Node.js embodies the above bullet point into a fine server development environment
  • JSON, the Javascript native data format, is all the good things of XML with none of the bad
  • feels like a unix-like web-plumbing philosophy
  • can keep the same language syntax hat on when writing front end web apps and back end servers