You are currently browsing the category archive for the ‘NoSQL’ category.

gocanvas.com is great : you can create form apps for your business without programming.

Replacing paper with mobile touchscreen and having your data in the cloud is such a big win… BUT there are a couple of crucial missing pieces that are vital to business  :

  • more than one kind of thing !
  • relationships between things

 

The Problem

In real life and in business, there are a few different kinds of things, such as :

  • Customer
  • Address
  • Vehicle
  • Branch
  • Product
  • Invoice
  • Event
  • Group
  • Project
  • Property
  • Contract
  • Supplier
  • Shipment
  • Task

And these things have important relationships between them : A Customer may have several site Addresses.  A Job may have several different kinds of Checklist attached.  An Invoice will have several Products for each Job for a Customer .. and so it goes.

 

Relations

A good app [ mobile or otherwise ] will have a data-model that reflects the REALITY of your business, and it will be flexible enough to handle the relationships between the data items.

diagram_consult

This is the old-school database stuff they figured out in 1940, which has been the staple of business IT for the past 60 years – they call them ‘Relational’ databases for a reason.

These days we want more flexible data, there is the whole NoSQL movement, key-value stores, XML and JSON.  To model reality and business we need natural tree and graph structures, not everything is a table.  Just dont throw away the baby with the bathwater : we need relations.

If you don’t have Relations you cant do the fun stuff :

  • one to many [ Comment Wall can have many Posts ]
  • trees and heirarchies [ Deep Product Catalog, Organization chart ]
  • graphs and social networks [ facebook, linkedin, crm, sales ]
  • people interacting [ messaging ]

This lack of relationships is killer for IT departments – they cant run a retail business or a supply chain or a sales organisation or a manufacturing plant without this flexibility.  Its also killer for startups exploring new business models who need an app to manage their operations or customer needs accurately and in a flexible way.

 

The Solution : Relations

The solution is to have a tool that is great for making forms… but is also great for handling relations between different kinds of things.

We need some deeper engineering that allows you to link things together,  so you can build these kinds of real apps for business :

  • Product Catalog
  • Topic Wall with Comments
  • Invoices where you can select existing products, or add a new one
  • Jobs that can be assigned to different stakeholders
  • Flexible forms with multiple sub-clauses [ Property report with any number of rooms ]
  • Social Network features …

 

Its early days, but this part we got right from the start – CollabAPI does support relations.

This means we can create apps for real business scenarios :

  • a custom CRM
  • sales business social network
  • a manufacturing process workflow
  • a real-estate property management system

 CollabAPI Beta

If you’d like to be on the beta list to try CollabAPI App Builder tool for your business mobile app… get in touch.

The more Things and Relations your app has, the better !

gord

justgord at gmail dotcom

Been ultra busy lately on two distinct web projects, both of which really need a good architecture so they can scale easily.

One nice way to do this is with services which message each other – if you have a fast, persistent reliable message queue you can have many processes grab jobs off the queue and thus scale out over many cores.  I feel this is a natural way to scale out node.js apps.

I immediately discarded SQS [ Amazon’s scalable queue service ] as it basically polls a web url to check for messages, so it really is not a message queue at all.

Another nice option is Mongo tailable cursors, which is the message queue approach that mongo uses internally to replicate between mongo instances.   This worked sort of ok, but didnt strike me as ultra high performance.  see mongoMQ npm module for a simple api wrapper.

I was very impressed with Redis Publish/Subscribe semantics… which are very fast, but dont have persistence.

In the end I decided to use a combination of Redis pub/sub for notifications, and Redis List operations rpush/lpop to store/retrieve the actual message data.  I wrapped this in a simple node.js api, which Im calling redpill.

I have an initial implementation of a typical web app which has users and info items.  This performs rather well, with web request response times under 50ms whle 1000 items/sec are being inserted as a background task.

The nice thing is that using message queues allows you to scale up by running any number of servers.

See code + comments here :

redpill : persistent messaging using Redis primitives

gorgon : demo web server architecture with messaging between server components

Lots of things to optimise, but basically a good proof of concept and initial working demo code for this approach.