Xilla related – observations and links

Hacking Xilla

Ive been hacking on a proof-of-concept for Xilla, really to try out some of these ideas and see where they go.. at some stage Ill upload that to the projects home –  WebXilla.com

spreadheet limitations?

In the meantime, given my comments about how people use spreadsheets instead of databases and why, I want to keep a link to this Hacker News discussion – “What Cant you do in Excel?”

The discussion is fascinating, the guts of it boils down to –

• its got to be on the web, in the browser [shareable, publishable, multiuser]
• spreadsheets should be more like databases [data types, data relations, named attributes]
• we want to use spreadsheets as a live interface GUI for databases, sometimes
• need a better language for programming [eg. python, ruby, lisp – anything but VB!]
• need a better language for querying [ SQL or an alternative]
• need better naming of data ranges [names not C1:C37]
• better import/export/conversion/integration

To this I add my own wishlist –

• browse data links [relations] as web hyper-links
• runtime editable schema
• data-app prototyping environment, RAD tool
• complete versioning, check-pointing and backup
• fine grained user access controls & permissions

A spreadsheet that has all of these is no longer a spreadsheet, or a database but a new new thing – so lets stop asking how to improve spreadsheets and ask how to build this data-web-‘Xilla’.  I’m calling this new thing Xilla, because it needs a name.

Its tempting to deny or address the opinions above by saying ‘if you need better data management, grow up and use a real database’ .. but the real point is that spreadsheets solve part of a problem, databases solve part of an overlapping problem – and where we are now is that neither solves the problem really well, or at least well enough.

The reason people stick with spreadsheets is because they partly work, and they don’t want to move to a database and lose what flexibility they have – living with the limitations is better than fixing a schema in stone and writing a web app that is tied completely to that schema [and losing all the nice computational tools and UI they currently have in the process].  In an organizational context, theyll have to give up their dato to the experts, and once they do that theyll never see it again, or at least have no rights to change and redefine the structure or adapt it to the problem they have as it evolves.

I think quite a few startups are basically porting the spreadsheet or traditional database onto the web.. but thats not really what is needed, or its only 10 to 15 percent of whats needed… This approach rules out the new kind of app that might come into existence if we really revisited the assumptions.

Independently of users data needs, theres a lot of movement towards a pure attribute based or key-hash style of database.  David Heinemier Hansen mentions that Rails uses the database as a kind of big hash table – data is given a key and thats used to get the data back, so most of the other DB features arent really used.  Amazons Cloud DB service seems to do a similar thing, and many many web apps are getting huge scalability and performance gains by going for the memcached api with a similar style.

Combine that with the ubuquity of XML, and the fact that treelike data doesn’t fit easily within a table database structure and it seems the whole “data on web” space is ready for an overhaul, and there are sneak peeks of how it might be done the right way.

There is a better way to manage data – its there in front of us, perfectly formed inside the block of marble, just waiting to be uncovered.

Visual DSLs in finance?

It seems to me the time is ripe for Visual DSLs to appear in the financial quantitative domain – you can imagine the pure housekeeping involved in having 2500 trade strategies operating across 3500 tick feeds.. hmmm.

So you might be asking what the hell is a DSL?  A DSL or Domain Specific Language can mean many things, but usually falls into 2 classes – Visual and Textual.

Say I want to create a web app that generates some HTML page dynamically… regardless of whether I program in Ruby or Lisp I really want a nicer way to open and close HTML tags than

print('<html>\n') ... print(</html>\n')

… a textual DSL gives you a language that is a much closer match to the problem .. I want to just write something like