Turned off by another buzzword driven column? Stay with me – I actually want to discuss a quite different topic than the talked up combining of hardware across a network (gridcomputing.com is a good place to find information about distributed computing)

Grids are a favorite mean to layout complex business data, and – combined with other widgets such as tree views – allow the user to analyze processes otherwise not easily comprehensible. A lot of computing goes into the real-time preparation and rendering of those grids, hence I call it “grid computing”.

One of many areas where grids are computed is the roam of financial trading systems, which I happen to spend my days building. So rather than talking about aviation, power plant or IT systems monitoring, I’ll touch on some issues, I deal with every day:

Traders need to be presented with a wealth of information to support their decisions. In addition to the traditional price quotes on several ECNs, level 2 data about the depth of the market and auxiliary information about the traded instrument need to presented. In many scenarios, it adds up to several thousand message between the backend servers and the trader’s workstation. Of course, no human being and not even a trader can view a 1000 or even a 100 screen updates per second so information needs to be filtered down to a human readable rate. Unfortunately, our users are very particular about their view into the data, which means all trading systems need to be highly customizable. They also aggregate data into compound views, so a traditional publish subscribe metaphor even with seemingly orthogonal subjects won’t necessarily help to cut down sufficiently on data rates. For example even though a trader is covering let’s say IBM but not Microsoft, he’d still most like subscribe to market data of both companies. But I digress – take my word for it: users want LOTS of data and want them NOW. So their machines are the best what money can buy and still usually don’t keep up with high volume trading times, particularly market opening and closing.

Our systems usually throttle data into bursts of updates at a rate of 2 to 10 per second, which means we could be 0.99 seconds behind the market. Processing may add another half second or so, and we are getting into an area that is not acceptable to users, and since we already have the best hardware at the desk, only server side processing may help here. Traditionally, one hopes that some data consolidation and the application of business logic can be handled on the server to get the data rate down and minimize client side processing.

One interesting solution I have seen recently uses the MVC pattern. Instead of having the Model and the Controller being implemented on the client, all View requests are forwarded to a middle tier server by a Controller Proxy. The Controller on the server then manages the communication between Model on the server and View on the client. I’m not convinced that this approach necessarily improves the performance of the client, since we now substitute data traffic by event traffic, but it’s worth a try.

While watching the last presidential debate on the Internet, I had one other idea (not that they had anything new to say but with all the empty phrases, my mind started wandering) How about composing the view completely on the server, and use video-streaming to deliver an image to the trader? The data rate will remain constant regardless of market activity, and redundancy/failover can be provided more easily. The user may not even notice that one streaming server went down. It should work great for non-interactive content, even for a highly customized presentation. However, some issues may arise, when interaction is required.

First of all, I’m not quite sure how one can capture the area of click on a video frame, but I’m sure there are ways, and it’s just my current ignorance about video streaming technology. One would still have substantial timing issues to overcome: If a user would click one screen area that shows an instrument at the right price to buy, the system would still have no concept about what the screen area “means”. It’s all pixels and metadata need to be kept (presumably) on the server to translate pixels row x/column y at frame 123 into a “buy” instruction. Not an insurmountable problem, I suppose and one that I would like to work on one of these days.

Unless, you already have a solution, in which case you are invited to share. Alternatively, you can run with the idea, find capital, and build it. Then, I could at least brag amongst my friends of having had a cool idea that made somebody else rich and famous. More realistically, I’m pretty sure somebody is already working on this, so again let us all know. I’m sure we would all like to hear more about it.