When it comes to managing and analysing data, Kx Systems' 64 bit relational database, kdb+, is designed to help financial institutions move a step closer towards the 'real-time' enterprise.

Data is the lifeblood of any company. Banks maintain large databases of historical information on customers and transactions and as trading volumes rise, they are also being bombarded by live streaming data. Increasingly, regulation in the form of MiFID and RegNMS, as well as pre- and post-trade compliance, requires firms to not only capture information in real time, but also to analyse it "on-the-fly" by comparing it against historical data in an effort to gain a competitive advantage or risk management leverage.

However, according to Simon Garland, CTO, at "high-performance" database provider Kx Systems, banks' historical data infrastructure does not lend itself to the demands of the real-time enterprise. As Garland explains, a lot of data has been stored in a way where it cannot be accessed fast enough for real-time trading. This historical data is supplemented by real-time and streaming data, but the ability for banks to analyse this information in a meaningful way has been hindered by the need to go back and forth between historical and real-time databases, which increases retrieval time. Garland says this can put many companies at a disadvantage in terms of program trading strategies, real-time risk management and compliance reporting.

Leveraging relational database technology in June 2003, Kx Systems released its next generation 64 bit database, kdb+, designed to help banks process "resource-intensive" applications such as program trading, real-time operational and market risk management by comparing streaming and historical data "on-the-fly." According to Garland, Kx Systems was one of the earlier data management vendors to make 64 bit processing available on a wide variety of platforms. Kdb+ combines both historical and "intra-day" transaction data into a single relational database thereby reducing latency. Garland says it also removes the "artificial distinction" between real-time and historical data.

"Kdb+ enables firms to do new and interesting things with the data, looking for interesting patterns, for example, based on historical events," says Garland. "They can query data in real time. It is about being able to make sense of this [streaming] data and being able to decide how they want to react," he explains. Relational database technology has its limitations, though, particularly when it comes to working with more complex data such as time series. However, Garland says that Kx uses a sophisticated array programming language (called q) to speed up the management and analysis of real-time and historical data.

The latest version kdb+ v.2.3 provides firms with the capacity to process higher volumes of historical, streaming and real-time data simultaneously. Kdb+ v.2.3 is multithreaded, which Garland says allows customers to make better use of new CPU architectures, giving them extra capacity and enhanced speed. Kdb+ is used by a range of firms including investment banks wanting to capture real-time trading positions, perform VWAP calculations on historical data, as well as sell-side firms in the US looking to comply with RegNMS in terms of capturing data on trades that fall inside or outside of NBBO (National Best Bid & Offers).

©2006 Financial i. All rights reserved. Used with permission.