Real-time Insights and Decision Making with Fast, Big Data

9 Nov 2016 | , , ,
Share on:

By Dave Thomas

Businesses are processing exponentially more data from clicks, swipes, micropayments, cyber packets, social feeds and meter readings today. The financial services industry has been doing this on a large scale for the past two decades.

FIS has coped with steadily increasing data volumes by using a simple scalable data architecture composed of a real-time database (RDB) and a historical database (HDB). One of the major benefits of this architecture is that it is easy to scale up or out simply by adding additional RDBs and HDBs.

The in-memory RDB addresses real-time business needs while the HDB is an immutable record of all past transactions. By querying both the RDB and the HDB one can always get a consistent view of the world and the state of the business. Increasingly, RDBs are being placed in huge non-volatile memory and HDBs are being stored on fast SSDs. Working smartly with today’s memory solutions enables businesses to query all their data in near real-time providing a consistent picture directly assembled from the raw transactional data as required.

This widely used RDB/HDB architecture has lately been re-discovered by the Big Data community. Variations on the model have been described as Lambda Architecture, Gartner’s Hybrid Transaction/Analytical Processing (HTAP) and Forrester’s Translytical DB and Event Sourcing.

Download this full report to read more about Gartner research on HTAP and a cybersecurity use case where we use Kx technology with the best practice data architecture design.

SUGGESTED ARTICLES

Kx Product Insights: Inter-Trading Alert

5 Dec 2018 | , , ,

by Aidan O’Neill Kx has a broad list of products and solutions built on the time-series database platform kdb+ that capitalize on its high-performance capabilities when analyzing very large datasets. Kx for Surveillance is a robust platform widely used by financial institutions for monitoring trades for regulatory compliance. The Surveillance platform instantly detects known trading […]

Kx extends relationship with NASA Frontier Development Lab and the SETI Institute

The Exploration of Space Weather at NASA FDL with kdb+

4 Dec 2018 | , , , ,

Our society is dependent on GNSS services for navigation in everyday life, so it is critically important to know when signal disruptions might occur. Physical models have struggled to predict astronomic scintillation events. One method for making predictions is to use machine learning (ML) techniques. This article describes how kdb+ and embedPy were used in the ML application.