Real-time Insights and Decision Making with Fast, Big Data

9 Nov 2016 | , , ,
Share on:

By Dave Thomas

Businesses are processing exponentially more data from clicks, swipes, micropayments, cyber packets, social feeds and meter readings today. The financial services industry has been doing this on a large scale for the past two decades.

FIS has coped with steadily increasing data volumes by using a simple scalable data architecture composed of a real-time database (RDB) and a historical database (HDB). One of the major benefits of this architecture is that it is easy to scale up or out simply by adding additional RDBs and HDBs.

The in-memory RDB addresses real-time business needs while the HDB is an immutable record of all past transactions. By querying both the RDB and the HDB one can always get a consistent view of the world and the state of the business. Increasingly, RDBs are being placed in huge non-volatile memory and HDBs are being stored on fast SSDs. Working smartly with today’s memory solutions enables businesses to query all their data in near real-time providing a consistent picture directly assembled from the raw transactional data as required.

This widely used RDB/HDB architecture has lately been re-discovered by the Big Data community. Variations on the model have been described as Lambda Architecture, Gartner’s Hybrid Transaction/Analytical Processing (HTAP) and Forrester’s Translytical DB and Event Sourcing.

Download this full report to read more about Gartner research on HTAP and a cybersecurity use case where we use Kx technology with the best practice data architecture design.

SUGGESTED ARTICLES

Kx Insights: Machine learning subject matter experts in semiconductor manufacturing

9 Jul 2018 | , ,

Subject matter experts are needed for ML projects since generalist data scientists cannot be expected to be fully conversant with the context, details, and specifics of problems across all industries. The challenges are often domain-specific and require considerable industry background to fully contextualize and address. For that reason, successful projects are typically those that adopt a teamwork approach bringing together the strengths of data scientists and subject matter experts. Where data scientists bring generic analytics and coding capabilities, Subject matter experts provide specialized insights in three crucial areas: identifying the right problem, using the right data, and getting the right answers.

Transitive Comparison

Kdb+ Transitive Comparisons

6 Jun 2018 | , ,

By Hugh Hyndman, Director, Industrial IoT Solutions. A direct comparison of the performance of kdb+ against InfluxData and, by transitivity, against Cassandra, ElasticSearch, MongoDB, and OpenTSDB