Kdb+ Transitive Comparisons

6 Jun 2018 | , ,
Share on:

By Hugh Hyndman

Last summer, I wrote a blog discussing my experiences running kdb+ on a Raspberry Pi, in particular making use of published benchmark content from InfluxData to generate test data, perform ingestion, and invoke a set of benchmarking queries. As a result of kdb+’s excellent performance, I concluded that it would be a perfect fit for small platform or edge computing.

I felt that I owed it to the Kx community to take things a step further: to run performance tests against all of the products that InfluxData documented, including Cassandra, ElasticSearch, MongoDB, and OpenTSDB – and go beyond the Raspberry Pi and use a variety of other server configurations.

The difficulty with doing this is that I didn’t have time to install and configure these technologies (let alone on the Raspberry Pi), so I decided to take a different approach and exploit the old transitivity argument, where if a is greater than b, and if b is greater than c, then it follows that a is greater than c.

So, using this logic and taking InfluxData’s benchmark results at face value, I concluded that all I had to do was run the tests on my hardware and compare my results with theirs to get a broad comparison across all the other technologies. Moreover, as InfluxDB had pretty much outperformed all the other databases in their tests, I reckoned that if kdb+ outperformed InfluxDB, then by transitivity, kdb+ was the fastest of them all!

To read more about the data, queries and hardware environment that I used and the resulting performance figures please click here 

 

SUGGESTED ARTICLES

Decision Trees in kdb+

5 Jul 2018 | , , , ,

The open source notebook outlined in this blog, describes the use of a common machine learning technique called decision trees. We focus here on a decision tree which provides an ability to classify if a cancerous tumor is malignant or benign. The notebook shows the use of both q and Python to leverage the areas where they respectively provide advantages in data manipulation and visualization.

ML feature engineering with kdb+

Feature Engineering in kdb+

28 Jun 2018 | , , , , , ,

By Fionnuala Carr As part of Kx25, the international kdb+ user conference held May 18th in New York City, a series of seven JuypterQ notebooks were released and are now available on https://code.kx.com/q/ml/. Each notebook demonstrates how to implement a different machine learning technique in kdb+, primarily using embedPy, to solve all kinds of machine […]