socket sharding in kdb+

Socket Sharding with kdb+ and Linux

25 Jan 2018 | ,
Share on:

In the latest Kx technical whitepaper, Marcus Clarke discusses the use of Socket Sharding, which was introduced as a new feature in kdb+ V3.5. Socket Sharding enables the use of Linux’s SO_REUSEPORT socket option, which allows multiple sockets to listen on the same IP address and port number combination. The kernel will then allocate connection requests across the listening processes.

In this paper, Marcus investigates several scenarios where processes are running with this socket option enabled. These include adding more listener processes on the fly, sending a large volume of requests, routing connections when one listener process is busy and rolling updates to minimize downtime.

On Linux systems, the SO_REUSEPORT option was introduced in kernel version 3.9, so the commands in the examples below will fail if run on an older version of Linux.

Marcus Clarke is a kdb+ consultant for Kx and has worked at a number of leading financial institutions in both the UK and Asia. Currently based in New York, he is designing, developing and maintaining a kdb+ system for multiple asset classes at a top tier investment bank.

© 2018 Kx Systems
Kx® and kdb+ are registered trademarks of Kx Systems, Inc., a subsidiary of First Derivatives plc.

SUGGESTED ARTICLES

Kx Partner Companies Recognized for Excellence and Innovation

26 Jan 2018 | , , , , , , , ,

Over the last few months a number of startups using Kx technology have been recognized for excellence and innovation in their industries, including BrainWaveBank, 3DEO and RxDataScience (RxDS).
RxDS, is at the forefront of new types of interactive data science software applications for the pharmaceutical and healthcare industries. It uses Kx technology to power the Big Data analytics engine in its apps. Silicon Review noted RxDS’ rapid growth in revenue and customer acquisition in naming RxDS as one of the 30 Fastest Growing Private Companies to Watch.

Sensors Working Overtime

11 Jan 2018 | , , , ,

Kx recently became an official team supplier to Aston Martin Red Bull Racing who is using Kx technology to handle mission-critical aerodynamic data. Below is an article published by Aston Martin Red Bull Racing on 11 January 2018 which explains the importance of this aero data, and working with Kx technology, for improving car performance for the F1 Team. It outlines how Kx’s in-memory, time series database software, capable of handling millions of events and measurements every second, provides a platform for analysing data on the RB14 and its successors.