Socket Sharding with kdb+ and Linux
25 Jan 2018
In the latest Kx technical whitepaper, Marcus Clarke discusses the use of Socket Sharding, which was introduced as a new feature in kdb+ V3.5. Socket Sharding enables the use of Linux's SO_REUSEPORT socket option, which allows multiple sockets to listen on the same IP address and port number combination. The kernel will then allocate connection requests across the listening processes. In this paper, Marcus investigates several scenarios where processes are running with this socket option enabled. These include adding more listener processes on the fly, sending a large volume of requests, routing connections when one listener process is busy and rolling updates to minimize downtime.