distributed computing with kdb+/q

Distributed Computing in kdb+

21 Dec 2016 | , , ,
Share on:

Kx financial engineer, Connor Gervin, gave a talk at a Kx Community NYC Meetup in 2015 that is worth revisiting. Connor described kdb+’s built-in multithreading and multiprocessing capabilities, which are an essential part of every serious kdb+ programmer’s’ toolkit. With these features, programmers can make the best use of multicore hardware when solving increasingly complex problems over ever-expanding datasets.

In a guided overview with real code examples, Connor outlines how the multicore and multithreading aspects of parallel execution operations; multithreading input queues; multithreaded access to the HDB; segmentation and MapReduce all work together to remove memory limits and single-thread computing constraints.

Connor demonstrates a kdb+ solution to the complex problems posed by massive datasets that shows programmers how to significantly improve vector operations, disk performance and optimize queries.

Watch Connor’s presentation on the Kx Systems’ YouTube channel (here) and while you’re there, subscribe so you never miss new videos about Kx and kdb+.

 

© 2018 Kx Systems
Kx® and kdb+ are registered trademarks of Kx Systems, Inc., a subsidiary of First Derivatives plc.

SUGGESTED ARTICLES

Sensors Working Overtime

11 Jan 2018 | , , , ,

Kx recently became an official team supplier to Aston Martin Red Bull Racing who is using Kx technology to handle mission-critical aerodynamic data. Below is an article published by Aston Martin Red Bull Racing on 11 January 2018 which explains the importance of this aero data, and working with Kx technology, for improving car performance for the F1 Team. It outlines how Kx’s in-memory, time series database software, capable of handling millions of events and measurements every second, provides a platform for analysing data on the RB14 and its successors.

kdb+ for industrial internet of things 4.0

Kx Insights: IIoT for Predictive Maintenance and Big Data

9 Jan 2018 | , , , , , ,

IIoT for predictive maintenance enables more extensive monitoring of equipment and processes at a much lower cost than traditional methods and delivers actionable warnings to prevent or minimize the consequences of an impending failure. Where IIoT for predictive maintenance is deployed in a well-designed program using Reliability Centered Maintenance (RCM) it will reduce surprise outages, lost production, extensive repairs, secondary damage and increase safety.

MiFID II and kdb+/Kx

MiFID II, The Day After

4 Jan 2018 | , , , , ,

Don’t pop the champagne yet for the launch of Europe’s Markets in Financial Instruments Directive (MiFID II). While January 3rd was the go-live date, and the industry is thought to be 80% compliant, there are major outstanding issues to contend with in 2018. Yesterday was simply a milestone marking the turn to the home stretch.
Although MiFID II has been in the works for many years, its Level II and Level III guidance was still being finished in the second half of 2017, which has meant an uncomfortable level of ambiguity for those who must implement MiFID II.