kdb+ and Python

Kdb+ and Python: embedPy and PyQ

15 Nov 2017 | , , , ,
Share on:

By Andrew Wilson and James Hanna

In September, Kx announced a range of initiatives to put machine learning (ML) capabilities at the heart of future technology development. The first library to be released as part of this initiative is embedPy, which exposes powerful Python functionality to q developers.

EmbedPy loads Python into kdb+, so Python variables and objects become q variables, and either language can act upon them. Python code and files can be embedded within q code, and Python functions can be called as q functions.

The library is currently released in a beta version available on code.kx.com here.  It will be enhanced and fully released over the coming weeks, so keep checking back for the latest changes.

EmbedPy is the mirror image of PyQ, a set of software components that simplify the running of a Python interpreter alongside a kdb+ server. Developed by Alexander Belopolsky of Enlightenment Research, PyQ covers all Python libraries, with a primary focus on numerical libraries such as NumPy and SciPy.

PyQ functions as an extension to kdb+ and features zero-copy sharing of data between Python and the q programming language. Python functions can be called from q, and q functions can be called from Python. For Python programmers, PyQ gives direct access to data in kdb+ without the need to program in q. For q programmers, PyQ offers easy access to the numerous computational and visualization libraries available for Python.

Kx has acquired the rights to PyQ and it is now available here. In line with Kx’s other Fusion for kdb+ interfaces, it is released under the Apache 2 open source license free of charge, and the code is available here.

© 2018 Kx Systems
Kx® and kdb+ are registered trademarks of Kx Systems, Inc., a subsidiary of First Derivatives plc.


Migrating a kdb+ historical database to the Amazon Cloud

18 Apr 2018 | , , ,

If you are currently migrating, or considering migrating an historical kdb+ database (HDB) to the Cloud, you will want to read this white paper on the kdb+ developers’ site which looks at popular storage solutions available within the Amazon Web Services (AWS) Cloud. The paper also compares and contrasts the performance of kdb+ on EC2 instances versus physical hardware.

Head of Products, Solutions and Innovation at Kx on Product Design and the Vision for the Future

16 Mar 2018 | , , ,

As the SVP of Products, Solutions and Innovation at Kx Systems, James Corcoran is part of a new chapter in software development at Kx. Since joining Kx parent First Derivatives as a financial engineer in 2009, James has worked around the world building enterprise systems at top global investment banks before moving to the Kx product team in London. James sat down with us recently to discuss his perspective on product design and our technology strategy for the future.

Kdb+ Utilities: Essential utility for identifying performance problems

28 Feb 2018 | ,

If you are a kdb+/q developer, you will find the utilities created by Kx Managing Director and Senior Solution Architect Leslie Goldsmith to be a valuable resource. The “Kdb+ Utilities” series of blog posts gives a quick introduction to the utilities, available at Leslie Goldsmith’s GitHub. In this third part of the series we look at Leslie’s qprof, which allows a programmer to drill down into q functions or applications to inspect performance and CPU usage in a fine-grained fashion.