Kdb+ and Python: embedPy and PyQ

15 Nov 2017 | , , ,
Share on:

By Andrew Wilson and James Hanna

In September, Kx announced a range of initiatives to put machine learning (ML) capabilities at the heart of future technology development. The first library to be released as part of this initiative is embedPy, which exposes powerful Python functionality to q developers.

EmbedPy loads Python into kdb+, so Python variables and objects become q variables, and either language can act upon them. Python code and files can be embedded within q code, and Python functions can be called as q functions.

The library is currently released in a beta version available on code.kx.com here.  It will be enhanced and fully released over the coming weeks, so keep checking back for the latest changes.

EmbedPy is the mirror image of PyQ, a set of software components that simplify the running of a Python interpreter alongside a kdb+ server. Developed by Alexander Belopolsky of Enlightenment Research, PyQ covers all Python libraries, with a primary focus on numerical libraries such as NumPy and SciPy.

PyQ functions as an extension to kdb+ and features zero-copy sharing of data between Python and the q programming language. Python functions can be called from q, and q functions can be called from Python. For Python programmers, PyQ gives direct access to data in kdb+ without the need to program in q. For q programmers, PyQ offers easy access to the numerous computational and visualization libraries available for Python.

Kx has acquired the rights to PyQ and it is now available here. In line with Kx’s other Fusion for kdb+ interfaces, it is released under the Apache 2 open source license free of charge, and the code is available here.


A comparison of Python and q for data problem solving

8 May 2019 | ,

This article takes a simple, real-life problem and analyzes different solutions in Python and q. The problem leads us to discover nice areas of both programming languages, including vector operations, Einstein summation, adverbs and functional form of select statements. Each solution has lessons that deepen our IT knowledge, especially when we consider performance.