kdb+ and Python

Kdb+ and Python: embedPy and PyQ

15 Nov 2017 | , , ,
Share on:

By Andrew Wilson and James Hanna

In September, Kx announced a range of initiatives to put machine learning (ML) capabilities at the heart of future technology development. The first library to be released as part of this initiative is embedPy, which exposes powerful Python functionality to q developers.

EmbedPy loads Python into kdb+, so Python variables and objects become q variables, and either language can act upon them. Python code and files can be embedded within q code, and Python functions can be called as q functions.

The library is currently released in a beta version available on code.kx.com here.  It will be enhanced and fully released over the coming weeks, so keep checking back for the latest changes.

EmbedPy is the mirror image of PyQ, a set of software components that simplify the running of a Python interpreter alongside a kdb+ server. Developed by Alexander Belopolsky of Enlightenment Research, PyQ covers all Python libraries, with a primary focus on numerical libraries such as NumPy and SciPy.

PyQ functions as an extension to kdb+ and features zero-copy sharing of data between Python and the q programming language. Python functions can be called from q, and q functions can be called from Python. For Python programmers, PyQ gives direct access to data in kdb+ without the need to program in q. For q programmers, PyQ offers easy access to the numerous computational and visualization libraries available for Python.

Kx has acquired the rights to PyQ and it is now available here. In line with Kx’s other Fusion for kdb+ interfaces, it is released under the Apache 2 open source license free of charge, and the code is available here.



The Exploration of Solar Storm Data Using JupyterQ

7 Jun 2018 | , , , ,

At Kx25, the international kdb+ user group conference held on May 18th, I made a presentation that demonstrated how to use a JupyterQ notebook to perform analytics on solar storm data.
My experience working with solar storm data began last year when I was a visiting data scientist at the NASA Frontier Development Lab (FDL) in Mountain View, California. The FDL is hosted by the Search for Extraterrestrial Intelligence Institute (SETI): a not-for-profit research organization founded in 1984, whose purpose is to drive research in the space sciences. Within SETI, FDL is an applied artificial intelligence research accelerator established to maximize new AI technologies and apply them to challenges in the space sciences.

Kdb+ Transitive Comparisons

6 Jun 2018 | , ,

By Hugh Hyndman, Director, Industrial IoT Solutions. A direct comparison of the performance of kdb+ against InfluxData and, by transitivity, against Cassandra, ElasticSearch, MongoDB, and OpenTSDB