Message from the Kx COO Mark Sykes

29 May 2018 | , , , , , , , , , ,
Share on:

A couple of weeks ago, I had the pleasure of meeting hundreds of our community at the Kx25 event in New York City. Kx enjoys a hugely supportive developer and business community across the world. I see constant support and advice offered between customers, amazing attendance at 133 global Meetups in countries across six continents, and a healthy stream of ideas and comments shared via email, GitHub and other support platforms. We like to add to that and bring as many people as possible together at an international kdb+ user conference every couple of years, and Kx25 was our largest to date, held in part to commemorate our amazing journey over the last 25 years…

These gatherings have proven to be pivotal moments for Kx. Having the opportunity to get broad feedback from our user community is a hugely important exercise for us. We know that many of our users have built their careers around kdb+, and we recognize our responsibility to them. That is why we work so hard to constantly improve the world class software we deliver. Kx is unique in its facility to both listen, and respond, with significant changes to each new annual release of kdb+ and our supporting range of products, solutions and tools.

Our last such gathering was KxCon in 2016. Held in Montauk over three days, we were able to gather a whole host of ideas that rapidly formed part of our development roadmap for the next two years.  Since then, we released kdb+ version 3.5 which was significantly faster than previous versions and included some new features that were heavily influenced by community feedback. Next came kdb+ on-demand, a subscription version of kdb+.

We are finding that customers are using kdb+ on-demand to work on their desktops for non-continuous compute, on the Cloud, and increasingly on premises to provide cheap scaleout of processing when more cores are needed for short periods of time. It is a very flexible mechanism for burst processing and it can co-exist with all of our other licensing solutions. We followed this by making the product free for all non-commercial use, a move that has proven so popular that thousands are using it at any particular moment. It’s another way we are introducing kdb+ to more and more people and propagating the use of the language.

Another change we made following KxCon was to give customers the ability to use unlimited cores when needed. If you have a project with four developers on it, and you need to use unlimited cores for that particular project, drop me an email, you’ll be surprised by how we are able to facilitate that so you don’t need the full enterprise licensing for these use cases.

Over the past two years we have also invested in making our developer resources at code.kx.com more accessible to both our existing community and to users exploring kdb+ for the first time. Whilst we kept the previous wiki version of code.kx.com, we have committed to our new Developers’ site and add all new material to it.

We have also invested in building up an internal machine learning team and partnering with Brainpool.AI, a consortium of machine learning experts. As more of our customers begin their own machine learning projects we have committed to staying ahead of the curve and developing machine learning libraries and integrations so that ML can be performed within the kdb+ fabric. This started with putting Python and its rich ecosystem of libraries inside kdb+ via our embedPy library, and more kdb+ native ML libraries are being released over the next few months.

Continuing our commitment to giving more tools to new users we are now providing the complete guide to kdb+ programming, Q for Mortals, by Jeffry Borror, for free on our Developers’ site. It has proven to be one of the most popular sections of the site. More recently we have published a series of videos called Q for All, presented by Jeffry Borror, which gives an overview of programming in q over the course of 15 short videos.

Starting in 2017 we began a continuing project of extending the use of kdb+ to other languages and tools like Kafka, Java, R and FFI called Fusion Interfaces, and yes, HDFS and Spark are in the pipeline. As part of the Fusion project we built embedPy and acquired the popular PyQ product, making kdb+ a natural partner to a Python programmer. PyQ and embedPy have gone into production and are already being used in front- and back-office mission critical applications.

Our Python tools, like our new JupyterQ notebooks, have proved to be hugely popular. But the fact that they have gone into production does not mean that we are going to stop development on them. We already have a roadmap for those products and we are going to carry on with that development. And we want feedback, the more you can tell us about what you want, the more we can add that to the product.

On the heels of our expanded connectivity with Python at Kx25 I announced that kdb+ is now available directly from the world’s largest Python resource, the Anaconda’s distribution platform expanding kdb+ exposure to hundreds of thousands of Python users each year.

Other announcements this year included our partnership with Magnitude/Simba who are building a new, more powerful ODBC driver to kdb+, which will be made available to our users at no cost.

What is next for Kx? I was especially glad to be able to announce the addition of kdb+ to the Google Cloud Launcher, which allows kdb+ to be quickly deployed from the Cloud. This is the first of many Cloud announcements you can expect from Kx in the next year, plus watch this space for more developments with Docker and Kubernetes.

We are prioritizing which partnerships, interfaces and features to tackle next based on feedback from you, our user community. , or send your feedback to and tell us your wish list, your pain points, or just share your experiences. I’d love to hear from you.

Mark.

SUGGESTED ARTICLES

cryptocurrency microstructure with kdb+

Combining high-frequency cryptocurrency venue data using kdb+

19 Sep 2018 | , , , ,

By Eduard Silantyev   Eduard Silantyev is an electronic trading systems developer and a cryptocurrency market microstructure specialist based in London. Follow Eduard on LinkedIn or Medium to read more of his blogs about cryptocurrencies. The original title of this blog is “Cryptocurrency Market Microstructure Data Collection Using CryptoFeed, Arctic, kdb+ and AWS EC2 | Handling […]

Signal processing in kdb+

Signal processing with kdb+

6 Sep 2018 | , ,

In the latest in our ongoing series of kdb+ technical white papers published on the Kx Developer’s site, Kx engineer Callum Biggs examines how kdb+/q can be used instead of popular software-based signal processing solutions. Signal processing is used for analyzing observable events, such as IoT sensor data, sounds, images and other types of pulses […]

Python vs kdb+ for data analytics

A comparison of Python and q for data analysis

21 Aug 2018 | , , , ,

Guest blogger Ferenc Bodon illustrates using Python, SQL and kdb+ for data analytics in this blog. He takes an example that goes just one step beyond the simplest use cases by performing some aggregation based on multiple columns. Anybody who analyzes data tables will bump into this type of problem, probably on the third day.