Running kdb+ on Google Cloud (GCP)

8 November 2018 | 5 minutes

By Himanshu Gupta

Our guest blogger, Himanshu Gupta, is an expert kdb+ programmer who frequently writes about kdb+/q, as well as other technologies on his blog, enlist q, where this post originally appeared. In this latest article Himanshu gives a tutorial on using kdb+ on the Google Cloud. Check out enlist q, and subscribe while you are there so you don’t miss Himanshu’s latest articles.

A lot has changed in the kdb+ world recently, and as the world moves to the cloud, it was only a matter of time before kdb+ did as well. A few months ago, KX announced that users will now be able to spin up an instance of kdb+ on Google Cloud (GCP) and pay per usage. This is drastically different than how we have used kdb+ in the past. If I remember correctly, you usually had to get an enterprise license and/or pay per core. With kdb+ on GCP, you have the option to only pay for kdb+ when you use it. You can also scale it easily and not have to worry about burying yourself in piles of paperwork trying to get approvals for new hardware. We all know how difficult and time consuming that can be. And especially these days with so much volatility in the market, it’s necessary for your infrastructure to be able to scale on demand. However, if you are a hardcore kdb+ developer, you might question whether kdb+ can perform just as well on a cloud infrastructure. Well, Google recently published the results of the latest STAC benchmark on their blog that confirm that kdb+ can indeed be as powerful on cloud. I am not going to go into the details of the results, but I will show you how you can spin up a kdb+ instance quickly on GCP.

Follow these steps:

STEP 1: Go to GCP Market Place and search for kdb+. That will take you to the kdb+ landing page (link):
kdb+ on Google Cloud Platform - KX

kdb+ on Google Cloud Platform – KX

As you can see on the page, there is a 30-day trail available for free.

If you scroll down, you will see more information about pricing. The numbers might differ based on your location. Interesting thing to note here is the grey box on left. It states that the solution trial will not charge for license fee but you will still have to pay for the compute costs. Be careful when you start up your own instance. I will go ahead and start one for this tutorial but make sure to terminate it once you are done, or else you will get a hefty bill at the end of the month.

kdb+ Pricing on Google Cloud Platform - KX

kdb+ Pricing on Google Cloud Platform – KX

STEP 2: Click on “Launch On Compute Engine” to spin up an instance. A new window will pop-up asking you to create/select a project.

GCP Launch On Compute Engine - KX

GCP Launch On Compute Engine – KX

STEP 3: Click on the “+” sign to create a folder and then enter a name for your project. I have selected kdbOnGCPTutorial.

Google Cloud Platform New Project - KX

Google Cloud Platform New Project – KX

STEP 4: Click on ‘Create’ and a pop-up on top right will let you know once the instance is up:

Create Project GCP - KX

Create Project GCP – KX

STEP 5: Now that we have created a new project, go back to the main kdb+ landing page and click on “Launch On Compute Engine” button again. This time, you will see your new project on the pop-up. Select that project.

Launch On Compute Engine - KX

STEP 6: This will take you to a new page where you need to agree to some Terms and Conditions to start your free trial. Chose your country and fill any other additional information. Once you have filled all the information, “AGREE AND CONTINUE” button will appear at the bottom. Click on that button to continue.

Deploy kdb+ to Google Compute Engine - KX

STEP 7: On the second page, you will need to enter some personal information. Once you have done that, click on “START MY FREE TRIAL” button at the bottom.

For some reason, this didn’t take me to any new page which is weird. So, I clicked on “Launch On Compute Engine” button one more time and this pop-up asked me to enable billing.

Enable Billing on Google Cloud Platform - KX

Once you take care of that, GCP finally starts building your kdb+ instance. You will see this on your page while you wait:

New kdb+ Deployment - KX

STEP 8: After about 2 mins, a new page appears asking me to configure details for my kdb+ instance:

GCP New kdb+ Deployment - KX

Since we are not doing anything crazy here, we will stick to defaults. Accept the Terms and Conditions and click on “Deploy” at the bottom of the page:

Google Cloud Platform Networking - KX

You are now taken to “Deployment Manager” page where you can manage, well, all your deployments. My instance took about a 1 minute to be deployed.

GCP Deployment Manager Page - KX

STEP 9: Now that our instance is up and running, we need to connect to it. We will use SSH to do so. On the right hand side, you can see instructions on how to connect to your kdb+ instance via SSH. Click on the ‘SSH’ box and a new window will pop-up. This pop-up is basically your terminal.

Google Cloud Platform SSH - KX

That’s pretty much it. You can now easily start a q session by typing “q”.

Google Cloud Platform, Start a q Session by Typing “q” - KX

STEP 10: Once you are done, go back to the “Deployment Manager” page and ‘Delete’ your deployment:

GCP, Deploy Marketplace Solution - KX

It took about 2-3 minutes to terminate my instance.

That’s it! Hope you found the tutorial helpful. As mentioned earlier, be careful when starting instances on any cloud platform and make sure to terminate them once you are done.

Demo kdb, the fastest time-series data analytics engine in the cloud

    For information on how we collect and use your data, please see our privacy notice. By clicking “Download Now” you understand and accept the terms of the License Agreement and the Acceptable Use Policy.