The Exploration of Space Weather at NASA FDL with kdb+
4 Dec 2018
Our society is dependent on GNSS services for navigation in everyday life, so it is critically important to know when signal disruptions might occur. Physical models have struggled to predict astronomic scintillation events. One method for making predictions is to use machine learning (ML) techniques. This article describes how kdb+ and embedPy were used in the ML application.
Machine learning: Using embedPy to apply LASSO regression
23 Oct 2018
By Samantha Gallagher The use of kdb+ for machine learning in financial technology and other industries is expanding following the release by Kx of the powerful embedPy interface, which allows the kdb+ interpreter to manipulate Python objects, call Python functions, and load Python libraries. Now Python and kdb+ developers can fuse both technologies together,
Random Forests in kdb+
12 Jul 2018
The Random Forest algorithm is an ensemble method commonly used for both classification and regression problems that combines multiple decision trees and outputs and average prediction. It can be considered to be a collection of decision trees (forest) so it offers the same advantages as an individual tree: it can manage a mix of continuous, discrete and categorical variables; it does not require either data normalization or pre-processing; it is not complicated to interpret; and it automatically performs feature selection and detects interactions between variables. In addition to these, random forests solve some of the issues presented by decision trees: reduce variance and overfitting and provide more accurate and stable predictions. This is all achieved by making use of...
Kx on the Google Cloud Platform
26 Jun 2018
At Kx25, the international kdb+ user conference held in New York City on May 18th, Kx announced that kdb+ is now available on the Google Cloud Launcher. Antonio Zurlo of Google Cloud Platform (GCP) gave a presentation about Google Cloud and described an example of how to use kdb+ on the GCP.
Classification using K-Nearest Neighbors in kdb+
21 Jun 2018
As part of Kx25, the international kdb+ user conference held May 18th in New York City, a series of seven JuypterQ notebooks were released and are now available on https://code.kx.com/q/ml/. Each notebook demonstrates how to implement a different machine learning technique in kdb+, primarily using embedPy, to solve all kinds of machine learning problems, from feature extraction to fitting and testing a model.
Dimensionality Reduction in kdb+
14 Jun 2018
Dimensionality reduction methods have been the focus of much interest within the statistics and machine learning communities for a range of applications. These techniques have a long history as being methods for data pre-processing. Dimensionality reduction is the mapping of data to a lower dimensional space such that uninformative variance in the data is discarded. By doing this we hope to retain only that data that is meaningful to our machine learning problem. In addition, by finding a lower-dimensional representation of a dataset we hope we can improve the efficiency and accuracy of machine learning models.