SEMICON 2018 Snapshot: Data and the Era of AI
24 Jul 2018
By Bill Pierson The future of the semiconductor industry is looking bright judging by the breadth of new developments, initiatives and innovations on display at SEMICON West in San Francisco this July. Industry leading companies presented the latest technical and business insights into today’s opportunities and challenges, particularly in the areas of smart manufacturing and
Machine learning techniques featured in JupyterQ notebooks
19 Jul 2018
Machine learning with kdb+ has been a theme of the Kx blog over the past couple of months because of the release of a series of JupyterQ notebooks on the Kx ML GitHub. As more different kinds of developers work with ML techniques, the uses for kdb+ in ML applications is growing. The release of embedPy, which loads Python into kdb+, so Python variables and objects become q variables and either language can act upon them, has been a catalyst for this trend. With embedPy, Python code and files can be embedded within q code, and Python functions can be called as q functions.
Random Forests in kdb+
12 Jul 2018
The Random Forest algorithm is an ensemble method commonly used for both classification and regression problems that combines multiple decision trees and outputs and average prediction. It can be considered to be a collection of decision trees (forest) so it offers the same advantages as an individual tree: it can manage a mix of continuous, discrete and categorical variables; it does not require either data normalization or pre-processing; it is not complicated to interpret; and it automatically performs feature selection and detects interactions between variables. In addition to these, random forests solve some of the issues presented by decision trees: reduce variance and overfitting and provide more accurate and stable predictions. This is all achieved by making use of...
Kx and NASA FDL: Space Weather, GNSS and Exoplanets
10 Jul 2018
By Robert Hill Kx is delighted to once more be partnering with the NASA Frontier Development Laboratory (NASA FDL) team on two exciting challenges facing the space sector. This follows from last year’s successful solar activity detection work, which resulted in the ‘FlareNet’ tool (supported by Kx and Lockheed Martin) that demonstrated the potential for
Decision Trees in kdb+
5 Jul 2018
The open source notebook outlined in this blog, describes the use of a common machine learning technique called decision trees. We focus here on a decision tree which provides an ability to classify if a cancerous tumor is malignant or benign. The notebook shows the use of both q and Python to leverage the areas where they respectively provide advantages in data manipulation and visualization.