The Exploration of Space Weather at NASA FDL with kdb+

4 Dec 2018 | , , ,
Share on:

By Deanna Morgan

As part of Kx’s continuing relationship with the NASA Frontier Development Lab (FDL), I recently had the opportunity to work as a visiting data scientist at the NASA FDL, in Mountain View, California. The lab is an applied artificial intelligence (AI) research accelerator, hosted by the Search for Extraterrestrial Intelligence Institute (SETI), in partnership with the NASA Ames Research Center. The program brings commercial and private partners together with researchers to solve challenges facing the space science community using new AI technologies.

NASA FDL 2018 focused on four areas of research – Astrobiology, Exoplanets, Space Resources and Space Weather – each with their own separate challenges. I worked with the Space Weather 1 team on the first of the Space Weather challenges, which aimed to forecast Global Navigation Satellite System (GNSS) disruptions.

A GNSS is a network of satellites providing geospatial positioning with global coverage. The most famous example is the United States’ Global Positioning System (GPS). Such a network relies upon radio communications between satellites and ground-based receivers, which can be subject to interruptions in the presence of extreme space weather events.

Space weather refers to changes in radiation emitted by the Sun, leading to fluctuations in the Earth’s ionosphere. Changes to the electron density in the ionosphere cause fluctuations in the amplitude and phase of radio signals, referred to as phase scintillation. Radio signals propagating between GNSS satellites and ground-based receivers are affected by these scintillation events and can become inaccurate or even lost.

In a society that has become dependent on GNSS services for navigation in everyday life, it is critically important to know when signal disruptions might occur. Given that space weather events occurring between the Sun and the Earth have a nonlinear relationship, physical models have struggled to predict scintillation events. One method for making more accurate predictions is to use machine learning (ML) techniques.

Throughout the project, we examined how to to train ML models to predict scintillation events. We showed how to harness the power of the kdb+/q time-series database platform when performing data analytics and additionally, how to use embedPy for importing the necessary Python ML libraries. Publicly available historical GNSS data was used throughout.

The full paper and JupyterQ notebooks explaining the data origins, data pre-processing, feature engineering and the ML models used can be found here on the Kx Developer’s site.

Additional information about Kx at NASA FDL is below:

Case study: Kdb+ Used at NASA Frontier Development Lab in Predictive AI tool

The Exploration of Solar Storm Data Using JupyterQ

VIDEO: The Exploration of Solar Storms at NASA FDL

 

Deanna and Kx gratefully acknowledge the FDL Space Weather 1 team, Danny Kumar, Karthik Venkataramani, Kibrom Ebuy Abraha and Laura Hayes, for their contributions and support.

SUGGESTED ARTICLES

kx and machine learning

Machine Learning Toolkit Update: Multi-parameter FRESH and updated utilities

25 Apr 2019 | , ,

This latest toolkit release, is the first in a series of planned releases in 2019 that will add updates to the functionality of the FRESH (Feature Extraction based on Scalable Hypothesis tests) algorithm and the addition of a number of accuracy metrics, preprocessing functions and utilities. In conjunction with code changes, modifications to the namespace structure of the toolkit have been made to streamline the code and improve user experience.

Web Scraping – A Kdb+ Use case

24 Jan 2019 | , ,

By Abin Saju Web scraping is a method through which human readable content is extracted from a web page using an automated system. The system can be implemented using a bot/web crawler which traverses through domains or through a web browser which mimics human interaction with a page. There are many use cases for the […]