We are pleased to inform our customers, colleagues and friends that KX, the maker of kdb+, the industry’s most trusted time series database and analytics engine is strategically partnering with Snowflake. The partnership will make KX’s industry-leading Data Timehouse technology, kdb Insights, available to Snowflake customers, enabling them to analyze and gain insights from temporal and vector data.
What does this mean?
Temporal and vector data is the driving force behind digital transformation and generative AI. There is growing demand from Snowflake customers to run such workloads on the Snowflake Data Cloud, while KX customers, which includes all top 40 banks, Formula 1 teams such as BWT Alpine, healthcare organizations like Syneos, and many more, want access and interoperability with centrally stored warehouse-governed data at scale, such as the services provided by Snowflake. The partnership helps data scientists, data engineers, and developers use data organized by time and vector running kdb Insights capabilities directly inside their Snowflake environment. This delivers significantly higher levels of model and analytic performance while reducing complexity and cost. Furthermore, this helps drives integrated applications beyond the warehouse, but dependent on it, such as MLOps workflows where live events inform workflows back into the centrally managed warehouse, and vice versa.
The first release under the agreement is KX for Snowflake, a new Python-first integration of kdb Insights that enables users to run supersized Machine Learning (ML) and AI workloads and vector-processing for model development and training, and vector search for generative AI. These workloads are typical in anomaly detection, risk strategies, predictive healthcare, and predictive maintenance, and deliver 10x to 100x greater performance, speed and efficiency compared to standard Python analytics, for example those using Pandas.
The mechanism by which the interoperability specifically occurs is Python-driven, interoperating via Snowpark, using its Python API and PyKX, to enable Python code and scripts to query the warehouse without leaving their Snowflake Data Cloud. Through the interface, users can create user-defined functions, and stored procedures, to run machine learning programs or to allow time-series aggregations (common to capital markets workloads), statistical and time-series analytics (for example regression methods, which underpins much of modern data science)
Fig 1: Snowpark and PyKX together enable Python queries and analytics inside the Snowflake Data Cloud, for example machine learning model training
The collaboration unifies previously siloed data teams, helping them work better together with a simpler, more efficient, and more sustainable alternative to existing data management solutions for time and vector data. This closes the “time to insight” gap between the edge and data warehouse and compresses product life cycles and the development of real-time MLOps pipelines, including those that run outside of Snowflake but rely heavily on it, for example offering a tangible bridge between in-warehouse model training and live real-time model inference.
Fig 2: Model training occurring inside the Snowflake Data Cloud, offloading feature to Feature Stores and, with KX, supporting live real-time machine learning inference. New events could trigger new training instances back inside the Snowflake data warehouse.
“This partnership brings the power of kdb Insights, the world’s fastest vector processing data timehouse, to Snowflake, enabling users to run production-ready time series analytics and vector-processing workloads without having to change code or leave the Snowflake Data Cloud,” said Ashok Reddy, CEO KX. “Built on kdb+, our vector processing database, kdb Insights, is the ideal engine for building applications and models for generative AI and vector search use cases. We look forward to working closely with Snowflake to further the adoption and integration of KX technologies across its platform.”
KX for Snowflake will be made available on Snowpark, Snowflake’s developer framework that brings support for Python, SQL, and other programming languages to the Snowflake environment. For more information, please click here.