Exploring the Gartner® Report: Applying AI – Techniques and Infrastructure

By Steve Wilcockson

AI has had many false dawns, but we are far removed from early days of “fuzzy logic.”

With the recent explosion of generative AI, AI is clearly evolving from a sideline research curiosity into an exciting core business imperative. The Gartner Applying AI — A Framework for the Enterprise, illustrated below, outlines how enterprises should approach their AI initiatives:

In outlining the fundamental techniques and practices for implementing AI within the enterprise, the report states, “IT leaders struggle to implement AI within AI applications, wasting time and money on AI projects that are never put in production.”

Organizations want simplicity and flexible tools for more tasks and use cases, from research to production. At KX, we help to “squish your stack” and get more data science applications into production sooner. Here’s how:

AI Simulation

Decades into the lifespans of digital transformation, there is little “low hanging fruit” left. Efficiencies are harder to find, anomalies harder to spot, and decisions involve more people, from Subject Matter Experts to IT Teams, CTOs, and FinOps teams.

Simulation and synthetic data have always been important.  From scenario stress testing to digital-twin modelling, results are inferred from simulations over swathes of historical and synthetically generated data – KX does this 100x faster at a 10^th of the cost of other infrastructures. Common simulation case studies include:

• Monte Carlo simulations for pricing and risk analysis
• Wind Tunnel analytics in FI
• Stress testing across more scenarios, particularly amongst high volume, outliers and burst conditions

 Data Science

Your investment in AI infrastructure is wasted if data scientists cannot explore data and models easily and flexibly. With KX, tooling is driven from SQL and Python, data, and modelling languages of choice.

• Researchers, developers, and analysts use their existing skills and code bases, from model research to reporting, and from cloud data pipelines to real-time model deployments
• With PyKX, get 100x power of kdb to Python developers and notebooks
• With Pro-code interfaces, users can collaborate and augment functionality from different programming languages empowered by the unsurpassed performance of kdb

AI Engineering

As AI goes mainstream, enterprise features and guardrails are needed to ensure stability, robustness, and security. For governance, cataloguing and security, kdb integrates with tools of choice to share data and AI insights throughout the enterprise safely, securely and with speed.

• Industrialize application development by supporting the reuse of existing libraries and skills
• Adopt open standards and cloud protocols to simplify integration of governance protocols, facilitate access and authentication, ensure security, safety, resilience, and high availability
• Abstract infrastructure concerns so that developers can focus on getting projects to production sooner, providing golden insights and ensuring business value

Compute Infrastructure

AI consumes very Big Data. That, unfortunately, means huge storage and compute requirements.  With KX, storage and compute efficiencies mean cost savings and lower environmental overheads.

• KX operates directly on data, eliminating the need for latency-inducing transfer and costly data duplication
• It’s column design provides optimized storage and better compression levels
• With a low KX footprint, CPU use is optimized meaning less need for power-hungry GPUs

Generative AI

With AI no longer simply discriminative or code-queried, but also generative and prompt-queried, KDB.AI has been built to apply what’s a given for GenAI workflows with greater efficiency, and augment with direct raw vector processing power. Sample use cases range from more accurate fraud detection and insightful risk management to automated financial analysis and improved regulatory compliance – all harnessing real time information, and particularly temporal data, for improved decision making.

• With proven vector-based representation and processing, KDB.AI provides more efficient vector database vector embeddings storage and search for all types of LLMs
• Additionally, storing and computing over raw data for large data models, including, but not limited to, time-series data models
• Prompt integration for better prompt engineering and more flexible intuitive querying

Gartner, Applying AI — Techniques and Infrastructure, Chirag Dekate, Bern Elliot, 25 April 2023. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.