Real-Time Data, SaaS, and the Rise of Python – a Prediction Retrospective

19 December 2023 | 4 minutes


Beware predictions! Many are made, most are forgotten, but some can come back to haunt you. So, it was with some trepidation we revisited predictions from 2018 on how Analytics-as-a-Service might become “the next big thing.”

Back then, the worldwide spend on public cloud services was predicted to grow by 17% over the next four years and SaaS to increase by 20%. Figures in 2021 revealed the actual SaaS growth was 21% and later figures in 2023 show it maintained that same trajectory. Gartner got it right back then and Precedence Research now predicts it will be followed by a healthy 14% increase by 2032. But what about our predictions on how the market for SaaS Analytics services might unfold?

SaaS Analytics: Our Predictions

1. Predictive Maintenance

Due to the massive proliferation of sensor readings across tools and plants, we forecast that “processing real-time data and drawing on historical data will enable the predictive maintenance system to anticipate critical situations before they happen and send out the appropriate alert when a problem is identified.”

Did it come true?
A recent Verified Market Research report noted that the market continues to expand because of “the need for real-time data insights, the rising popularity of big data and data science, and the rising acceptance of cloud-based solutions.”

So, the business drivers have remained constant – using predictive maintenance to reduce downtime, faults, and emergency repairs to increase asset availability and in turn grow yield and lower costs.

2. SaaS-based Analytics

We envisioned that the availability of SaaS-based analytics would lower the cost of entry and make it accessible to a wider market of potential users.

At the time, the high cost of infrastructure, difficulty accessing data and analytics, and the high-level training and skills required for personnel managing predictive programs were a challenge for all but the bigger corporations.

Did it come true?
Reports from Tracxn, a financial data aggregator, confirm that established companies like Altair, Augury and PTC now provide SaaS-based manufacturing analytics services to their portfolios. They are complemented by the services of new dedicated SaaS Analytics entrants like Siera.AI mentioned in a recent report from whoraised.

Equipment manufacturers like Siemens and Applied Materials now extend proprietary analytics and diagnostics services to a wider set of equipment across the organisation. This approach mirrors the provision of trading analytics services by tier 1 finance brokers to tier 2 and tier 3 customers in KX’s other major vertical, financial services.

What we Didn’t Predict

1.) The widespread adoption of Python as the data scientist’s tool of choice.

In response, we developed PyKX, a Python-first interface for the q language and its time-series vector database kdb. The great benefit of PyKX is that it enables Python developers to continue working with familiar tools and existing libraries to explore data using the speed and power of kdb.

PyKX coupled with kdb’s unrivalled high performance and scalability helped it address the second unforeseen.

2.) The amazing progress in machine learning capabilities and the explosion in AI adoption.

By enabling the reuse of existing libraries in the training of models over vast amounts of data, AI has enabled faster rollout and operation of machine learning initiatives.

The lesson learned is that while everyone knows it’s best to anticipate and prepare, sometimes you have to be nimble and react.

And What Doesn’t Change Anyway

For some things you don’t need a crystal ball to reveal them. The benefit of processing real-time data augmented by historical data for trend-analysis machine learning is a simple example. Doing so enables predictive maintenance systems to anticipate critical situations, take corrective (and ideally automated) action or, at minimum, instant alerting when a problem is identified.

To deliver these services, data from a myriad of sensors and devices, along with relevant asset parameters and algorithms, must be combined in a platform capable of handling high volume and performance demands.

Discover more about the ROI of real-time analytics.


Demo kdb, the fastest time-series data analytics engine in the cloud

    For information on how we collect and use your data, please see our privacy notice. By clicking “Download Now” you understand and accept the terms of the License Agreement and the Acceptable Use Policy.