Products
kdb+
kdb Insights
KDB.AI
Delta
Sensors
Surveillance
Market Partners
Cloud Service
Data Lake
Market Data
Services & Support
Learn
Documentation
KX Academy
Developer Blog
YouTube Channel
Connect
KX Community
Slack Community
StackOverflow
Events
Build
29 March 2021
KX recently hosted a webinar entitled “Leverage the Microsecond Mindset in the Cloud” in which a number of industry experts discussed the migration of data analytics to the cloud and how the conversation has shifted from “why?” to “how?”.
Joining Kathy Schneider, CMO at KX, in the webinar to share their thoughts on the challenges involved, and the advantages of getting it right, were:
As part of the webinar Vincent Lam, VP Product Marketing at KX, gave an introduction to KX Insights, the newly launched cloud-first streaming analytics platform from KX.
Nathan Sykes gave an overview of the use of data science in the F1 world and the demands it places on supporting systems. Clearly speed is a dominant factor and he outlined the many manifestations it has: from increasing it on the track to reducing it in the pits, from maximizing it in the wind tunnel to minimizing its delay in making decisions. The corresponding system challenges are to ingest, process and visualize, with equal speed, the 10s of millions of events per second from sensors, often operating in extreme conditions, in order to make critical decisions in each of these areas.
He explained how his decision to select KX was based on the success achieved within one half-day of working with the technology, during which data was collected, metrics were calculated and a complex heat map was generated combining views of rev range, fuel flow and potential engine damage modeling. The speed of implementation and deployment was in stark contrast to other solutions, due largely to the supporting tools it provided and also the low footprint which enabled it to run on relatively low-resource commodity hardware and further save on TCO.
But its operational speed was crucial too, particularly in an environment that seeks, where possible, to replace the human factor in decision-making by automation. The experience with KX has shown that, even in the cloud, incredibly low latency round-trips are achievable. This was strikingly illustrated by the fact that data transmissions from the track to servers in the UK or France and back, were operating faster than their video stream. In effect, decisions were being made faster than they could be seen on their TV screens!
The upshot is that Alpine 1 now use KX throughout their operations include on the car, trackside, wind tunnel, CFD and Dyno. Moreover, they are increasing the data sources and the resulting analytics they can perform, with particular emphasis on cloud-based development because of the cost-benefits and flexibility it provides.
Many of these points were mirrored in the overview of KX Insights that Vincent Lam then presented. These included its cloud-first approach giving access to low-cost storage and a wide ranges of microservices for faster development, its Kubernetes-enabled ability to integrate into data and processing workflows across multiple systems, both in the cloud and on-prem, and the simplicity of being able to install it instantly from cloud marketplaces and consume just those components that are needed. KX Insights is about leveraging the cloud for all its benefits in terms of services, protocols and technologies that make developing insights easier and faster.
In the discussion that followed the panelists reflected on trends in cloud adoption. Dan Seal commented that from a software vendor perspective he has seen a marked uptick in client commitment to migrating to the cloud, particularly in the last two years, with it being exemplified in concrete plans as opposed to merely speculative thoughts. Ashish Mojmumbar remarked that for cloud providers like him, the role has changed from that of outbound evangelist on the benefits of the cloud to inbound collaboration with clients asking for advice and direction on ways the cloud can help them achieve competitive advantage, new streams of revenue and, in more extreme cases, remain relevant.
Part of the uptake was based on the recognition that many of the perceived barriers to cloud adoption have been addressed, particularly around areas like security where the expertise of cloud providers is now seen as a core competence of their domain and greater than that of their customers. Similarly, areas like latency have been addressed through investment in dark fibre enabling the provision of latency SLAs.
As noted by Daniel Mok, however, some industries remain reluctant to embrace public clouds, particularly those in areas like defence where data IP or residency may be an issue, so the need for on-prem processing will remain. This led to a discussion on the importance of deciding on a holistic data strategy, be it single cloud, hybrid or multi-cloud with counsel that securing executive sponsorship to underpin it is equally important.
Overall there was consensus that there is now a wider realization that the cloud offers much more than simply savings on infrastructure. All noted that the “lift-and-shift” approach that characterized much early adoption is now being replaced by a broader vision that sees the cloud as an opportunity for significant transformation across systems, operating models and company culture. The catalysts in these areas are the services, connectivity and development flexibility that the cloud offers. This was very much endorsed by Nathan Sykes in the term “build and breakdown” that characterized how he got the best out of the cloud, referring to the ability to swiftly prototype what could be done, improve and fine-tune it, then port it to a different environment and scale it up as needed in a way that neither budgets nor access to services would allow on-prem.
Final comments from all participants reflecting on their respective journeys to the cloud reverted to the theme of speed. They all said they should have done it earlier!