By Steve Wilcockson,
In May 2022, Treliant’s Capital Markets practice announced a partnership with KX. The partnership helps organizations of all types deliver large-scale digital transformation and business and regulatory-driven change projects. Treliant’s expertise and experience in conjunction with KX’s world-leading time series database and analytics engine, a key part of the critical infrastructure for many of the world’s leading financial institutions, enables organizations to enjoy insights at the speed of thought and unrestricted observability of data.
Treliant’s deep technical and subject matter expertise across the breadth and depth of KX’s capabilities is impressive. They’ve become a firm favorite with KX clients.
The Treliant team has put together a trifecta of excellent technical blogs, the latest of which, “Getting Started with kdb Insights,” does exactly what it says on the tin. It shows how with minimal setup and just 10 lines of code, you can configure an on-cloud trade analytics infrastructure – thus bringing kdb’s extraordinarily efficient and performant analytics to microservices- and cloud data analytics infrastructures.
As the Treliant team shows, “[kdb Insights] provides more open integration with cloud-native analytics stacks, to enable you to take your machine learning operations pipelines to the next level.” At the very highest cloud level, “the benefits to being cloud-based are endless, some of the more obvious being cloud’s elasticity, scalability, cloud security, and ease of onboarding/set up.”
Tier 1 banks and key capital markets are certainly embracing cloud, increasingly overcoming constraints such as very low latency requirements, regulatory, co-location, and data delivery needs. As one top 5 bank has exemplified, with KX they’ve delivered the most successful cloud adoption in its history taking just 4 months, achieving storage cost savings of some 80%+. And yes, that is a complex, ultra-performant front office trading infrastructure running cloud natively, not a relatively procedural back or middle office infrastructure lift and shift upgrade.
The Treliant team shows how you can quickly set up cloud instances of kdb insights, highlighting in particular the Stream Processor and Reliable Transport microservices, enabling a powerful cloud-optimized version of kdb+ tickerplant. Having set up instances, they show how easy it is to run a randomized TAQ (trade and quote) generator, then, with a dataUpdate function, publish data to Trade and Quote tables.
dataUpdate:{
publishTrade (tradeSchema upsert {.z.p,x} each flip (genTrade 1+rand 20)
publishQuote (quoteSchema upsert {.z.p,x} each flip (genQuote 1+rand 80))
};
In the constructed trade and quote pipelines, ingested data is displayed via the Writers function, with keyBy and stats functions enabling analysis.
tradeStream: .qsp.read.fromCallback[`publishTrade] .qsp.keyBy[`sym] .qsp.stats.sma[`price;10;`priceMovingAverage] .qsp.write.toVariable[`trade;`upsert]
The whole process takes about 10 lines of code. So easy, and the team is just scratching the surface of the cloud, microservices, Python and other config and pro-code capabilities available with kdb Insights.
It’s a simple, easily replicable example. Contact KX for a demo of kdb Insights. Our Sales Engineers will point you to the relevant functionality and key microservices to get you started.
In addition to the blog Getting Started with kdb Insights(May 9, 2023, Paul Walsh, Daniel Lam & Thomas Smyth), here are two other technical blogs from the Treliant team.
Improving Query Performance in kdb+ Databases (Sept 13, 2022, Paul Walsh & Thomas Smyth)
Introduction to Parallel Processing in kdb+ (Oct 18, 2022, Paul Walsh & Thomas Smyth)
Next up, how the Treliant team uses PyKX to infuse Python code bases and notebooks with ultra-efficient in-memory data and analytics advantages of KX instances, helping infuse Python research workflows with more data delivered faster powering ultra-efficient applications, better models, faster lifecycles from model to production analytic.
Happy coding!
