The kdb Insights portfolio brings the small but mighty kdb+ engine to customers wanting to perform real-time analysis of streaming and historical data. Available as either an SDK (Software Development Kit) or fully integrated analytics platform it helps users make intelligent decisions in some of the world’s most demanding data environments.
In our latest update, kdb Insights 1.10, KX have introduced a selection of new features designed to simplify system administration and resource consumption.
Let’s explore.
New Features
Working with joins in SQL2: You can now combine multiple tables/dictionaries natively within the kdb Insights query architecture using joins, including INNER, LEFT, RIGHT, FULL, and CROSS.
Learn how to work with joins in SQL2
Implementing standardized auditing: To enhance system security and event accountability, standardized auditing has been introduced. This feature ensures every action is tracked and recorded.
Learn how to implement auditing in kdb Insights
Inject environment variables into packages: Administrators can now inject environment variables into both the database and pipelines at runtime.. Variables can be set globally or per component and are applicable for custom analytics through global settings.
Learn more about packages in kdb Insights
kxi-python now supports publish, query and execution of custom APIs: The Python interface, kxi-python has been extended to allow for publishing and now supports the execution of custom APIs against deployment. This significantly improves efficiency and streamlines workflows.
Learn how to publish, query and execute custom APIs with kxi-python
Publishing to Reliable Transport (RT) using the CLI: Developers can now use kxi-python to publish ad-hoc messages to the Insights database via Reliable Transport. This ensures reliable streaming of messages and replaces legacy tick architectures used in traditional kdb+ applications.
Learn how to publish to Reliable Transport via the CLI
Offsetting subscriptions in Reliable Transport (RT): We’ve introduced the ability for streams to specify offsets within Reliable Transport. This feature reduces consumption and enhances operational efficiency. Alternative Topologies also reduce ingress bandwidth by up to a third.
Learn how to offset streams with Reliable Transport
Monitoring schema conversion progress: Data engineers and developers now have visibility into the schema conversion process. This feature is especially useful for larger data sets, which typically require a considerable time to convert.
Learn how to monitor schema conversion progress
Utalizing getMeta descriptions: getMeta descriptions now include natural language descriptions of tables and columns, enabling users to attach and retrieve detailed descriptions of database structures.
Learn how to utilize getMeta descriptions
Get started with the fastest and most efficient data analytics engine in the cloud
Feature Improvements
In addition to these new features, our engineering teams have been busy working to improve existing components. For example: –
- We’ve optimized getData for queries that span multiple partitions.
- We’ve introduced REST filtering for time, minute, and time span fields
- We’ve introduced End of Interval Memory Optimization to automatically clear large, splayed tables
- We’ve updated the Service Gateway to support JSON responses over HTTP
- We’ve introduced customizable polling frequency in File Watcher
- We’ve updated the Stream Processor Kafka writer to support advanced configuration
- We’ve introduced a “Max Rows” option in views to limit values returned
- We’ve enabled the ability to query by selected columns in the UI Screen to reduce payload.
To find out more, visit our latest release notes then get started by exploring our free trial options.