KDB.AI 1.4 has just been released, unlocking new capabilities to streamline workflows, boost performance, and open new possibilities for integrating data. In this release, we aim to make KDB.AI even easier to use and more adaptable to the growing demands of diverse applications.
Let’s explore.
Database layer support
To improve data management, KDB.AI 1.4 introduces a new database layer above tables, allowing for better organization and management.
schema = [
{"name": "id", "type": "str"},
{"name": "vectors", "type": "float64s"}
]
indexes = [
{
"name": "vectorIndex", "type": "flat",
"params": {"dims": 384, "metric": "CS"},
"column": "vectors"
}
]
table = database.create_table("my_table", schema=schema, indexes=indexes)
Each database can support multiple tables, each with multiple indexes. This helps reduce redundancy and simplifies the management of large datasets in enterprise environments.
Multiple index support
For users with diverse datasets, KDB.AI 1.4 introduces enhanced multi-index support:
- You can now create and manage multiple indexes on a single table, allowing for more flexible and efficient querying strategies
- You can now execute searches across different indexes simultaneously, enabling more comprehensive and nuanced query results. This is especially beneficial for multimodal datasets, such as images and text. For example, users can search using both an image embedding and a text embedding and then re-rank results accordingly
# Multi-Index Search for both texts and images
results = table.search(
vectors={
"text_index_qFlat": query_vector,
"image_index_qFlat": query_vector
},
index_params={
"text_index_qFlat": {'weight': 0.5},
"image_index_qFlat": {'weight': 0.5}
},
n=2
)[0]
Additionally, indexes can have different dimensions, allowing users to experiment with multiple indices and re-rank results based on dimension. This can significantly enhance your ability to conduct complex searches for more accurate results.
With hybrid search, developers can explicitly define separate indexes: one dense for semantic similarity and another sparse for keyword search. This was possible in previous versions, but with KDB.AI 1.4, we have introduced greater control over indexes and more granular weighting for fine-tuned ranking based on the importance of different data aspects.
kdb+ integration
KDB.AI 1.4 also strengthens integration with kdb+
- Direct table access: Enables users to reference and query kdb+ tables directly from KDB.AI, allowing for seamless similarity searches without needing to move data between systems. This helps maintain data integrity while using KDB.AI ‘s advanced search capabilities
- TSS searches on kdb+ tables: Support for running Time Series Similarity (TSS) searches on kdb+ tables extends powerful time series analysis capabilities to your existing kdb+ data, enabling better decision-making in financial and IoT applications
- Index creation on kdb+ data: Create indexes on kdb+ table data within KDB.AI, optimizing vector searches without modifying the original tables. This feature enhances search performance without compromising your existing data structures
New q API
We’ve also introduced a fully documented public q API for KDB.AI server that allows q developers to leverage KDB.AI ‘s capabilities directly within their q environment. This provides a seamless integration path for organizations heavily invested in q, reducing friction and providing a consistent toolset for developing advanced applications.
Enhanced REST API
The REST API has also been improved for a better developer experience:
- Improved adherence to RESTful conventions makes the API more intuitive and easier to work with
- More consistent error handling and response codes improve debugging and error management in applications using the API
Version information
Finally, we have included the ability to retrieve information on the KDB.AI server version to aid in compatibility checking and troubleshooting. This helps manage deployments and ensures consistency across environments.
To learn more, check out our latest documentation on code.kx, or view our migration guide on GitHub.
