by Przemek Tomczak
The onslaught of data continues. Predictions vary but some estimate the world will be generating 175 zettabytes annually by 2025, of which 90% would be from IoT alone. Drawing context and insights quickly is ever more important, and ever more challenging.
What is driving this demand is that organizations are deploying high-resolution sensors to improve the efficiency and effectiveness of their operations and products through analytics and automation.
Organizations are challenged with supporting analytics on higher data volumes cost-effectively. To address these challenges organizations had two options:
Now we have a new option for consideration by performance-demanding customers – Intel® Optane™ persistent memory (PMem). We partnered with Intel to test their persistent memory with our Streaming Analytics platform running mission-critical industrial workloads.
Use of Intel Optane persistent memory as a block-storage device with KX Streaming Analytics delivers 4× to 12× improved analytics performance over high performance NVMe storage – similar performance to DRAM for query workloads. For key data-processing workloads, we found DRAM requirements were significantly reduced. This lets organizations support more demanding analytic workloads on more data with less infrastructure.
Processed 1.6× more data per second than NVMe-only storage where data was read from PMem and written to NVMe storage
Organizations should consider Intel Optane persistent memory where there is a need to accelerate analytic performance beyond what is available with NVMe or SSD storage.
Please click on this link to code.kx.com for more details
KX Streaming Analytics has been at the forefront of helping organizations ingest, process, analyze, and deliver actionable insights on massive amounts of real-time and historical data. KX Streaming Analytics is an integrated platform that runs in the cloud and/or on-premises to enable organizations to reduce costs and improve productivity through anomaly detection, analytics, predictions, corrective action and automation based on both real-time and historical data. It provides full life-cycle data ingestion, processing, analytics, and data management enabling mission-critical operations, 24×7, with no downtime and no data loss – all of your data, in any environment, instantly.
Intel Optane persistent memory (PMem) redefines traditional architectures, offering a large and persistent memory tier at an affordable cost. With breakthrough performance levels in memory-intensive workloads, virtual machine density, and fast storage capacity, Intel Optane persistent memory accelerates IT transformation to support the demands of the data era.
PMem is an affordable alternative to expensive DRAM. It can provide huge capacity and accommodate demanding workloads and emerging tools such as in-memory databases.
PMem is propelling major infrastructure consolidation. The increase in memory size from Intel® Optane™ media is an opportunity to consolidate workloads. Concentrating them on fewer nodes reduces deployments and improves use of processors previously underutilized because of memory constraints.
PMem is the world’s smallest and fastest storage device sitting on a memory bus. Architects and developers are using it to innovate with restart and replication, and to break system bottlenecks.