The exponential increase in the use of connected real-time sensors to surface streaming data in the age of the Internet of Things presents significant challenges and opportunities for the emerging field of streaming analytics. Detection of anomalies in streaming data, in particular, has becoming an increasingly important application across a large number of industries for critical use cases – ranging from preventative maintenance to fraud prevention, fault detection, and systems monitoring. But the real-time nature of streaming data has presented challenges for applying classic AI and machine learning techniques to date.
Neuroscience and machine intelligence researchers at Numenta Inc. have demonstrated how a novel anomaly detection algorithm, based on their theory of how the brain works, can tackle the problem with a technique that meets the requirements of streaming data by processing data in real-time and offering continuous, online detection without supervision – while simultaneously making predictions. The technique is based on an online sequence memory algorithm called Hierarchical Temporal Memory (HTM).
Numenta researchers have described the technique in a new peer-reviewed paper, “Unsupervised real-time anomaly detection for streaming data”,* published in a special issue of Neurocomputing.
In the new paper, the researchers also present the results of using the Numenta Anomaly Benchmark (NAB), a benchmark containing real-world data streams with labeled anomalies. NAB, an open-source benchmark and tool designed to help data researchers evaluate the effectiveness of algorithms for anomaly detection in streaming, real-time applications, was first presented in 2015 during the IEEE Conference on Machine Learning and Applications. NAB provides a first-of-its-kind controlled open-source environment for testing a wide range of anomaly detection algorithms on streaming data. Numenta offers the open standard benchmark for the research community to use, add to, and even draw inspiration from for new, innovative techniques.
“While many anomaly detection approaches exist for time-series data, the majority of methods are limited and apply statistical techniques that are computationally lightweight for streaming analytics. The versatile properties of HTM, which are patterned after the principles of how the brain works, make it well suited for streaming anomaly detection,” said Numenta Research VP Subutai Ahmad.
“We are bridging the gap between neuroscience and AI by using brain function as a guide to solving machine learning problems and designing more intelligent systems,” added Ahmad.
The release of the latest technical paper in Neurocomputing, which Ahmad co-authored with Numenta researchers Alexander Lavin, Scott Purdy and Zuha Agha, is in keeping with Numenta’s open research philosophy. Numenta researchers’ previously published peer-reviewed works published in the journals Frontiers and Neural Computation, among others.
