Artificial intelligence (AI), is among the most interesting spaces in technology today. According to CapGemini, AI is creating jobs in a number of industries; nearly five out of six companies using AI say the technology is already adding jobs. In fact, while AI is expected to eliminate 1.8 million current roles, it’s actually creating 2.3 million, according to Gartner. It isn’t just jobs generating the buzz. Google search volume on the term has tripled in the last five years.

Moving beyond the hype however, most enterprises and companies are still light years away from fully taking advantage of AI. Why is this? What are the hurdles? What needs to change? This piece will argue that the biggest gap to fully enabling AI in business settings rests with a component of computing that most people rarely consider: memory.

Instant Insights to Actions

From a business value perspective, in order to capture the art of machine learning (ML), a lot has to happen in real time. To become insight driven or insight-centric, the goal is to get from data to analytics to action with a latency of only sub-seconds in the pipeline. Businesses must advance beyond traditional analytics perspectives, which separate data inputs and transactional systems from the analytics systems. Ultimately, if a firm wants to continue to grow and be competitive, it must turn to unifying analytical and transactional processing.

Traditional databases work well for business intelligence analytics, which are mostly historical or retrospective in nature. Businesses run reports, which are usually predefined questions, against their data. How many sales did we have over the past 30 days? What are the sources of our revenue? What zip code/ region/state/country generated the most revenue or the most profit?

However, the analytics world moves much faster than this. It’s no longer sufficient to derive business insights.

In contrast to the traditional computing paradigm of moving data to a separate database, doing some processing, and then saving it back to the data store, with In-Memory Computing, everything is put in the in-memory data grid and distributed across a horizontally scalable architecture. This is accomplished at low latency, because you’ve taken out the disk I/O that prevents workloads and mixed heterogeneous workloads from happening in real time.

The in-memory computing paradigm powers the unified paradigm. Instead of separating transactional databases from analytics databases, which leads to disk I/O and disk bottlenecks, the idea here is that by working with in-memory data stores we can easily eliminate bottlenecks and handle mixed workloads within the same architecture.

Machine Learning: Walking Before We Can Run

Machine learning is largely a subset of full-blown AI coverage, and begins by running algorithms against historical data, creating a hypothesis or set of hypotheses, and then continuing to analyze as new data is being created. Deep learning is a branch of ML, which relies on data sets to iteratively train many-layered neural networks. These trained neural networks are used to interpret the meaning of new data, with greater speed and accuracy.

For this to be effective, however, algorithms need access to cold, warm, and hot data (historical, recent, and real-time, respectively). To draw a parallel to enterprise architecture, cold data can be stored on disk, warm data in flash storage, and hot data in RAM. This multi-tiered approach, combined with software that can seamlessly connect these tiers, largely meets the needs of companies looking to incorporate ML into their operations.

When AI seeks information that’s buried in cold data (i.e. on a disk), this can create a bottleneck in organizations leveraging traditional architecture. 

Classic multi-tier architectures require a storage layer that has multiple “tables” (main store and delta stores) along with, multiple layers for hot, warm, cold, and archive/history data. These tiers are built on top of yet more tiers for durable storage and sit underneath an additional management and query tier. The In-Memory Data Grid tier can ingest and store multi-tera datasets, which eliminates most of the layers required in traditional table-based database paradigm.

The typical architecture referenced above:

Disk (cold data) → Flash (warm data) → RAM (hot data) can’t support the real-time applications and requests that AI requires.

AI requires the ability to draw information from all sorts of data, meaning the designation of “cold data,” should ultimately become obsolete.

Intel to The Rescue

The recent availability of Intel Optane storage class memory is an ideal way to solve the issue. Layering Optane between SSD and RAM is an ideal solution for fast data access at lower TCO.

Companies can then move their architectures from “Disk → SSD → RAM” to “SSD → Storage Class Memory (Optane) → RAM.”

This transition keeps more data closer to memory, making it more readily available for AI initiatives. It also offers the ability to perform significant write operations while concurrently reading files. In other words, these devices can ingest data and explain findings at the same time. This ability to analyze data at its point of creation is critical to companies looking to take advantage of all of their data, especially that which was most recently created.

By utilizing advanced off-heap persistence and customizable configurations of “hot data” per application, the grid can scale out into the multi-tera range at relatively low cost yet maintain the performance required per application.

In addition to reducing the number of tiers in both the general architecture, this approach successfully eliminates delta tables because the grid is the live operational store. Unlike traditional databases, the grid can handle massive workloads and processing tasks, ultimately pushing big-data store (e.g. Hadoop) asynchronous replication to the background to put the multi-petabytes in cold storage.

Creating a Transition Plan

This isn’t to say that every organization should rip and replace all of their hardware in an effort to prepare for the coming AI boom. Instead, companies can begin with ML initiatives with specific projects. With each project, the requirements for “cold, warm, and hot” data can be assessed and a corresponding data storage and management plan can be created. This second step will help companies identify how much of the data they consider to be “cold” actually carries more value than is believed.

The amount of data that needs to be reconsidered will help organizations decide which data should be stored in SSD solutions like Intel’s Optane drives.

A company that handles 100s of millions of transactions on a daily basis in environments characterized by high and spiky data volumes required a low latency, highly available, fully transactional solution with an optimized TCO. Spikey volumes, in trade reconciliation, can be caused by a variety of reasons and hard to predict. This company leveraged a multi-tiered data storage model where the ‘hot data’ is stored on RAM and historical data is stored on SSD to address the increasing data volumes and data spikes in a cost-effective manner that meets the demand for speed and service they offer their customers.

Ancillary Benefits

In addition to making AI achievable, this revised architecture delivers a variety of benefits for companies.

• Lower TCO: While making the investment to upgrade various systems will have immediate overhead, the long-term value of storage class memory can’t be understated. If a single AI initiative can generate hundreds of thousands of dollars in ROI, the aggregate outcome can pay for itself many times over. Additionally, SSD solutions are less expensive than RAM, offering simple, obvious cost savings.
• Lower RTO: Each year, some retailers spend a portion of their annual earnings calls discussing the cost of downtime on major shopping days like Black Friday or Cyber Monday. Tens of millions of dollars are lost each year when a company’s infrastructure fails to address the deluge of traffic. Recovery Time Objective, the ability to have a system go from “off” to “fully loaded and operational” can be reduced from hours to seconds through the redesigned architecture. It’s nearly impossible to ensure 100 percent uptime, but the ability to bounce back immediately can help to alleviate these issues.

Companies across a number of vertical markets will need to incorporate AI into their processes. In banking, Customer 360 initiatives won’t function without AI. The same is true of retailers looking to create omnichannel approaches that would connect customers and customer service initiatives. As the transportation industry continues to evolve and vehicles become self-driving, the need for effective AI and rapid access to data will only continue to increase. Any significant downtime across these verticals will not just be a headache for businesses, but could create potentially dangerous consequences.

Another example is predictive maintenance for industry 4.0, telco, and autonomous vehicles, where a prediction engine is first to predict when IoT equipment failure might occur, allowing planned maintenance activities to assure business continuity.

However, while AI boasts transformative abilities, there are hurdles that must be overcome to move beyond theoretical value and into tangible results. Enterprise architectures today are not yet equipped to meet the data intensive needs of true AI solutions. However, new storage class memory components like Intel Optane SSDs are closing the gap. As we’ve seen happen so many times when businesses are too slow to adopt new technologies, those companies that continue to ignore AI will fall behind the curve.