• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer

Electrical Engineering News and Products

Electronics Engineering Resources, Articles, Forums, Tear Down Videos and Technical Electronics How-To's

  • Products / Components
    • Analog ICs
    • Battery Power
    • Connectors
    • Microcontrollers
    • Power Electronics
    • Sensors
    • Test and Measurement
    • Wire / Cable
  • Applications
    • 5G
    • Automotive/Transportation
    • EV Engineering
    • Industrial
    • IoT
    • Medical
    • Telecommunications
    • Wearables
    • Wireless
  • Learn
    • eBooks / Handbooks
    • EE Training Days
    • Tutorials
    • Learning Center
    • Tech Toolboxes
    • Webinars & Digital Events
  • Resources
    • White Papers
    • Design Guide Library
    • Digital Issues
    • Engineering Diversity & Inclusion
    • LEAP Awards
    • Podcasts
    • DesignFast
  • Videos
    • EE Videos and Interviews
    • Teardown Videos
  • EE Forums
    • EDABoard.com
    • Electro-Tech-Online.com
  • Bill’s Blogs
  • Advertise
  • Subscribe

Troubled Times Ahead for Supercomputers

April 27, 2016 By Paul Pickering

In the world of High Performance Computing (HPC), supercomputers represent the peak of capability, with performance measured in petaFLOPs (1015 operations per second). They play a key role in climate research, drug research, oil and gas exploration, cryptanalysis, and nuclear weapons development. But after decades of steady improvement, changes are coming as old technologies start to run into fundamental problems.

When you’re talking about supercomputers, a good place to start is the TOP500 list. Published twice a year, it ranks the world’s fastest machines based on their performance on the Linpack benchmark, which solves a dense system of linear equations using double precision (64 bit) arithmetic.

Looking down the list, you soon run into some numbers that boggle the mind. The Tianhe-2 (Milky Way-2), a system deployed at the National Supercomputer Center in Guangzho, China, is the number one system as of November 2015, a position it’s held since 2013. Running Linpack, it clocks in at 33.86 x 1015 floating point operations per second (33.86 PFLOPS).

Supercomputers achieve their performance through parallel use of off-the-shelf components.Tianhe-2 currently has 16,000 nodes, each with two Intel Xeon Ivy Bridge processors, and three Xeon Phi coprocessors for a combined total of 3.12 million computing cores. Not just reliant on Intel, it also features a number of Chinese-developed components, including the TH Express-2 interconnect network, the front-end processors, the OS (Kylin Linux), and software tools.

Most machines on the TOP500 list make extensive use of graphics processing units (GPUs), or coprocessors, to handle computationally-intensive tasks. Nvidia chips appear in 66 of the TOP500, with another 27 using Xeon Phi.

In the short term, everything looks good. In Q1 2016, Intel is scheduled to introduce Knights Landing, its next-generation Xeon Phi with up to 72 cores and three teraflops of performance, and Nvidia’s Pascal GPU will cram 17 billion transistors into a single device using a 16 nm FinFET process from TSMC.

Over the last few years though progress towards the next big benchmark, exascale computing (1018 floating point operations per second), has gradually slowed.
The reasons are varied, and the solutions will demand a rethinking of supercomputer architectures. In 2008, Peter Kogge from Notre Dame University, together with a team of computer scientists and engineers, put together an influential study for DARPA that identified several fundamental problems.

Power Consumption
Power consumption is one: Tianhe-2 uses 17.8 MW running Linpack. That’s enough to power over 14,000 average US homes, plus another 6 MW for the cooling system. To scale up to the higher level using an expanded version of the same architecture would require around 540 MW, or 100 percent of the output of a high-efficiency gas-fired power plant, plus extra power for cooling. The power management problems inherent in scaling up performance even have their own term – the so-called “power wall.”

One option to reduce power consumption is to cut the operating voltage of transistors, which has been around one volt for the last ten years, to a few millivolts. Researchers are looking at several options, including FETs that make use of quantum tunnelling; switches based around MEMs technology; and nanophotonics, which substitutes light for electrons.
Three-dimensional memory, which allows structures to be placed closer to CPUs, will likely be part of the solution, too. In July 2015, Intel and Micron Technology announced 3D Xpoint, a non-volatile memory with a packing density claimed to be up to ten times greater than DRAM.

The Von Neumann Bottleneck
Other issues stem from the von Neumann architecture itself, which has been the dominant model for stored program computers since John Von Neumann first described it in 1945. The basic problem is an imbalance between the number of cycles needed to perform a mathematical operation and the much larger number of cycles needed to fetch the data for that operation from memory. Following Moore’s Law, shrinking transistor sizes has resulted in ever-increasing processor speeds, while the time needed to access memory has improved at a slower pace. The ratio can now be as much as 1:100, so a supercomputer is forced to spend the vast majority of its time moving data to and from memory.
This is spurring the adoption of data-tiering hardware and software that try to maximize CPU effectiveness by storing frequently accessed data in high-speed tiers such as solid-state drives, while storing other data in slower disk-based storage. Cray’s DataWarp and DDN’s Infinite Memory Engine are examples of data-tiering products.
There are several other lines of research aimed at eliminating the von Neumann bottleneck, including: Processing-in-Memory (PIM) technology; quantum computing; and even neuromorphic computing inspired by human brain structure.

The Changing Nature of Applications
The original applications for supercomputers involved solving systems of linear equations for fields such as nuclear weapons development, and performance on the Linpack benchmark has provided a close correlation to performance on real-world applications.

Over time, however, applications have diversified, and data-intensive supercomputer applications have become increasingly important. These applications have a low computation-to-data-access ratio, and Linpack isn’t as useful a metric.

Graph algorithms are a key part of many analytics workloads, leading to the development of the Graph 500 benchmark and the corresponding Graph500 list, which provides an interesting contrast to the TOP500.

There are other ways to rank supercomputers too. In an age of concern about global warming, the Green 500 list ranks the world’s most powerful machines in terms of floating point operations per watt; not surprisingly, it’s very different from the TOP500. Piz Daint, the #7 machine on the TOP500, barely breaks the Green 500 top 20 at #20, but the other top machines – Tianhe-2 (70), Titan (53), Sequoia (55), K computer (214), Mira (56) – are much further down the list. Conversely, the #1 Green 500 machine achieves over 7 Gflops/W, but its Linpack performance is only good enough for #136 on the TOP 500 list.

The Road Ahead
Although it might be possible to produce an exascale machine today with a brute force approach, a practical computer might be achievable by 2022 or even later. The US Department of Energy’s Exascale Initiative envisions a machine that consumes just 20 MW using one million 1-teraflop processors and 64 petabytes of memory.

Among the tasks identified will be the development of new programming paradigms to deal with massive data sets, and the detection of and recovery from soft errors. The resulting machine is likely to look quite different from the machines of the last twenty years.

You Might Also Like

Filed Under: Uncategorized

Primary Sidebar

EE Engineering Training Days

engineering

Featured Contributions

GaN reliability milestones break through the silicon ceiling

From extreme to mainstream: how industrial connectors are evolving to meet today’s harsh demands

The case for vehicle 48 V power systems

Fire prevention through the Internet

Beyond the drivetrain: sensor innovation in automotive

More Featured Contributions

EE Tech Toolbox

“ee
Tech Toolbox: Internet of Things
Explore practical strategies for minimizing attack surfaces, managing memory efficiently, and securing firmware. Download now to ensure your IoT implementations remain secure, efficient, and future-ready.

EE Learning Center

EE Learning Center
“ee
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, tools and strategies for EE professionals.
“bills

R&D World Podcasts

R&D 100 Episode 10
See More >

Sponsored Content

Advanced Embedded Systems Debug with Jitter and Real-Time Eye Analysis

Connectors Enabling the Evolution of AR/VR/MR Devices

Award-Winning Thermal Management for 5G Designs

Making Rugged and Reliable Connections

Omron’s systematic approach to a better PCB connector

Looking for an Excellent Resource on RF & Microwave Power Measurements? Read This eBook

More Sponsored Content >>

RSS Current EDABoard.com discussions

  • How to best test Electrolytic capacitors for premature failure reduced life?
  • TMP117 > Can not read temperature on LCD
  • Disappearing clock
  • 12VAC to 12VDC 5A on 250ft 12AWG
  • MTEE component in the ADS (Advanced Design System) software

RSS Current Electro-Tech-Online.com Discussions

  • An Update On Tarrifs
  • Is there a Power Bank that can also be charged with electricity?
  • Electronic board faulty?!?
  • Question i-nears headphones magnetic drivers
  • LED circuit for 1/6 scale diorama
Search Millions of Parts from Thousands of Suppliers.

Search Now!
design fast globle

Footer

EE World Online

EE WORLD ONLINE NETWORK

  • 5G Technology World
  • Analog IC Tips
  • Battery Power Tips
  • Connector Tips
  • DesignFast
  • EDABoard Forums
  • Electro-Tech-Online Forums
  • Engineer's Garage
  • EV Engineering
  • Microcontroller Tips
  • Power Electronic Tips
  • Sensor Tips
  • Test and Measurement Tips

EE WORLD ONLINE

  • Subscribe to our newsletter
  • Teardown Videos
  • Advertise with us
  • Contact us
  • About Us

Copyright © 2025 · WTWH Media LLC and its licensors. All rights reserved.
The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media.

Privacy Policy