Cray brings top supercomputer tech to businesses for a mere $500,000 Technology powering world's top supercomputers now in an entry-level package.

A Cray XC30-AC server rack.

Cray, the company that built the world's fastest supercomputer, is bringing its next generation of supercomputer technology to regular ol' business customers with systems starting at just $500,000.

The new XC30-AC systems announced today range in price from $500,000 to roughly $3 million, providing speeds of 22 to 176 teraflops. That's just a fraction of the speed of the aforementioned world's fastest supercomputer, the $60 million Titan, which clocks in at 17.59 petaflops. (A teraflop represents a thousand billion floating point operations per second, while a petaflop is a million billion operations per second.)

But in fact, the processors and interconnect used in XC30-AC is a step up from those used to build Titan. The technology Cray is selling to smaller customers today could someday be used to build supercomputers even faster than Titan.

Titan uses a mix of AMD Opteron and Nvidia processors for a total of 560,640 cores, and uses Cray's proprietary Gemini interconnect.

XC30-AC systems ship with Intel Xeon 5400 Series processors, and it's the first Intel-based supercomputer Cray is selling into smaller businesses, what it calls the "technical enterprise" market. (Cray's previous systems for this market used AMD processors.) Perhaps more importantly, XC30-AC uses Aries, an even faster interconnect than the one found on Titan.

In short, it's "the latest Intel architectures with the latest Cray interconnect" that will be installed in "future #1 and top 10 class systems," Cray VP of Marketing Barry Bolding told Ars. "It's like buying a car model where you're getting exactly the same engine you're getting in a top-of-the-line BMW. The only thing that's changed are some of the peripherals that make the system easier to fit into a data center and make it more affordable." Compared to systems like Titan, Cray says XC30-AC has "physically smaller compute cabinets with 16 vertical blades per cabinet."

Oil and gas firms or electronics companies performing complex simulations are among the potential customers for an XC30-AC supercomputer.

XC30-AC is a followup to the XC30 systems which are meant for larger customers and typically cost tens of millions of dollars. The "AC" refers to the fact that the smaller systems are air-cooled instead of water-cooled. The power requirements aren't as immense, and using air cooling makes it easier to install in a wider range of data centers.

XC30 systems scale up to 482 cabinets and 185,000 sockets with more than a million processor cores. The XC30-AC goes from one to eight cabinets, with each holding 16 blades of eight sockets each for 128 sockets in each cabinet. With an Intel 8-core Xeon processor in each socket, that adds up to 1,024 sockets and as many as 8,192 processor cores in an eight cabinet-system. A single cabinet with about 30TB of usable storage and 128 sockets would cost about $500,000, while eight cabinet systems with 140TB of usable storage and 1,024 sockets hit the $3 million range.

To begin with, the XC30-AC supports only Intel Xeon processors with Sandy Bridge architecture. Those will be updated to server-class Ivy Bridge chips later on. Nvidia GPUs and Intel Xeon Phi chips will become available as co-processors by the end of the year, Cray said.

While XC30-AC systems will be smaller than traditional supercomputers, the Cray Aries interconnect makes it incredibly fast, Bolding said. He noted that Ethernet interconnects generally aren't fast enough for the world's fastest supercomputers. InfiniBand has really taken off, being used in about half of the top 100 systems and two of the top 10. But the top five systems in the world all use custom, proprietary interconnects such as Cray's or IBM's.

 A look at the XC30 architecture.

Aries supports injection bandwidth of 10Gbps and 120 million get and put operations per second. "Injection bandwidth" is less than the full system's bandwidth. As a paper on the interconnect's architecture notes, the "global bandwidth of a full network exceeds the injection bandwidth—all of the traffic injected by a node can be routed to other groups."

Latency is another key factor, with Aries providing point to point latency of less than a microsecond, Bolding said. Moreover, latency remains strong when a cluster is going full blast. "When a system is very busy and sending messages from one end of the machine to another across a fully loaded network where everything's working at once, Cray's latencies are literally almost as good as they are in point to point. They go up to around two or three microseconds," Bolding said.

The speed allows memory to be shared across processors. "No matter how many nodes you have you can actually treat it as if it's a shared memory machine, every node can talk to every other node, directly into the memory of that compute node," Bolding said. "That's something that is very powerful for certain types of applications and programming models."

Aries also features more sophisticated network congestion algorithms than the previous generation, preventing messages from getting backed up during times of high usage.

As for software, XC30-AC comes with the SUSE-based Cray Linux Environment also used in Titan, allowing customers to run almost any Linux application, Bolding said. While some of the Cray's other systems are designed to run any form of Linux a customer wants, the XC30-AC comes with software optimized for the system. This allows it to be ready to go shortly after it comes out of the box, instead of requiring a week of setup.

Who will buy an entry-level supercomputer?

Cray isn't the financial success it once was, with its latest earnings announcement showing a year-over-year drop in quarterly revenue from $112.3 million to $79.5 million. The company also experienced a net loss of $7.6 million. Cray fared better in fiscal 2012, with full-year revenue of $421.1 million and net income of $161.2 million.

High-performance computing revenue is on the rise, with supercomputing products ($500,000 and up) leading the way according to IDC. HPC and supercomputing revenue is growing faster than the server market as a whole.

Cray is hoping to take its share of that revenue by selling both the smallest and largest supercomputer-class systems. While the XC30-AC was just announced today, it's been shipping for a few weeks. Early customers include an unnamed "Fortune 100" commercial electronics firm whose R&D department needs a powerful machine for simulations.

The oil and gas industry has a need for such machines to model oil fields. Biotechnology, engineering, and various manufacturing industries may provide interested customers as well, Cray says.

We've written about the trend of Amazon and other cloud services being used for supercomputing, with one-off jobs costing up to several thousand dollars an hour. Those are generally for customers that have only occasional need for a supercomputer, however. Many businesses would use a supercomputer often enough that owning one is more cost-efficient. Cray is betting a lot of Fortune 500 companies and universities that can't afford giant clusters costing tens of millions of dollars will be interested in systems like the XC30-AC.

"The complexity of problems that mid-range customers, technical enterprise customers face today are becoming so complex that they do need a tightly integrated supercomputer," Bolding said. "They can't always get away with a more conventional Ethernet cluster."