AWS to use Habana Labs AI Processors
3 December, 2020
"The new instances will leverage up to 8 Gaudi accelerators and deliver up to 40% better price performance than GPU-based EC2 instances for training deep learning models"
Intel achieved a major brealthroug this week when Amazon Web Services (AWS) announced the installation new computers powered by Gaudi accelerators from Habana Labs, an Intel company, to run part of its Elastic Compute Cloud (EC2) services. “The new instances will leverage up to 8 Gaudi accelerators and deliver up to 40% better price performance than GPU-based EC2 instances for training deep learning models,” said AWS CEO Andy Jassy.
Based in Caesarea, Israel, Habana labs has developed dedicated chips for Deep Learning Training and Inference. In December 2019, it was acquired by Intel for approximately $2 billion. The acquisition was part of Intels’ shift from the CPU-only domain to the XPU – a mix of architectures across CPUs, GPUs, FPGAs and more. The current AWS reflects an important move, since till now AWS was dedicated to NVIDIA-based GPU architecture.
Gaudi has been designed from the ground up for accelerating DL training workloads. Its heterogeneous architecture comprises a cluster of fully programmable Tensor Processing Cores (TPC) along with its associated development tools and libraries, and a configurable Matrix Math engine.
An 8-card Gaudi EC2 instance can process about 12,000 images-per-second training the ResNet-50 model on TensorFlow. Each Gaudi processor integrates 32GB of HBM2 memory and features RoCE (RDMA over Converged Ethernet) on-chip integration used for inter-processor connectivity inside the server.
Availability of Gaudi-based EC2 Instances is targeted to the first half of 2021. Software for Gaudi accelerators includes a SDK and other tools that simplify building and training models. The SDK will be natively integrated with common ML frameworks like TensorFlow and PyTorch, and provide the ability to easily port existing training models from using GPUs to Gaudi accelerators. Habana will build on Gaudi’s current efficiencies with its next generation TSMC 7nm for its next processor, Gaudi2.
Posted in: News