A global memory shortage is forcing large data center operators to take extraordinary measures. Nikhil Cherian, senior director of Google Cloud Supply Chain Infrastructure, said that Google has begun to dismantle decommissioned servers and extract reusable DDR4 modules.
Image source: cloud.google.com
Google has developed special hardware adapters that allow you to connect previous generation memory (including DDR4) to new servers for artificial intelligence systems. To do this, the company returns decommissioned servers, disassembles them and removes reusable memory modules. Cherian said the AI industry has quickly moved from compute-bound to memory-bound. Currently, high-performance memory accounts for more than 75% of the component cost of a typical AI server.
Shortages have been accompanied by rapid price increases. According to TrendForce’s forecast, server memory prices will increase by 13% to 18% in the third quarter, and PC memory prices will increase by 18% to 23%. Goldman Sachs’ forecasts are close to these figures, with analysts expecting growth rates of about 18% and 17% respectively. In August, the spot prices of 8GB DDR4 and DDR5 modules reached US$142 and US$133 respectively.
Rising prices and memory shortages are forcing technology companies to look for ways to reduce the amount of memory required per unit of computing power. Google has solved this problem at both the hardware and software levels, optimizing the function library, model architecture and KV cache compression algorithm.
Google’s new AI accelerator also aims to reduce reliance on external memory. In particular, TPU8i, designed for inference, uses a multi-layer memory hierarchy: each accelerator is equipped with 288 GB HBM3E, and the server system uses DDR5 to perform host-level tasks. However, the DRAM shortage proved so severe that in some cases, Google decided to install DDR4 modules taken from decommissioned servers instead of DDR5.
The memory shortage therefore forces one of the largest operators of artificial intelligence infrastructure not only to optimize new accelerators and software, but also to organize internal links for the reuse of old server components.
If you find an error, select it with your mouse and press CTRL+ENTER.
