The RAM shortage has already affected the largest providers of AI infrastructure: Google has begun putting decommissioned servers back into service in order to extract available DDR4 modules for new systems, according to industry sources.
Nikhil Cherian, senior director of supply chain infrastructure at Google, said the AI market has quickly shifted from a shortage of computing power to a shortage of memory. Today, high-performance memory can account for about 75% of the cost of AI server components. To reduce its dependence on supplies, Google is tearing down decommissioned machines and setting up its own component recycling system.
The company has even developed special adapters that allow you to connect previous-generation memory to the new AI servers. It is said that Google specifically returns decommissioned servers to remove available DDR4 modules from them. At the same time, engineers are optimizing the function library, model architecture and KV cache compression in an attempt to reduce the amount of memory required to perform operations. This year, Google also launched two versions of TPU: TPU8t for model training and TPU8i focused on reasoning.
