Tech giants are assembling AI supercomputers of unprecedented scale. Microsoft, Meta, Amazon, and Elon Musk’s xAI are building clusters boasting 100,000 or more H100 GPUs, with xAI planning to double its Colossus system to 200,000 GPUs — including the upcoming H200 model. To put this in perspective, OpenAI reportedly trained GPT-4 using roughly 25,000 A100…