The cloud has an address: the resilience of AI data centres

Concrete and copper

In the Financial Times, Chris Miller, the author of Chip War, describes a brutal reality: data centres have become targets of war. This year, strikes in Ukraine, in Russia and in the Gulf have exposed the physical vulnerability of the infrastructure that carries our digital economies.

Behind our apps, our payments and our artificial intelligence there are buildings, cables, transformers and servers. Concrete and copper: a very real infrastructure whose destruction can interrupt essential services. The cloud has an address, and that address can become a target.

The equation works against the defenders. According to the article, a drone costing a few thousand dollars can cause millions of dollars of disruption, and one operator reported that data stored in one of its Gulf data centres was permanently destroyed by a strike.

Concentration concentrates the risk too

The race for gigantic compute centres concentrates the risks too: the bigger and costlier a campus, the heavier its loss. And the buildings of one site sometimes share the same dependencies: one power supply, one cooling system, the same telecommunication links.

Every new megacampus therefore deserves a question that matches the investment: what happens if this site becomes unavailable? Which services stop, for how long, and which data is lost?

Broadening the idea of sovereignty

Owning the servers and keeping the data on your own territory brings control. Keeping them available during a crisis also takes the ability to carry on operating elsewhere.

The article shows it: neither state ownership of data centres nor data localisation requirements were enough to protect them from strikes. Digital sovereignty also depends on the infrastructure’s ability to withstand a physical attack, and to recover from it.

Distribute, diversify, recover

This reading opens a reflection on the place of distributed infrastructure, geographic diversification and the ability to recover. The continuity of compute is designed from the start:

• truly independent sites, which share neither their power, nor their cooling, nor their links;
• inference services ready to switch to another site;
• for large training runs, regular checkpoints and a planned restart on another site;
• isolated backups, replication and regular restoration and failover drills.

The questions to ask

• What happens if this site becomes unavailable?
• Do the buildings share power, cooling or links?
• Which services must switch to another site, and how quickly?
• Where are the backups, and are they isolated from the main site?
• When were restoration and failover last tested?
• Has the cost of resilience been built into the investment decision?

A criterion as decisive as the price of electricity

Resilience could become a criterion as decisive as computing power or the price of electricity. We are building the future around AI. Have we thought enough about what could switch it off?