Partly because we don’t want to be locked into using Intel for both of DOE’s open science machines. In the lead up to the exascale or near exascale machines (2012-now) I don’t remember hearing much about AMD, so presumably Intel and IBM were the only vendors supplying general purpose CPUs.
Because Power 9's have on-chip NVLink ports.
This. CPUs are hungry.
Because all the Blue Gene/L/P/Q code is POWER.
Sort of, but Blue Gene tech was always an Argonne thing, Oak Ridge has been the one doing GPUs for awhile now. Unless something weird happens Argonne and Oak Ridge will always be running different types of computer since we don’t want to be locked into one specific vendor or hardware configuration.
Aren't all the new DOE machines GPU at this point? I understand A21 and NERSC9 both are, though the vendor isn't announced yet as far as I'm aware.