Finance as a profession has not really embraced remote work (yet?), so I'm not sure why you think the fact that parts of the system are electronic matters. The workers are still collocated in big financial cities, which used to be London, but seems like it'll be Frankfurt now. As for why it's a banking city, well, it just feels like it. There's big buildings associated with banks everywhere, and it has that new building/office park financial sector feel. Plus there's literally a huge monument to the Euro.

I work in software like most of us, software being absolutely electronic, but my company hasn't really embraced remote work either, so hence I'm in NYC.

"Banking city" does not mean there's plenty of bank branches around :-)

Companies in the same industry tend to gravitate around each other so they can cluster talent, lobby governments and "build synergies" (sorry for the buzzword), among other reasons. The City of London is that kind of cluster.