Handel became a subject of Britain in 1727 by an act of parliament[1]. The text of the act itself[2] clearly referred to him as "George Frederic Handel" with no umlauts. Considering he spent the rest of his life in Britain as a naturalised British citizen there's little reason on insisting to use the umlaut for his name.
[1] https://archives.parliament.uk/collections/getrecord/GB61_HL...
[2] https://digitalarchive.parliament.uk/HL/PO/PB/1/1726/13G1n3