agreed, although some HE algorithms with more limited functionality (such as vector ops), can do a bit better. There has also been some work on GPU enabled HE.