For a Mac GUI app, try Hexfiend: https://hexfiend.com. (App store link: https://apps.apple.com/us/app/hex-fiend/id1342896380?mt=12)
Maybe scrolling is non trivial, but there is definitely a lot of room for optimizations. You could store large chunks in memory and populate them asynchronously with smaller chunks from file for example.
edit: I completely missed the point with the size, comex's answer is way better