I guess you can view it like alcohol. All alcohol is toxic, but it only becomes a problem if you have too much.

So (what I assume) GP is saying is that it wouldn't be an issue he didn't use HN so often.