You need to communicate how unhappy you are. What would break your relationship is if you quit without ever voicing your discontent - it coming as a complete surprise is what will make it seem like you "screwed" him and the company over.

Communicating how unhappy you are, the reasons why, will make it so that the hope of a friendly exit is possible.

You can also communicate that you're looking to leave in the next X months, and that you're willing to invest the time to help find a replacement (and interviewing that individual, etc).