A compromise might be to hide navigation elements or margin notes until you hover / click on an element, sort of like how Wikipedia enables page previews on internal links.