You have a very good point (so I'm not sure why you were downvoted).

I currently read Japanese at an upper-intermediate level. For example, I can read most of Harry Potter in Japanese an only look up a word every page or two.

The biggest problem with switching Japanese to "all gana" is the homophones. Many similar words sound the same but have different nuanced meanings such as hear and listen -- 聞く and 聴く -- both pronounced "kiku". This happens so often, that I think dropping kanji would be a disaster.

An additional benefit to the kanji is that when you get to know enough of them, you can begin to guess the meaning of new words, similar to learning Latin roots and morphemes like "un-" or "re-".