How is the Pantheon and the Invalides any different from a cemetery? Do you think the mere act of visiting a burial site is immoral?
St. Paul's in London has tombs in it, and it charges a fee! It's a super normal practice.
How else could you manage the amount of people who would go?