Only seven schools in the UK are actually 'public schools'. It's a precise legal status. So 'private school' and 'public school' does not mean the same thing.

Originally they were the only schools that educated members of the general public, rather than people just from the local village or church or things like that, so the name does make sense.