Technically your personal ID is not public knowledge. The first eight digits are, which contain your birth date. The last four digits are always obfuscated. In reality, it is common to provide your personnummer verbally in non-private settings so if someone within earshot had some interest in remembering your ID they could.

Edit: The above might not be entirely accurate, check out the thread below. Some of these lookup sites may reveal the personnummer on registration and possibly payment. I've never tried this so can't confirm for sure, however.