$560 out of $200k is 0.3%. $4500 out of $200k is 2.3%. Either way, yeah it's a small fraction.
https://www.owenscorning.com/en-us/insulation/pink-next-gen-...
Sure it's not good for you, so using PPE is certainly warranted, but it's on a whole different level of badness compared to asbestos.
[1] Per wikipedia, there are some varieties of mineral wool used for high temperatures (think insulating industrial furnaces and such) which are carcinogens.