Wait, so the property management company (which would be responsible for facilitating the mold remediation at the very least, or paying for it if they own the property or were liable for the issue in the first place) told you that mold was no big deal and your upstairs neighbors just need to caulk a little more around their bath tub (also the responsibility of the property owner) and that calmed you down?
Mold and fungi can certainly be harmful to humans. The interests of your landlords and property managers are not aligned with yours in this case.