Wouldn't the food need to warm up to room temperature before bacteria would breed like crazy? If you defrost it from frozen to cold-but-defrosted, shouldn't you be ok, even if the surrounding air is room temperature?
Yes, that's my point. If you defrost in the fridge it's ok. If you defrost at room temperature then it's dangerous.
It's not extremely dangerous. Many people do it every day and don't die. But it's better not to.