Maybe there are different types of downside; and different people have different experiences?

For example, if I take some sort of sleep aid (e.g. US OTC antihistamines, Ambien, or low-dose benzos) then after ~2 nights my subsequent ability to fall asleep without help is noticeably impaired. That’s already the start of a physiological change that, if not reversed, would continue to a dependency (albeit not necessarily an ‘addiction’.)

It’s a process of analogue change, not binary, and individuals’ pace of change will vary.