What's a drug? If sugar causes addiction, withdrawal, and it's unhealthy, then it definitely is a drug.
Caffeine is, by itself, a drug in a different sense. It causes a slight addiction and can cause slight withdrawal. It isn't all that unhealthy in itself, however, especially not in the amounts we find naturally in foods. It becomes more of an issue if one is taking caffeine pills and other such things. Eating unhealthily might be more unhealthy, honestly - even though overall, having an unhealthy diet isn't a drug.
Caffeine is, by itself, a drug in a different sense. It causes a slight addiction and can cause slight withdrawal."
This seems like a drug in the same sense, can you clarify?
Rather than making stuff up, here's dictionary definition of drug: a medicine or other substance which has a physiological effect when ingested or otherwise introduced into the body.
As a user of both sugar and broccoli I can anecdotally attest that they do not have the same physiological effect on my body.