You may have been more sensitive to the side effects than most.
The side effect that most annoyed me was actually loss of appetite, which is basically accepted as 'normal' when taking stimulants. Anxiety was probably a bigger issue with Adderall, though. I don't recall ever talking to someone who felt zero side effects from any these medications.
Gut reactions and vague impressions will get us by, just like they always have