The main difference is that the magnitude is greater in people with ADHD than those without. A reduction in hyperactivity, for example, is not really apparent in people that are not overtly hyperactive to begin with.
Beside, ADHD is kind of a nebulous label for a lot of symptoms. Having ADHD vs. not having ADHD is not exactly binary.