It seems such a scan causing a cancer would be due to a very low probability event on a cellular level. From that point of view the above calculation looks reasonable. It is only after that when the additional macroscopic influences caused by a specific person's body/environment come into play that your point applies.
This was interesting so I started from that 0.05 % per 1 rem and did the inverse calculation. For 2500 rem this gives about 1-(1-0.0005)^2500 = 71 % probability of developing cancer. I also made a plot from 0 up to 10000 REM[2]. The incident rate seems to be pretty much approaching 100 % around 10k, so our results differ by about a factor of 4.
[1]https://en.m.wikipedia.org/wiki/Roentgen_equivalent_man#cite...
[2]https://www.wolframalpha.com/input/?i=1+-+(1+-+0.0005)%5Ex+f...
EDIT: The initially mentioned 2500 rem figure must be over a long enough time for the exposure not to cause more immediate issues than cancer.