I live in California, I have never noticed them.
I went to school in the LA Inland Empire area - there was a quake I felt last year there during class. It was really close but I'm pretty sure it was just short of 4.0.
We have hundreds (thousands?) of quakes per year, but the likelihood your average resident feels a quake in any given year? I think it's low.