I agree with this entirely. My grandma does not want to mess with a computer or the TV box. My grandpa did all of that before he died.
She uses a telephone primarily. She has a cell phone with big buttons and big text (but not a jitterbug) but she uses her landline more because of the bigger buttons, and that's what she's used to. She will never text anyone.
My other grandfather didn't have his voicemail set up for 2+ years on his phone. He was concerned about getting charged minutes for listening to voicemail, because he thought that your minutes were billed on 1) listening to the voicemail and 2) while the other person was talking. In effect, he thought he was going to get double billed for the voicemail.
So I think the biggest hurdle is education.
My parents, on the other hand, eventually got in to texting. They use a netbook (no tablets were out when they bought it) because the computer is mobile and they can sit outside on the patio with it, or at the table, etc. My mom's only complaint is that the computer is a bit small to spend hours researching something, like a vacation. I think that will be your argument against tablets.
It's probably worth looking at both demographics, because your 55+ today are going to be your 65+ in a few years.