Only being able to take calls on speakerphone, not having access to the long tail of apps you might need on occasion, and not being able to effectively input text into the apps you do have are the biggest problems for me.
Its alright for the kids and old people thing they have now where you really want a life alert/GPS tracker combo and calling/texting is a nice bonus but the form factor isn't viable for solo use.