I know it’s not a phone, but a few friends and I built this for the Newton MessagePad back in 1998-9 with exactly this vision.

Ref: http://npds.free.fr/