Actually, the PDP-10 didn't have any byte size at all, it was a word-addressed machine. (An early attempt to implement C on this machine came a cropper because of this.) It did have a Load Byte and a Store Byte instruction, which allowed you to select the byte size. Common formats were Sixbit (self-explanatory), ASCII (5 7-bit bytes and an unused bit), and (more rarely, I think), 9-bit bytes.
My first machines were the IBM 7044 (36-bit word) and the PDP-8 (12-bit word), and I must admit to a certain nostalgia for that style of machine (as well as the fact that a 36-bit word gives you some extra floating-point precision), but as others have pointed out, there are good reasons for power-of-2 byte and word sizes.