there's something to that
as a side note, i have a quibble with your definition of 'byte'. lots of computers have been word-oriented, with words being generally the smallest addressable unit
word sizes in word-addressed computers have included 12 bits (pdp-8), 16 bits (nova and tandem), 18 bits (pdp-7 and ga144), 20 bits (mup21), 32 bits (zuse z4, chifir, and maybe some ti dsps, not sure), 36 bits (pdp-10, ibm 704, ibm 709), 60 bits (cdc 6600), 64 bits (cray unless you count the parity bits), and so on
it was not normal practice to describe any of these words as a 'byte'; more commonly, you would divide the word into (possibly variable-size) 'bytes', each representing one character, even though you couldn't load those bytes individually from memory