How illiterate people use their mobile phones (2012) | Hacker News Reader

How illiterate people use their mobile phones (2012)

researchgate.net

155 pointskvee184 comments

My PhD thesis focused on voice interface design for people with limited literacy skills. One of the most surprising discoveries for me was that menus and even lists of items aren’t natural concepts that exist outside of a context of literacy. Even for voice interfaces, a touch tone menu (“for X press 1, for Y press 2...”) was a lot harder to navigate than an equivalent voice-based menu (“would you like X, Y, or Z?”). As a side project during my final year of my thesis research, I wrote an iPhone app that unexpectedly propelled me into the world of entrepreneurship, so I ended up pursuing the startup life after I graduated. But this space is still fascinating to me.

Speech Interfaces for Information Access By Low Literate Users I presume?

Is it online somewhere?

I would also be interested to read your thesis, if you don't mind linking to it :)

edit: also, pop looks very nice! congrats!

https://www.cs.cmu.edu/~jsherwan/JS-thesis.pdf

Yes, that’s it! An easier and shorter read may be the final paper I published in that domain, which talks more about the difference between literate and low/non-literate users:

http://www.cs.cmu.edu/~jsherwan/pubs/orality-hcid-itid09.pdf

Maybe that explains why Alexa is popular? I never really understood it since of it's limited capacity. I guess I don't see the appeal because I'm a programmer and used to high capacity, low user friendly tools like terminal UI/UXs.

Yes, and I think what’s interesting is to consider is that the ability to use a terminal is a very recent (and from a certain perspective, strange) skill, considered from the perspective of the kind of hardware we have for information access, storage and transmission.

What’s far more “normal” is storytelling (with heroes and villains), rhythm & rhyme, and lots of repetition. But even simple conversational interfaces are way more normal (and fit our mental hardware well) than terminal-y interactions.

You're confusing old with normal and your conclusion attempts to "appeal to nature" (a well known error of logic): https://en.wikipedia.org/wiki/Appeal_to_nature

In context of user interface, "Natural" = "Familiar". So, very low barrier to extract value, less chance of making mistakes.

It's the best kitchen timer in the world, no matter how many dishes you're cooking.

About 15 years ago when my grandfather was in the later years of his life we bought him a DVD player. He was an educated man, but he was never interested in computers or technology. Despite it being the mid-2000s he'd managed to completely ignore almost everything about computers until then.

When I was trying to teach him to use it, I was amazed to find that even the simple (in my mind) concept that pressing the "up" button the remote would move the menu selection up on the screen, and that to play the DVD you needed to select the "Play movie" option and choose "enter" was completely foreign to him.

I think as "digital natives" with both long term exposure and a real interest in technology, it's very easy to forget how many layers of implicit knowledge our systems are built on.

>I think as "digital natives" with both long term exposure and a real interest in technology, it's very easy to forget how many layers of implicit knowledge our systems are built on.

Agreed, see replies to my comment on the linked post when trying to explain hardware usage from the perspective of older folks who aren't used to it, lots of replies along the lines of "they can this through Settings->General->blah"

https://news.ycombinator.com/item?id=23598685

As a software engineer that spent a great deal of his life embedded in UI philosophy, I'm still so lazy when it comes to learning other systems. My cell phone for instance - the iPhone, there are still things that my girlfriend shows me how to do on it that I had no idea about. I just can't be bothered to learn most technology beyond what I need to use it. I've got too much else to spend time on. I'm sure I don't know half of what my TV does, I use Alexa to turn the lights on and off, timers and alarms and that's about it.

Most of my devices are used almost exclusively for Teams meetings, communications or actual software development. When I'm raking through other software, I'm often exposed to UI concepts that make zero sense to me, yet they seem intuitive to many others that are beyond my comprehension. I wonder if this is because I'm getting old or if it's because people being forced to learn bizarre UI concepts and just accept it and go along with it.

It's weird to me that people just seem to accept complicated systems and I'm still stuck on the "This is too complicated to be useful to anyone. We're supposed to have computers working for us by now, not us working for our computers!" model.

I switch between Windows 10, Windows 11, MacOS Monterey and Pop!_OS interchangeably, easy. My main OS is Linux and mostly in the terminal. I can't comprehend that you cannot "figure out" different UI's.

Are you familiar with spoon theory?

https://www.healthline.com/health/spoon-theory-chronic-illne...

I have from 25-50 or maybe more things I need to get done in my normal everyday life - most of which either improve my life in some significant way, or improves someone else's in a way that pays my bills. I run out of spoons about 15 before I need to learn a new something that will add minor improvement to my life in some way. This includes things like holding down a key longer gives you the alternative characters for it, or a new operating system or device, or all the little features on my TV or the many thousands of ways Alexa can be harnessed to do some really cool things.

On a day where I have nothing to do and I feel particularly inclined, I might learn one of those things - but chances are I'll spend it doing something else that brings me joy. Cooking, spending time with loved ones, going for a ride on my bike or a swim. These are the choices and trade offs I make. Admittedly, that hampers my ability to use certain devices because I don't understand them completely, but it gives me time to enjoy my life in other ways.

I didn't know about the spoon theory, my comment was a bit snarky I guess? (My apologies)

I can't work with two screens, there have it.

I didn't touch MacOS for a long time, but I'm able (and I think you can too) to get manual, guide, documentation in seconds and adapt to it. I'm sure you can do the same thing as me.

I must say that I'm all about efficiency and administration outside computers. (Do you have ever thought about gaming? To use the right skill slot, to gain maximum velocity at that given level?) I apply "gaming skills" to my life.

I'm going to make a blog about it.

This is really interesting. It reminds me of a conversation I had recently with a friend of mine who struggles a lot with spelling. His first language is English and even so sometimes it is hard to decipher what he means when he messages me. Recently he asked me how to spell something and I said, "You spell it exactly how it sounds," and he told me that doesn't help him and he didn't understand what that meant. So I spelled the word out loud and sounded it out as I went. At the end he told me that his brain doesn't work that way- that it's nearly impossible for him to hold the sounds in his head while he tries to spell. I was really quite awestruck by that distinct differences in our brains.

I can’t blame him, English has a pretty irregular spelling.

English mostly isn’t.

"i BeFoRe E eXcEpt..."

Ugh.

"You spell it exactly how it sounds" such a bullshit advice when the language in the context is English, which one-to-one letter-phoneme correspondence is hilariously bad.

This reminds me of a coworker who recently immigrated from Brazil asking about the pronunciations of some English words and being astonished that we don't just use accents to differentiate it. Like, just use accents. No more ambiguity and problem solved.

We'd have to cover every word with accents. We'd have to have accents for "letter has no effect on sound," "these three letters actually represent this other letter," and "pretend you're French when you say this, but French with a severe head injury."

There was some English guy that tried to create an English alphabet where each letter would correspond to exactly one sound, so around 44 letters in total. Kinda like Serbian Cyrillic.

I can't remember his name or the name of his project though.

That's doomed from the start. There's no way English speakers could deal with a written language with 19 different vowels.

And which dialect!?

Shavian? https://en.wikipedia.org/wiki/Shavian_alphabet

Yeah, that's the one.

Not heard of this particular script before (although I see the answer was already posted), but it seems a bit redundant to have created an incompatible system when shorthand was in widespread use in the UK at the time, albeit mostly only used in a secretarial context.

> We'd have to cover every word with accents.

Seems to work fine for Vietnamese.

If every word has accents, then don't all words have no accents? Solved!

Have you ever tried learning Vietnamese as a second language? Shit's hard!

But not because of the script.

Based on my very phonetically written native tounge, I think I could map most of the English language down to Hungarian letters (there are some exceptions in both directions)

Just for note, it would look something like this:

Bézd on máj veri fonetikalli vrittn nétív táng, áj (sz)ink áj kud mep…

(Sz is a single letter in hungarian in this context)

> French with a severe head injury

There's a word for this: Quebecois

I would way rather standardize the sounds like how Japanese does it (mostly) than have accents. I've never liked accents. I'm considering learning Ido over Esperanto because of this very issue.

> I would way rather standardize the sounds

This has been tried.

https://luminusdadon.wordpress.com/2006/05/04/how-english-be...

Zis mad me smil!

If you consider each letter+accent combination as distinct, it is the same as having standardized pronunciation for each symbol?

Most people don’t know this but a lot of Japanese words have pitch accents that you need a dictionary to learn if you don’t learn the word audibly and the accents are not in the English/Japanese dictionaries, only in specialized Japanese/Japanese dictionaries. When Chinese speakers learn Japanese they do well at learning it because it’s similar to Chinese pitch.

Esperanto doesn't have accents that emphasise stress the same way other langauges do. The little markers over the characters indicate that it's a completely different letter with a completely different vowel/consonant sound (and they have their own entries in the alphabet).

For example, "Co" would be pronounced like "Tso", whereas "Ĉo" would be pronounced like "Cho".

Pronunciation is completely standardised, with stress (almost?) always placed on the penultimate syllable.

I've been to Italy, it seems hand gestures are also an excellent language facilitator.

That's a terrible idea in English. It's fine in languages with more consistent accents, but in English I can do that for my accent, and then americans can't read it at all. It doesn't even work within Britain

Except in Portuguese some of the accents can only be used in the strong syllable, so there's still ambiguity in how to pronounce accentless vowels in some words, mainly due to regionalisms.

Right? Though the march of enlightenment is eternal, suffrage is not bourne but rather embraced. Through bright corridors stream draughts of fairies, otherworldly.

Phonetics lies, asunder.

What is that quote from? Google comes up blank.

I'm guessing that was not a quote at all, but rather a response to "You spell it exactly how it sounds"

I found the AI that was trained on James Joyce!

Well, that's the point. You tell someone to spell it exactly how it sounds if it is one of the words that is spelled phonetically.

Which might still depend on your accent!

It's still not horrible advice if the goal is to just communicate some idea that he would otherwise not know how to type at all.

"i pushed da bisikel off da ruf"

Is difficult to read, but not impossible to understand.

English is really good at "graceful degradation".

I love the euphemism! But, compared to what? Spanish degradation is arguably much more graceful.

I would assume Japanese or other pictogram based languages would score very low here, but I am not sure how you would measure it objectively.

It's pretty common for speakers of Japanese and Chinese to substitute simpler characters with a similar phonetic value when they can't remember how to write the full character. In such a situation (particularly with handwriting) one can also just invent a character on the spot, along the same pattern most characters were invented, with a meaning + sound indication combination. In context, it usually works but reads awfully much like the English example above.

Interesting I was aware of the first though it seems more limited. However, I agree the second is a much closer equivalent.

Japanese has a couple syllabary to fall back on. If you don't know the Kanji for a word you can use the hiragana/katakana version which is pretty much 1:1 with how it sounds. If you use too much hiragana can become a bit unpleasant to read though.

What’s the Chinese equivalent? As I understand they have multiple distinct spoken languages sharing the same writing system.

I don't know any Chinese really so I can't comment too much. Taiwan has a syallbary called Bopomofo and PRC has Pinyin but I think those are only really ever used for typing/teaching. Parts of Chinese characters do have a phonetic component typically and from what I've heard this maps better onto Chinese than Japanese where possible sound could have very little or nothing to do with the phonetic component. So you could maybe write another character with that component maybe?

> they have multiple distinct spoken languages sharing the same writing system

It's... complicated. If you sort of squint the writing system is maybe the same in the sense that across different Chinese varieties cognates are almost always written the same way (e.g. Cantonese and Mandarin). But a lot of non-prestige Chinese varieties don't really have a standardized writing systems (although a very interesting phenomenon is that speakers can generally tell you when a given word has an associated character(s) they could write down for you and when they can't think of one). So usually when a writer of a non-prestige Chinese variety writes she/he is writing in Mandarin, not in the variety itself.

Nonetheless, there have e.g. been historical Chinese novels written in non-prestige varieties (e.g. in Min or Wu) so in that sense yes the same writing system can and has been used, but different authors might use different characters depending on how they understand the relationship of certain words to the prestige variety.

Even the different languages part is complicated. https://news.ycombinator.com/item?id=16844074

As for what phonetic "spelling" out happens in Chinese, usually a loan character is used (e.g. if live transcribing something where the transcriber hears some unfamiliar words, the usual solution is to write down characters with the same sound and mention that this is purely a sound loan in parentheses).

I'm not sure it's the structure of the language itself as much as it's how error-tolerant fluent speakers are.

Anyone under the age of about 60 seems to be completely used to a wide array of (sometimes bizarre) accents, Chinglish grammatical constructs and semi-illiterate spelling.

I think humans in general are good at error-correction of language, so I’m not sure English is any special in this regard.

https://www.bbc.com/pidgin/tori-60318986

My all-time favourite BBC Pidgin article is this one: https://www.bbc.com/pidgin/tori-41171196

Holy crap that was brilliant.

trying to teach phonics to a kiddo ...

letter E, called "eee", makes the sound "ae"

letter I, called "aai", makes the sound "ee"

letter A, called "aae", makes the sound "uh"

super confusing for some kids

and in my non-english-native schooling we were never ever taught "vowel sounds" in our English classes - we only learnt vowels as letters. beyond A for Apple and introducing common words like Car and House and Tree ... the curriculum never bothered about teaching how to speak English

I vividly remember learning the short and long vowel sounds in elementary school. With the marks over the vowels explained and used in lessons.

Things like băss and bāss. Obviously those marks can be confusing if they're substituted in for words where the spelled vowels are different than the pronounced vowels (like yo̅o̅ for "ewe"). I guess those were the advanced words we'd learn after the basics were taught. But first, the "Evil E" :)

This is why I'm grateful for my native language, Haitian Creole. Our alphabet has signs - formed with Latin characters - and each is mapped 1 to 1 to sounds. You write words like they sound, and you pronounce them like it is written. Most of the usual mistakes made are because they tried to use the French orthography - many schools teach French before Creole. Foreign words can be either written with the original orthography or the best approximation in creole.

I am also similarly thankful for the indian languages I know and grew up learning.

There is no difference at all between what you call a letter and how it sounds when it is used in a word. Reading is simply sounding out each letter as you go from left-to-right.

Point taken, but it’s not bullshit advice when the particular word in question actually follows typical English spelling rules (which, despite countless exceptions, do exist)

Hippo birdie two ewes

to quote an old greeting card, for example

You know you would think so, but for me it's actually been helpful many times for spelling a variety of words. I don't think I realized until checking out this thread that a lot of what I thought was spelling words how they sound was actually me memorizing the way they were spelled and somewhat skewing the pronunciation in my head in order to get the correct spelling.

That's interesting. Sounds like he may have auditory aphantasia (which I only know is a thing because of the few times it's come up on HN lately).

My wife can’t spell and I just tell her “hey just memorize how to spell everything”. I’m convinced that’s what we all actually do, at least with English.

I know this is common but I constantly mispronounce words phonetically in my head so I remember how to spell them.

Wed-nes-day

You know, I don’t pronounce this one as it is spelled, but I do pronounce the first R in February, unlike many.

Back when I worked in support and had to do a lot of chat/email communication, I did this for a lot of words - "environment" being one of the most frequent ones. Many years later, I realized that my "mispronunciation" for spelling it was actually closer to some people's pronunciation! I never realized it, but I was only familiar with the southern pronunciation, which totally lacks the second "n" sound ("in-vire-mint").

I think there's some sort of "deep learning" analog going on, where we memorize a lot and it subsequently leads to generalization (that can still sometimes overfit and be wrong). For me anyway, I write and read a lot, and having broad exposure to words coming from different root languages etc gives a good feeling for what spellings make sense. At the same time, I'd guess the majority of my adult vocabulary has come out of books, or at less adjacent to them, so it's pretty uncommon to be writing a word I've never seen before

I used to read a lot in my teens and I agree. I could spell difficult to spell words I didn't even know I knew how to spell, and it felt like at some level, I didn't explicitly remember how to spell them at all. It seemed pretty much impossible to give me a valid English word and have me misspell it. I could feel them sort of matching a cluster in my brain, like "by the way it sounds, it has to be like one of those words that does this thing"--I didn't think that, I could feel it as I spelled it out, and I'd have no conscious idea what the grouping is. I'm sure my brain had an unlabelled network of neurons that knew how to spell words in English that came from old Norse but got mangled by French influence and crazy random stuff.

People underestimate the amazing pattern recognition the brain can do if you feed it obscene amounts of input.

Well how else would we learn spelling?

Didn’t you literally have spelling lessons at schools as a kid where you’d have lists of words to learn?

I think the popular wisdom in the US at least is you learn phonics and learn patterns which is how you learn to spell. As if there’s some “trick” to it. But especially in English there’s a million different things that don’t follow the supposed rules. So just memorizing by virtue of being constantly exposed to words is the way to go. Eventually you can’t help but be mostly literate.

You can only "spell it exactly how it sounds" in languages like Italian or Spanish, not in English.

"Oaxaca," just like it sounds

That's a really good example. Was it supposed to be a bad one?

The only glaring exception I run into in Spanish is "que" being pronounced "ke."

In my learning, I've definitely found that holdover words with "X" are the strangest relative to what is taught. "Oaxaca" has at least two different pronunciations depending on where you are in Mexico, since the pronunciation has evolved over time. See also: xerez.

Not the best at Spanish, but it always seems to me that all "x"s drift based on location, not just a few, making it an accent for me rather than an inconsistency. "ll" is the same way.

> Not the best at Spanish, but it always seems to me that all "x"s drift based on location, not just a few, making it an accent for me rather than an inconsistency

Doesn't seem that way to me; the native speakers I’ve known tend to pronounce the X’s different ways in:

México / Oaxaca

vs.

Xochilmilco

vs.

Ixtlan / Tepexpan

I'm not as familiar with American place names, but there's an obvious reason why those would have pronunciations that couldn't be spelled easily in Spanish.

"X" in Mexican might be shorthand for sharp mystery consonant.

In each of those, the “x” originates from Nahuatl where it had a sound of (or close to) English “sh”, which is also the sound “x” had (maybe still has, though I’m pretty sure there has been shift in Iberian accents/dialects since then) in certain Iberian regional accents of Spanish at the time, which is how “x” got into the Spanish spelling; it wasn't a stand-in for a mystery consonant.

Your comment is correct. As far as I know, "x" still sounds /ʃ/ in Galician, Basque and Catalan. In Spanish, it became /x/ centuries ago, and the spelling changed to J/G. Nowadays you only see the X in old-fashioned spellings like Quixote, Xerez o Ximenez.

https://es.wikipedia.org/wiki/Reajuste_de_las_sibilantes_del...

https://en.wikipedia.org/wiki/Phonological_history_of_Spanis...

This is because such words are not derived from Latin like most Spanish words, but are from native languages.

Yeah, but every language has those peculiarities in spelling. AFAIK Serbian Cyrillic is the closest one to "spell how you hear it" because every letter corresponds to a single sound and there's little deviation from this.

With the Serbian Latin/Croatian alphabet it gets a little complicated because it uses digraphs such as lj(љ in Cyrillic), nj(њ)and dž(џ).

> With the Serbian Latin/Croatian alphabet it gets a little complicated because it uses digraphs such as lj(љ in Cyrillic), nj(њ)and dž(џ)

People are extolling Italian here, but note that for those three sounds it uses the digraphs gl, gn, and gi.

"Oaxaca" isn't Spanish, it's a Nahuatl loanword.

Even those have a whole bunch of exceptions. The best language for spelling by sound is probably German, it has an extremely clear pronunciation guide to the point where most first year German students can quite accurately read signs and essays aloud without having the faintest idea of what they're saying.

Works decently well in Russian too.

Even those have a whole bunch of exceptions

Are you sure? I'm a native Spanish speaker and mastered the "whole bunch" (ce, ci, que, qui, gue, gui) by the age of four.

Spanish is the one I'm familiar with and there are a bunch of collisions in pronunciations like b/v and h/j (which are sometimes discernable but often aren't) along with a fair number of silent letters.

> Spanish is the one I'm familiar with and there are a bunch of collisions in pronunciations like b/v and h/j (which are sometimes discernable but often aren't)

That doesn't sound right. What variety of Spanish are you familiar with that sometimes distinguishes B from V?

Exactly, there are none. Some persons make them a little different because they also speak other languages, like French or English where they're different. There's no problem wiht that, but it's not needed either.

In Spanish B and V sounds exactly the same. H and J are strictly different. H is the only silent letter.

Let me remind you that I was answering to your comment where you say:

...read signs and essays aloud without having the faintest idea of what they're saying.

This can be done just learning the independent sound of each letter and the six exceptions that I mentioned.

Graphic accent rules are a bit more complex, but we also learn them by the time we're seven or eight years, so we can use in our writing. They're not essential for reading.

> In Spanish B and V sounds exactly the same

Depends on accent. Some regions do diferenciate. Same with S/C/Z

The person I was responding to said that German can be read without knowing the meaning, unlike Spanish. That's not correct.

Depends on accent. Some regions do diferenciate.

Maybe because, as I wrote in a previous comment, they speak another language that does have different sounds.

But again: it's irrelevant for a foreigner learning to read standard Spanish. When you learn English they won't teach you cockney, they teach you some standard pronuntiation.

This video discussing the v in Latin is interesting, and mentions b/v in Spanish. But watching this gives some insight on how these sounds merged over time, and has interesting clips of dialects of other Romance languages where this effect occurs to a greater or lesser extent. The comments are well worth reading too.

https://www.youtube.com/watch?v=hovf-UK-toQ

I think Japanese takes the cake, here. If you can hear the syllables you can spell the word, period.

Actually yea - I hadn't even thought of Japanese - or Korean. Korean is basically a phonetic language with a built in pronunciation guide!

The statement was about a specific word, not the language as a whole.

Spanish can be problematic due to the influence of Nahautl, Quechua, and other languages of people colonized by Spain.

But it's still a million times better than English.

This is interesting. It reminds me of how people usually ask of people with aphantasia to "just visualize". We tend as a group to forget that individuals are very different from each others. Some people have no inner monologue, others have no images, some can never hear the difference in pitch while other have a pitch perfect hearing, etc.

Sounds like the distinction between successfully teaching reading strategies, and being taught unsuccessful[1] strategies. He might have learned how to read in a different manner than you.

[1] https://www.apmreports.org/episode/2019/08/22/whats-wrong-ho...

I was learning Japanese in Tokyo while also working on a small-business accounting app. Japanese accounting is extremely specific, so even for someone enrolled in classes, there was no way I would be learning all those kanji any time soon. I impressed a Japanese co-worker though, because I quickly learned visually where the buttons I needed to click were so I could copy the app features from iPhone to Android. Turns out you don't have to be able to read almost anything as long as you see something used correctly and can remember button locations visually etc.

Chinese characters are hard to learn even if you are Chinese. Places that use them have high illiteracy rates but many people learn how to "fake it".

This is very much untrue. Writing with a pen, maybe. Rare, obscure characters from ancient texts, sure. Basic daily-use characters are understandable to even most of the elderly in super remote villages.

I live in Shanghai. This isn’t true. Lots of people over 50 in Zhejiang, one of the richest provinces in China, are functionally illiterate, knowing maybe 1,000 characters. In poorer provinces people who are younger than that will be illiterate. In Western China there are tons of villages where no one speaks Mandarin so they sure as hell aren’t going to be literate in it. A very large majority of people under 30 are literate because in the Han core areas most people finished primary school but there are lots of illiterate people.

When I was young I played a lot of Japanese games on the MSX like this: try stuff, see what happens, memorize, repeat as needed. I played through SD Snatcher and some other games like this.

Kids can have a lot of patience in a weird way like that. Today I'd probably give up after a few minutes.

When my younger brother was very young (perhaps 5 or 6 years old), he inherited my old Gameboy Color with my copy of Pokemon Crystal, which I had left saved at some random point midway through the game. He, not knowing how to read, never realized that there was such a thing as saves or progress in the game. In his mind, that save point was his "start point", where he would always begin his adventures and the pokemon I had left him with were the only ones he ever used. He loved to run around in the wild battling pokemon and occasionally making very accidental progress away from his "start point".

One time, he managed to stumble his way into using Cut to remove a tree and entered a whole new world of things to explore! After a while he become totally lost, and wanting to return to familiar territory he simply restarted the Game Boy and was right back where he knew. We continue to find much humor in the degree to which children can make any random experience a fun adventure.

My daughter loves to play “that spaceship game” (Among Us) despite not knowing how to read. She had great fun running around and looking at everything. She knows how to join lobbies. It’s probably good she doesn’t know how to read as I’d imagine there are a lot of very angry 12 year olds throwing expletives her way for being “bad” at the game.

Japanese was something we Argentinians really didn't know back in the 90's when I was a kid (we still don't usually know it, but now I know some people who are picking up classes).

But then there was Captain Tsubasa's FamiCom game, which some friends had, only in Japanese - and it amazed me how easily they went through all of the menus to change whatever settings they wanted.

Kids, spare time, the desire to play some videogames, and a country whose market wasn't that attractive - an awesome combination.

I worked at Office Depot for a while, and Chinese couple brought in their laptop with a problem. OD didn't do repairs, but I had worked at a Gateway 2000 call center and a local computer shop, so I knew my way around.

Their computer was completely in Chinese, but that didn't stop me from just quickly clicking through to the control panel and fixing the problem.

They were practically slack-jawed staring at me, and then asked, "Do you speak Chinese?" I told them I just know where the buttons are. :D

It was experiences like that (helping a customer beyond what the store supported, in extreme situations) that make me miss retail sometimes. All the BS makes sure I don't go back.

This reminded me why I loved my retail job, but I think its partially because I was retail inside an amusement park: I missed out on most of the BS (people were generally very happy to be there/ having a great day).

From that I find myself assuming it was one of the smaller parks, to avoid the "people going insane over wringing every last bit of 'fun' out of an incredibly expensive vacation" problem.

Reminds me of a helpdesk job I had back when Windows 98 was the standard. Before the display settings had that handy confirmation countdown, it was surprisingly easy to set your display to something the monitor couldn't handle, leaving you with no way of seeing what was on the screen to change it back. Eventually I memorized the series of keystrokes required to get into display properties and reduce the resolution, which usually fixed the problem.

yes, back in the good old days when buttons stayed put instead of dynamically rearranging themselves to put the most recent one of the top.

I've spent many years in countries where I understand very little of the local language, so this has been basically my life when interacting with local apps etc. I just learn which buttons to press, largely through trial and error or a few sessions with a second phone and Google Translate in camera mode, and can get quite "fluent" with the app without ever really learning the language. This kind of "UI literacy" is not universal though; I encounter other people who are not at all capable of doing this without understanding what the text on the buttons means.

> The speed at which they traversed the phone menus was the same as for literate people. We often had to ask our interviewees to slow down when they were showing us how they performed certain tasks on their phones. They mastered important functionality through rote learning: “After I have clicked on this icon I need to go down twice and then – click! - I’m done.” This was the same technique that they used to learn how to operate other important digital interfaces such as ATMs and game consoles. Family or friends assisted during the memorization phase and they repeated the procedures in their presence as many times as needed.

This reminds me of a certain facet of computer literacy: exploration. There are users who will only seek out menu items in software if they already know they are there and where, compared to users who will seek out the items they are looking for and can imagine that they probably do exist somewhere.

It's unimaginable to the latter user that one would possibly have to "learn" how to use Google Docs having only ever been exposed to MS Word, while the former is completely lost because the menus have to be re-learned and all the possibilities available to them must be re-memorized.

Beautifully put. I've experienced this dichotomy so many times in many different contexts. Being literate and being able to explore are extremely empowering gifts.

I was a contractor once and was basically used as a scapegoat after being asked to move some UI bits for the sake of consistency. This was moving the OK and Cancel buttons on every single dialog window so they were in the same consistent place.

The customers shit the bed and the change rolled back and an “unspecified contractor” was blamed for “an unauthorised change”.

I chose not to renew that one.

They should have done them one at a time. You can reasonably gaslight the client that way. I sometimes forget whether or not OK should be on the left or right of Cancel.

https://help.duo.com/s/article/7231?language=en_US

The Duo authentication app recently swapped the 'Accept' and 'Deny' buttons and it was unbelievably frustrating.

It's amazing that I use computers all the time and I can't remember what the standard placement of OK/Cancel is, offhand. It's like how nobody can remember which way the Queen faces on a coin.

Actually that is easy to remember, because OK/Cancel is the wrong Windows way and Cancel/VerbOfAction the enlightened Apple way.

https://news.ycombinator.com/item?id=17269938

> Windows puts OK first Apple puts OK last

Gnome, iOS and Android followed Apples style, which influences Web Apps.

That makes me think of how many times I've had to ask people "what does the error message on the screen say?" or tell my parents "the words that it says on the screen - read them, the program/website is trying to tell you something". When it comes to software many users are functionally illiterate because they don't even try to read alert windows or error messages, assuming that they will be meaningless.

Yup, I've seen this a lot. This type of user basically operates like a script, they move through a memorized list of steps and when they can't go forward they simply throw an exception.

not even really an exception, they're more a like a PHP script that just ignores errors and continues

> they don't even try to read alert windows or error messages, assuming that they will be meaningless

I work in IT and I can assure you that software developers have been doing their best for the past 20 years to live up to that expectation.

I'm currently breaking my head trying to fix a "you didn't fill out all fields" error for a form with only hidden fields. I keep thinking how useful the message must be to the user.

I still recall my father having various error messages from software (browsers, website, Word, etc), and when I'd translate them it was almost always a case of the error message being meaningful to me _but not in those words_, and so when I'd translate them for him he'd ask, "why didn't the error message say THAT?". It still is something I think about occasionally when trying to write logging or error messages -- how we got there, how to fix it, etc.

Can you give an example?

For a long time on Windows, I had to "copy" a complex directory structure, never use "move". A file lock could bork it.

No meaningful indication what file was locked. No meaningful indication what was locking the file. The process simply stopped.

It got much better after Win 8 onwards.

THIS.

So much this.

I've made that gripe too, but there's something about error messages though that makes them very hard to take in. I don't know why, but I debug crap for a living, and I still every now and then find myself searching for a bug in Google, only to realise that there are instructions for resolving it in the error message.

I can't quite put my finger on why I can't adapt. It's some combination of the facts that 1) there's often a huge dump of information, much of it so technical that it's useful only to the developers, 2) it's fairly rare that it offers a truly useful suggestion, and 3) many of the messages I encounter ask me to do something that makes things easy for the developer rather than the user (update the software, contact the manufacturer).

I think it's something we as an industry ought to work harder on, because I don't think the failure is only in the users. I think, although I can't explain exactly why, we're communicating very poorly.

Anybody have ideas on why the age divide on expectation + exploration vs. memorization seems so stark?

In my experience the top predictor of whether a person can use a computer to do what they intended to do is age. Not IQ, not other mechanical or technical competence, not education, not cultural background, not linguistic aptitude, not reading speed, not creativity, not engineering or product design experience... mainly age.

(with outliers of course)

My guess is it's about how curious about new things you were when computers got popular. Computers are, compared to a lot of real world devices, a lot more complicated than pretty much any consumer tool that predates them, but also more forgiving. Most consumer appliances before computers had at most a couple of settings, all of which were clearly defined and handed to you. Anything beyond that involved aggressive tinkering that may or may not go well.

Computers, on the other hand, have interfaces that are vastly larger than any human brain could hope to contain. So they require exploration. This comes naturally to children, less so to fully matured adults, and is extremely rare in people middle-aged or beyond. So you get a line based on how old they were when they first got their hands on a computer.

It probably has to do with older people being introduced to computers as big, complicated machines with consequences if you mess up, while younger people were first exposed to computer games as kids and knowing it's possible to undo stuff. Until they end up coding a buggy smart contract and losing 6 figures.

The thing I've noticed about people on the other side of the divide is this irrational fear that doing anything incorrect, even once, is going to completely brick their device and cost a bunch of money. So they develop these weird superstitions and go through the ritualistic rote-learned menu clicks because exploration doesn't feel safe.

I don't know the origin of this attitude.

My elder relatives even went as far as writing down (on paper of course) the exact steps to do some specific thing. Which will of course inevitably fail because some popup or whatever putting them into some (to them) inconsistent state.

>In my experience the top predictor of whether a person can use a computer to do what they intended to do is age.

From what I've seen, it isn't age, even IT people. Sorry.

Relearn everything?

I use the Apple, Google and Office Suite interchangeable, you don't need to relearn everything.

This is no surprise to me.

My dad wrote batch files when I was a kid. 98.bat give a list of items numbered 1-15. 3.bat then took you to a games directory wherein 99.bat gave you another numbered list list of 30+ games to choose from.

I memorized nearly every number on both menus before I could read.

Same but for porn.

…hopefully not before you learned to read.

This is basically how my parents and grandparents navigate computers, despite being perfectly capable of reading. It infuriates me when tech companies just move shit around in the interface for no good reason and I have to teach them how to use it all over again. UI/UX people consistently fail this slice of people by not keeping to a stable interface.

Funny story from a school teacher:

One of the kids she has is determinedly illiterate (7 years old, pretty poor family background) but said kid, after "borrowing" one of the class iPads during lunch hour managed to search up a pile of pornography and got caught.

Teachers were somewhat perplexed how this kid who barely manages to write his own name searches up some fairly complicated terms. He did it by simply using the iPad speech recognition system.

Suggestion engines are powerful too. When my son was just about 3, he could navigate YouTube from and arbitrary point to Spider-Man and His Amazing Friends or a particular garbage truck video.

This. I've heard stories about kids effectively using the suggestions as a multi link menu system.

They open the site, look for one familiar suggestion to start off on, and follow the web from there.

Kids can do genius stuff, they are very very open to experimentation and out of the box thinking(as they don't have a box to start with, I guess).

I recall someones kid discovering that clicking on ads in games opens the safari within the app, finding their way from there into Google, therefore bypassing the parental restrictions.

On a related note, my daughter (grade 1) uses speech recognition to do spell checking and confirm how a word is spelled. Mind, she does complicated words like "imposter" and "infected".

(real example, and yes she's found "Among Us" playthrough videos...)

Small sample size, but still very interesting. I was shocked that some people think text should be removed from the UI for them. I would hope the goal would be for them to learn to read and I have to imaging after seeing certain words like "close" or "save" over and over they would get some basic understanding of the shape of the word to help them in new applications or other areas of their life.

Favourite fact of the moment : illiterate humans built the walls of jericho as it was built 2,000 years before writing was invented.

Text is a jet fuel for humanity, not a requirement for success. I am only saddened that "we" have let down those 800M by not giving them a basic education

I know someone with very bad dyslexia and it's not that he didn't get a basic education, it's that he can't understand written text or menu items more than one layer deep. He calls if wants to communicate which is very endearing but I also have major misgivings with modern UI paradigms. The "three dashed lines" that you see everywhere are infuriating.

Do visual icons like the three lines for a menu help or hurt the usability of an application for people with dyslexia?

I had assumed iconography was a boon for illiterate people- thinking primarily of symbols on shop signs or a striped barbers pole, for example.

> Do visual icons like the three lines

They hurt usability even for people perfectly capable of reading. Especially when Material fanboy fill everything with whitespace and bans black for the text.

It always blows my mind seeing a 4-year-old, who can't even read, navigating YouTube on their parent's phone better than some adults I know with PhDs.

Though as a kid I do remember playing RPG games in a language I don't know at all.

I learned English as a kid by trying to read the manual for game maker. I'd sometimes get my parents to translate little bits, but they had better shit to do than sit there and translate line by line.

Somehow I managed to bootstrap that into (IMHO) pretty fluent English. Ask me to do something similar today and I am 100% sure it'd be impossible.

Little kids are impressive.

Interesting, my initial reaction to reading the abstract was: why are we thinking about UX instead of investing the money and time in fixing the root cause - illiteracy? But then I thought: It's not that people can't read or write, it's that they often move to new countries and have to deal with a completely new language. So: I guess the solution would be less text, more visuals and symbols. But: are symbols universally understandable? Not all of them for sure, but definitely a portion of them; especially those pertaining to our biology and lowest common denominators as humans. Pointing inwards with a finger at own's open mouth should be an universally understandable symbol for hunger; a pictogram for sun should communicate heat -- but it could also communicate other things such as good fortune, daytime etc. and this is where problems arise: precision suffers, misunderstandings happen. Can we do away with language altogether? I was thinking about starting with the universally understood axiomatic symbols and using them to create more and more complex symbols to achieve precision. But I guess that's exactly what language and math does, even historically starting with cave paintings, through graphic languages and whatnot, increasing abstraction and information density. So this brings me back all the way to the initial problem, with the feeling like I've barely understood anything, not even mentioning a solution...

I remember reading a very interesting paper on how can we make nuclear waste dumps marked as unsafe for 1000s of years. It employed things like surrounding it with sharp, outward pointing dangerous objects, building a maze-like concrete structure around its general area and trying to create universally understandable pictograms to say “stay away”, even for a possible post-nuclear-catastrophe society.

Unfortunately I couldn’t find that paper again, but it was really interesting read and you might also find it interesting. Another similar topic that comes to mind is the https://en.m.wikipedia.org/wiki/Voyager_Golden_Record

The key phase to search for is "this is not a place of honour", and you're looking for https://www.damninteresting.com/this-place-is-not-a-place-of...

Wow, thank you very much for this! At times I even contemplated whether I just dreamt about this article, but I wouldn’t be that creative to imagine something like that :D

> It employed things like surrounding it with sharp, outward pointing dangerous objects, building a maze-like concrete structure around its general area

This actually sounds like the perfectly wrong choice - it reminds me of various movies/TV where such traps are used around temples or tombs to keep treasure and sacred artifacts safe, making them prize targets for graverobbers or archeologists.

See also various ancient tombs (Egypt being primarily on my mind).

I came across large numbers of illiterate people in jail. They would have no problem ordering their commissary even though they could not "read" any of the forms or labels.

The illiterate guys were usually the ones who were best at card games though.

The only illiterate folks I know are my niece and nephew, who are young kids.

I don’t know what I thought my 3 year old niece was going to do on her iPad when she asked if I wanted to watch “dog shows” with her, but I was surprised to find out that she knew about the voice to text feature built into iOS and used it to search YouTube.

It’s kind of amazing what you can do with just your voice and a determination to watch shows about dogs. :)

> I was surprised to find out that she knew about the voice to text feature built into iOS and used it to search YouTube.

That's something I noticed across most of the younger children in my extended family; They all love the Siri's, Alexa's and whatnots.

I think it's because such a natural way to interact for children that age, they love talking, and to them there is very real novelty to have this flat, obviously innate, object reply to them in rather human ways while still being able to "tinker" with it by asking it stuff that doesn't result in good responses.

Yeah an even the most patient of adult tires of the same question over and over but Siri doesn’t.

Buttons and prompts, man. Give people one question at a time, get their response asynchronously. The good ol' days were people had phone interactive prompts with number# instead of automated voice options call.

People have an intuitive feel for what they want based on the words shown to them, rather than listening and sitting through an automated voice and pressing buttons. Moreover the number# process allowed you to add options (like -2222*2#) that way you could easily go to your desired services.

But with more and more AI Voiced Customer Support popping up, I think we are adding more bureaucracy in the promise of efficiency.

I know a guy who uses whatsapp by clicking on peoples pictures, using voice and video calls and sending audio messages. It doesn't seem like there is a problem.

Nowadays people also use voice-to-text to send SMS / whatever messages ... and car systems don't even let you read messages, they just speak them out.

Not to mention all the video-training going on - literacy may be on the diminishing returns path for much of the population these days.

However nervous that makes us who are still surfing the last five centuries wave of ever-rising literacy.

I feel that the study is dated. Nowadays (at least on iOS) VoiceOver is pretty good: they can read almost all parts of the contents, maps, and even texts on pictures. Not sure how much of it is designed with illiterate people in mind, but wouldn't be surprised if they are. Mobile phones are truly becoming a device of empowerment now.

Very eye-opening paper. Thanks for posting this. I have never considered most of the things brought up in this. Really good to read.

My nearly literate 5 year old does pretty well in Kerbal Space Program. He learned how to do this by clicking around for hours and memorizing what happened.

recent thread on literacy overall

https://news.ycombinator.com/item?id=29734146

What would be interesting is to see how “low information” people use mobile phones.

What apps do they use?

How do they use search?

Would be fascinating.

Reply on news.ycombinator.com