Hacker News Reader

Comment by musesum | Hacker News Reader

Have been discussing with a visually impaired friend how to rethink Accessibility. Over the last 20 years, I've seen him use Accessibility on Windows and then moving over to the Mac and finally the iPhone. The main problem is that VoiceOver is a speech+touch interface slapped onto a visual+touch interface. So, roughly 1/3rd speed. Moreover, 3rd party developers need to label elements, which they rarely do.

I remember walking out of an Accessibility meetup at SalesForce waiting for a ride and seeing an attendee struggling with hailing an Uber. She was an experienced user but still struggling with her new iPhone. It had a larger screen, so all her muscle memory was foiled; the buttons had shifted.

So, my friend and I are looking into rethinking the interface starting with 1st principles, like Fitt's Law. My guess is maybe a 9X improvement for visually impaired and 3X for folks with regular vision. Reserved the name touch.ai for that purpose.

View on HN

> Moreover, 3rd party developers need to label elements, which they rarely do.

Seems like your approach would require 3rd party developers to build a completely new UI, so this doesn’t seem like a valid criticism of the current paradigm.

It’s also the case that existing designs take into account Fitt’s law. Apple and Google are well aware of these.

That said, I do share the intuition that an information architecture built around cognitive efficiency could be a lot better than we currently have.

I’m skeptical you’ll find particularly easy to achieve - a lot of the complexity of current UI is incidental, but equally a lot is not.

> Seems like your approach would require 3rd party developers to build a completely new UI

Nope; no changes to existing UIs. Instead a driver, which has its own problems of deployment, as that requires convincing the hardware OEMs.

> It’s also the case that existing designs take into account Fitt’s law. Apple and Google are well aware of these.

Hmmmm ... I don't think so. I remember Bruce Tognazzini wrote about how Apple messed up Fitt's law in a version of OSX, where it was one pixel away from supporting an infinite target.

More recently was Apple's addition of "reachability" which is an incredible kludge.

Apple and Google may have usability research, but that is nothing compared to the deep research done at Xerox Parc. I recall seeing studies on the D* machines which measured the efficiency of several text editors, broken down by select, cut, paste, etc. Xerox first tested and measured alternatives before deciding on the best one.

At the first WWDC for the iPhone (2007?), I went a design lab to review our first app. The designer suggest I put navigation at the top, out of reach. I asked him "What about Fitt's Law?" His reply was "who's that?"

> I’m skeptical you’ll find particularly easy to achieve

I agree; this is incredibly hard to achieve. It involves integrating several moving parts, leveraging: TPUs, crypto, and negotiating with OEMs like Apple. I believe the payoff is worth it.

> Nope; no changes to existing UIs. Instead a driver, which has its own problems of deployment, as that requires convincing the hardware OEMs.

I infer from this and your domain name that you plan on using ML to ‘read’ UIs and extract salient features into a canonical model, and then to transform this into your more efficient interaction paradigm.

Is that a fair read?

>> It’s also the case that existing designs take into account Fitt’s law. Apple and Google are well aware of these.

> Hmmmm ... I don't think so. I remember Bruce Tognazzini wrote about how Apple messed up Fitt's law in a version of OSX, where it was one pixel away from supporting an infinite target.

I’m curious if you remember the example. Also - Tog worked at Apple for 14 years, so they clearly did know about it at that time.

> More recently was Apple's addition of "reachability" which is an incredible kludge.

It’s kludge but it has nothing to do with them not knowing fitts law. It has a lot more to do with the iterative path which started with a screen that was small enough to be reached with one hand as a constraint. Market demand forced them to relax this constraint, and they haven’t caught up with the changes yet.

> Apple and Google may have usability research, but that is nothing compared to the deep research done at Xerox Parc. I recall seeing studies on the D* machines which measured the efficiency of several text editors, broken down by select, cut, paste, etc. Xerox first tested and measured alternatives before deciding on the best one.

Did you know that many of those folks went fron Parc to Apple and continued their research?

I don’t doubt this, but Apple has a huge number of designers. A developer evangelist is quite different from someone reporting to Alan Dye reviewing fundamental changes.

> I’m skeptical you’ll find particularly easy to achieve I agree; this is incredibly hard to achieve. It involves integrating several moving parts, leveraging: TPUs, crypto, and negotiating with OEMs like Apple. I believe the payoff is worth it.

Do you have the core technology proven out yet?

> I infer from this and your domain name that you plan on using ML ... Yep

> I’m curious if you remember the example. Also - Tog worked at Apple for 14 years

I couldn't find it, although he has another post on Fitt's Law. The phrase he used was "pulling defeat from the jaws of victory" - maybe it's on Archive.org

> it’s kludge but it has nothing to do with them not knowing fitts law.

Yeah, my characterization is a bit unfair

> Did you know that many of those folks went front Parc to Apple and continued their research?

I did have a chance to meet Jef Raskin a few times. He was between Apple stints, working on the Canon Cat. He's the one who turn me onto Card, Moran, Newell 1983 -- and thus Fitt's Law.

I don't doubt there is a lot of thought that goes into refining the UI. However, there's the quandary of breaking the current idiom, which limits change. However, for low vision folks, there is less of a switching cost. Same holds for newer devices, like glasses.

> Do you have the core technology proven out yet?

Tis incomplete. Some parts of the interaction model. Am a bit weak on the ML part. May license GPT-n. Looking to form a team. Just convinced my blind friend to join. He kinda pioneered AR in the 90's.

Reply on news.ycombinator.com