Comment by gavinpc | Hacker News Reader

Comment by gavinpc | Hacker News Reader

Imagine if you could also listen to these documents, with the author reading, and the text tracking.

I believe the effect of that one simple addition would amplify their effectiveness (or at least reach) by an order of magnitude.

After the massive work that their authors put in, the effort of one good read-through would be marginal. If the above surmise is true, you'd be in power law territory.

But you'd need a tool to make such an additional channel as trivial as doing one good read-through.

I'm a little surprised that Bret Victor doesn't do this for his pieces, considering his emphasis on the use of more human capabilities[0].

Implementing the read-back part is trivial [1], and available on all "modern" browsers.

Trickier is making the product that lets authors add this channel without manual tagging. But even this is becoming trivial with speech-to-text API's.

Anyway, I wholly agree with the OP that the first focus of documents in dynamic media should be improved exposition and, by extension argumentation.

[0] "The Humane Representation of Thought" https://vimeo.com/115154289

[1] For example, see a crude implementation at https://willshake.net/plays/Ado/1.1 (click a play arrow in the blue column.) The tracking code is at https://willshake.net/about/text_tracking

Can you expand on why adding a narrative would amplify the effectiveness of these explorable explanations by so much? The whole core of explorable explanations is that you can play with the models yourself to gain insights and reach conclusions the author may not have come to herself, to me, adding a narrative would only add to the linearity of the text itself, and therefor take away some of the "exploration" part

Glad you asked.

First, consider the difference between two people reading aloud: one is new to the material, concepts, and associated vocabulary, and the other is the author. They are reading the same text, but the author's rendition will carry an understanding of the material that is absent in the other one—and not found in the text, either. Doesn't that difference comprise relevant information?

Shakespearean actors do this for a living: they stick to the words on the page, but they add another layer of signals to help unlock them for people. Technical and expository writing can benefit from this effect as well. Don Norman argues, for example, that thinking requires emotion, and the voice is an easy way to add an emotional connection.[0]

So I believe that any document, explorable or otherwise, is amplified by a human reading, when that human really understands and cares about the material. And again, for subjects that warrant these specialized explanations—which tend to require some specialized language—the value added is particularly great, because it teaches people how to read something new.

Second, I strongly believe in an "all-in" approach for things that you care about. Clearly these authors care about reaching people, and I think there can be no doubt that the ability to click "play" brings in listeners who would never have been readers. And by giving a passive "flow" to the presentation, it helps sustain many readers who would have given up.

Adding a channel does not "take away" anything. Everything, including the interactivity and the opportunity for non-linear reading, is still there. (And note that the recording doesn't have to be linear, anyway.) The authors are not competing with themselves, they're competing with YouTube and notifications and other low-friction distractions.

So I'm excited about the OP. I would like this community—mature, passionate thinkers and teachers—to be the avant garde of future media.

[0] Emotional Design, Don Norman

> They are reading the same text, but the author's rendition will carry an understanding of the material that is absent in the other one—and not found in the text, either. Doesn't that difference comprise relevant information?

Not to me, as someone who's never run into the material before. There's no knowledge-carrying particle that's transmitted vocally.

In fact, to me it would be a distraction. I learn better visually; I can stop reading, and start again, and quickly re-scan what I'd read before to catch up. It's harder to do that with a recording of a voice, and takes longer.

> Not to me... to me it would be a distraction

Other people might say the same about the graphics, or the interactions.

The point is that it's not zero-sum—no one loses out by the presence of other channels, if the document is dynamic. But if some people do benefit, then there's a value proposition.

I imagine that the reason for adding narrative is to add another dimension that's proven to be engaging in other mediums; one akin to explanation videos like those found on Khan Academy or the educational sections of YouTube. Sometimes jumping into an explorable explanation without some guide to begin with (outside of text) may be overwhelming.

That being said, I don't know how a spoken narrative would integrate into a piece of text. You would likely not want someone to read through the text as written because a spoken explanation will generally differ in phrase structure (shorter, less verbose) than written text.

The thing about doing this kind of work is you have one sort of explanation in mind as you do it, and like any software, you work until you're there and then you call it done. They're not designed and built with every possible form of accessibility and exposition in mind. It's entirely likely no-one had previously considered doing a narration.

That said, there's also zero advanced tooling to create and edit synced audio, and nearly zero tooling for playback. I wrote my own for http://vitor.io/uxr101, available here: https://github.com/vitorio/hyperaudio-lite

Reply on news.ycombinator.com