EyeDraw – Draw with your eyes using computer vision | Hacker News Reader

EyeDraw – Draw with your eyes using computer vision

fabridigua.medium.com

85 pointspmontra20 comments

I worked with eye tracking software a decade ago and one of the most interesting and tricky problems was your eye would look at the cursor and over compensate, causing the cursor to keep drifting. It was in essence user error but oddly difficult to control. Does anyone know if this has been solved for?

I worked in a team that researched how computor vision might intersect with art and aesthetics. We worked with eye trackers to see if artists look at images any differently to non-artists. The difference was in the way that artists attention would frequently be caught by incidental background items (highlights, corners, features of varying kind). Non-artists would dwell on the immediate content... in the case of people, mostly faces.

I’m thinking of ways that project can have application the this one. Perhaps to train people to look in different ways? Certainly it has the potential to open up the process of drawing to matriculation.

As an artist, I’m itching to have a go at it.

A "killer app" for eye tracking on modern hardware is foveated rendering. A cheap webcam can be a huge force multiplier for a graphics card.

how fast does it have to re-calculate the position of the fovea to not cause visual pain?

If works it seems like that would solve the problem of somebody looking over your shoulder, when reading in a public space, too.

Several other comments are alluding to saccade [1], where your eyes unconsciously jerk around (saccade is the french word for jerk) any vision control system would have to allow for this.

In the simplest approach this would be a low pass filter, presumably you could throw a bunch of ML to filter them out more accurately

[1] https://en.wikipedia.org/wiki/Saccade

> Saccadic eye movement allows the mind to read quickly, but it comes with its disadvantages. It can cause the mind to skip over words because it doesn't see them as important to the sentence, and the mind completely leaves it from the sentence or it replaces it with the wrong word. This can be seen in 'Paris in the the Spring'. This is a common psychological test, where the mind will often skip the second 'the', especially when there is a line break in between the two.

Well, looks like I just failed the test, even though I made a genuine effort to read the paragraph very carefully after the the second sentence.

I missed it too, but jumped back to the opening single quote when I hit the closing single quote and found it. Must be a coding reflex.

Saccades are under voluntary control. They're the fast movements when you switch where you look.

There are microsaccades that are similar but don't seem to have any target we understand. In general these, and other microeyemovements like tremors and drifts, are not very well understood, and even their existence can be somewhat quoestioned.

Low pass filters are somewhat problematic for eye movements because the altenating slow and fast movements.

As a shameless plug here's what I think is still the state-of-the-art method for filtering eye movement data: https://www.nature.com/articles/s41598-017-17983-x

https://outline.com/Ay8vGx

JFTR, Topic link redirects to: https://towardsdatascience.com/eyedraw-how-to-draw-with-your...

And project repo is here: https://github.com/fabridigua/EyeDraw

Impressive what can now be done with such precision with just a regular webcam. Previously special hardware like the Tobii eyetracker was required for this level of precision.

This is a neat trick, but the precision is gonna be horrid with that approach. And it'll be needed to be tuned a lot for each new user.

Special hardware has never been really needed. Just a camera close enough to the eye. IR leds and filters can be added for some convenience.

Only thing special about Tobii etc hardware is the pricetag.

From the GIFs in the article, he appears to be moving his head while keeping his eyes steady (as a way to draw more smoothly.)

I think it could be useful to add voice commands for changing colors, etc.

I love the idea of giving the paralyzed a way to create art.

Nice work.

It's not obvious to me: Is the creator just a horrible artist, or does the system suck?

eyes naturally jump and lock onto things. They don't just slide around between focal points. So... probably neither a problem with the artist or the program.

It's a hilarious thing to say: the fact is eyes are just really bad at drawing and not evolved for that purpose

I can't read this article without a paid medium account? Never seen that before.

No JS and no cookies for the win!

now give me a cursor on my phone.

Reply on news.ycombinator.com