Thoughts on Image Processing in JavaScript? | Hacker News Reader

Thoughts on Image Processing in JavaScript?

7 pointsivankirigin14 comments

I intend to work on javascript tools to do image processing at DevHouseBoston. http://devboston.pbwiki.com/Dev3Projects

The goals are to build an open framework, to learn more JavaScript, and eventually to implement that crazy dynamic image resize method that's been floating around the internet. http://www.faculty.idc.ac.il.nyud.net:8090/arik/IMRet-All.mo...

I thought I'd ask this group if they knew of any existing tools to use. A cursory google search found little.

This might be easier to do in Flash, but I want to learn more JavaScript and get the sense that it will be more portable this way. Any thoughts on this issue?

View on HN

JavaScript in a browser doesn't give you enough low-level access to manipulate images. A good compromise is to build the UI in javascript, and have the image processing on the server. Then, use AJAX to give the user a nice and quick experience that feels as if the processing is happening on the client. Of course, you won't have as dynamic a behavior as if it's all on the client, but that's probably the closest you can get with JavaScript. Other options include using a JavaApplet that runs on the client, or a .NET component (IE on Win only).

Right, I've been told a number of times now that I can't get access to the image buffer in JavaScript. That's too bad.

I don't believe it can be made asynchronous and real-time. I suppose JavaScript could be used to control which resize scheme is used. While the bounds are changing, just scale and warp the image as a browser would today to make it fit a size. Once it stops changing, do an asych call to get a resized version of the image, with whatever fancy resize is needed.

I'd like to avoid .NET and Applets. Flash might be a reasonable solution.

I think you're right. I think the options for image processing on the client side are currently limited to VML ( http://en.wikipedia.org/wiki/Vml ) for IE or for Safari/Firefox/Opera, SVG or Canvas ( http://en.wikipedia.org/wiki/Canvas_(HTML_element) ) and from the little bit of tinkering I've done with both, I don't think you can read the values of pixels of image buffers with any of those options. If anyone can disprove this I'd be very glad to hear it. You can with Flash or Java applets but both of those ideas have other problems as well.

It certainly would be useful if it were possible.

I've also been trying to figure out how to control a scanner from in the browser. I'd like to have something on a page which would scan a document, display the image in the browser, and upload it to a server. I was hoping I could do it with Flash, because Flash does have a TWAIN interface, but I've not been able to get Flash to recognize scanner devices, only cameras. I'm starting to think applets may be the answer to that, but that seems like a long road to start on.

Does anyone have any ideas about controlling scanners within browsers?

I don't know if you can actually modify the image, but you could do things like change image width and height properties, or use css in ways to show manipulated image.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> </head> <style> body { margin: 0 } </style> <body> <script src="http://particletree.com/scripts/prototype.js" type="text/javascript"></script> <div id="image_container" > <img src="http://www.google.com/intl/en_ALL/images/logo.gif" id="image" style=" position:absolute; " /> </div> <input type="button" onclick="resize1()" value="double size" style="position:absolute; top: 260px;" /> <input type="button" onclick="resize2()" value="morph right side" style="position:absolute; top: 300px;" /> </body>

function resize1() { i.width = oldWidth + oldWidth ; i.height = oldHeight + oldHeight ; i.style.clip = 'auto'; if ($('image2')) {$('image2').remove();} }

function resize2() { i.width = oldWidth; i.height = oldHeight; i.setStyle( { clip: 'rect(0px 138px 110px 0px)'} ); new Insertion.Bottom( 'image_container', '<img src="http://www.google.com/intl/en_ALL/images/logo.gif" id="image2" style=" position:absolute; " /> '); i2 = $('image2'); i2.setStyle( { clip: 'rect(0px 552px 110px 276px)' , left: '-138px' } ); i2.width = oldWidth + oldWidth ; i2.height = oldHeight; } </script> </html>

You could use overflow:hidden instead of clipping. Then find the math formulas for the resizing, attach drag handler to image edges, overlay semi-transparent layers to emulate effects. To save it in the end though the user has to take a screenshot or your backend script has to process it for the user.

I'm told they've filed a patent on the method they used. Often a component can be changed in the actual implementation to avoid infringement, but that means more work.

Also you can find a paper for the work here: http://www.faculty.idc.ac.il/arik/

My first instinct was "no", but then I remembered this:

http://www.abrahamjoffe.com.au/ben/canvascape/

I don't know what kind of image processing you want to do, but the demo above definitely pushes the boundaries of what I thought JavaScript can do. I haven't had the time to dig in and learn how it works.

Really simple stuff - moving, resizing, cropping - is trivial to do with JavaScript and server-side with something like ImageMagick.

No matter what, beware of IE6. I've only recently come to truly appreciate how slow and buggy it really is. No idea if IE7 is better. So I'll change my answer to "Maybe, but not in IE".

Image processing is not possible with JavaScript.

What do you mean not possible? You can't touch pixels in an image buffer?

Is it possible that you are referring to Java rather than JavaScript?

No, I just didnt know before I asked whether browsers let JavaScript have access to the low level image information.

I mention below the best options: server side or flash.

Correction: JavaScript embedded in web pages cannot process images. Mozilla Rhino most certainly can, and I'm pretty sure Mozilla SpiderMonkey can as well if given XPConnect (or similar).

Hey Ivan, JS is a seriously crippled language when it comes to doing low-level processing. Like others have mentioned, cropping and resizing are trivial; however, to do anything even slightly sophisticated I'd suggest looking into flash.

Flash 8's image API -- specifically the BitmapData class -- can provide some low-level access to an image buffer (however the image must be round-tripped from localhost to the server if you want to edit local images).

Alternatively, IE provides some proprietary image filters which can be used to process images (but obviously not in a cross-browser way).

Ok, to counter my bummer parent post here are some things you COULD feasibly implement in javascript:

- color tinting (overlay semi-transparent colored elements on an image)

- image cut-and-paste (like cropping, place some bounded area of an image in a div with hidden overflow [the copy] and allow that to then be placed over the original image [the paste])

- image gradients, drop shadows, reflections (but these have been done to death and are usually too sluggish to use in a production app)

- some neat UI tricks like scroll wheel zooming and gesture-based panning (mootools has some demos which show what's possible - demos.mootools.net)

- if you really wanted to kill yourself, you could use flash (degrade to a server-side script) to build a pixel by pixel representation of the image using floated divs and background colors. Then create a JS singleton with your desired methods to be the image buffer's API. Thus, you're granted pixel-level access to the image and could (slowly) do whatever sort of processing you desire in JS.

Wow. Those options are bleak. I'd certainly learn more JS in the process, but it wouldn't be reusable or scalable.

Reply on news.ycombinator.com