HTML5 2D gaming performance analysis: Canvas 2D vs. WebGL vs. C++
scirra.com
scirra.com
Secondly, hardware support for WebGL is very dismal right now. For example, newly shipping MacBook Air's don't support WebGL.
Third, we found so many inconsistencies across different hardware and different browsers that it made it not worth it to work on a WebGL project for the time being, especially for a small team. We wrote a number of runtime checks, but we still could not account for all the bugs, or find ways around every one of them.
If you're writing a new 2D game in HTML5 from scratch, you'll have a much larger market share by going after a slightly simpler (artistically) game using canvas, rather than trying to go for more horsepower with WebGL. However, WebGL can be useful for porting, if you'd like to see an OpenGL game you've already made come to the browser.
The hardware provides a full-featured CPU with DMA access to the host's memory. Everything better go just right, or it's going to end up remotely exploitable.
There is a long history of problems with Flash. Silverlight has had vulnerabilities as well.
But those are 3rd party binary plugins. It's a lot easier to disable, uninstall, and move beyond 3rd party plugins than it is widely-adopted standards implemented by the browser vendors themselves.
It's because it is so appealing and has the potential to become quite popular that we're talking about it now.
John Carmack: "I agree with Microsoft’s assessment that WebGL is a severe security risk. The gfx driver culture is not the culture of security."
What part of WebGL gives a programmer unrestricted access to the host's memory?
Everything better go just right, or it's going to end up remotely exploitable.
The same could be said for practically any program or web app. Your web browser could have a buffer overflow in it's HTML parsing engine, yet I don't hear you speaking out against HTML... - http://secunia.com/advisories/12959/
Ever had a video driver bug crash an application or bluescreen Windows?
Your web browser could have a buffer overflow in it's HTML parsing engine, yet I don't hear you speaking out against HTML
It's a fair point. Security is about trade-offs. I am indeed concerned about HTML and do most browsing in a VM. But to me WebGL is farther out on the risk/benefit spectrum than HTML. I would not accept it for ads or ordinary page content. I would consider enabling it selectively for an app that I carefully decided to trust.
* Admittedly I have not tried, but I suspect it's going to be very difficult to get it to work well from within a VM.
* HTML parsers and renderers have indeed had plenty of vulnerabilities. They have taken years to get as secure as they are today and we may not have seen the last of the exploitable bugs in them.
* HTML parsers have been implemented from the beginning to accept untrusted data from the internet. 3D graphics drivers, on the other hand, are designed primarily for performance in a scenario where they are run by a single-user game on the local machine, often with Admin privs already.
* We saw how many years it took Microsoft (the company that could reportedly turn on a dime) to 'get' security to the point that they could ship a secure browser. I don't see much evidence that graphics vendors are even thinking about it yet.
* Apple knows that WebGL bugs will result in jailbroken iPhones. Guess how many years their OpenGL support level is behind the PC...on the very same GPU hardware? Last I checked, someone reported getting OpenGL 3.2 working on a Mac. I've had usable OpenGL 3.3 on a freaking Linux laptop for a couple of years now. More powerful GPUs are up at 4.2.
* A buffer overflow in HTML does not automatically amount to a kernel level compromise, (some recent font handling bugs in Windows notwithstanding :-). In fact, the latest generation of sandboxed browsers are building defense in depth mitigations. Any buffer mismanagement between the GPU, driver, GL, and WebGL seems very likely to result in complete pwnage. The GPU can access host memory directly with no access permissions.
Don't get me wrong - I love the idea of WebGL and want it to succeed. I just would hate to see it end up like Flash, struggling for years to retrofit security onto something that wasn't originally spec'd for it and a bunch of its users getting pwned in the process.
Yes, but I've also had audio, webcam, scanner & IDE drivers BSOD my computers over the years as well. Drivers usually get better with time, but crappy drivers aren't unique to video cards.
Admittedly I have not tried, but I suspect it's going to be very difficult to get it to work well from within a VM.
VM software is getting better at passing 3D through to the native hardware, so it may be easier than you think.
HTML parsers and renderers have indeed had plenty of vulnerabilities. They have taken years to get as secure as they are today and we may not have seen the last of the exploitable bugs in them.
This is the story of software in general. As long as there will be software, there will be bugs, some of them severe. Doesn't mean you just accept it & not try to prevent them, but you also don't stifle innovation because of the fear of causing them.
3D graphics drivers, on the other hand, are designed primarily for performance in a scenario where they are run by a single-user game on the local machine, often with Admin privs already.
Not really, they've supported multiple 3D accessible contexts for years & years now. Also if you're running Vista or Windows 7 you are more than likely already relying on the 3D hardware to render your 2D desktop. Web browsers are already using features of 3D hardware to further accelerate non-WebGL content.
Apple knows that WebGL bugs will result in jailbroken iPhones.
They also know that they messed up on the antenna design of the iPhone 4. Apple is not immune to flaws.
Guess how many years their OpenGL support level is behind the PC...
This doesn't really matter because WebGL is essentially a copy of OpenGL ES 2.0, which is closely based on OpenGL 2.0.
The GPU can access host memory directly with no access permissions.
So can your CPU, what's stopping rogue apps from doing so? Software. WebGL mainly involves writing GLSL scripts that act on buffers & arrays of data. You are limited to what data types you can upload into the video card.
Perhaps there are more vectors for vulnerability, but I'd rather see WebGL take off as a transparent standard, than rely on Adobe. Plus it's performance could be much greater than sluggish Flash. I admit I'd like to see more granularity as far as control. Last thing I want is WebGL ads eating up my GPU/CPU/Battery rendering stupid ragdolls physics simulations.
Absolutely. But webcam, scanner, and disk drivers are not accepting attacker-chosen data from the internet. The audio driver may occasionally. I would be surprised if there had never been any remotely exploitable security vulnerabilities, particularly involving driver support of complex audio compression formats.
VM software is getting better at passing 3D through to the native hardware, so it may be easier than you think.
Yeah, this will be something to watch.
This is the story of software in general. As long as there will be software, there will be bugs, some of them severe. Doesn't mean you just accept it & not try to prevent them, but you also don't stifle innovation because of the fear of causing them.
The world's botnet and antivirus vendors thank you for your commitment innovation.
Not really, they've supported multiple 3D accessible contexts for years & years now.
Right, but to what extent have they been really tested as a security boundary?
http://www.google.com/search?q=malicious+shader -> 30 results. It is the dawning of a new era.
Also if you're running Vista or Windows 7 you are more than likely already relying on the 3D hardware to render your 2D desktop. Web browsers are already using features of 3D hardware to further accelerate non-WebGL content.
No, web browsers are not accepting shader code from websites and feeding it to the GPU.
>The GPU can access host memory directly with no access permissions.
So can your CPU, what's stopping rogue apps from doing so? Software.
A lot of software built on very carefully designed hardware memory protections that took decades to mature into the brittle systems we have now.
Out of curiosity, do any GPUs have no-execute page permissions?
And what stops a GPU from modifying the execute-disable bits on CPU-managed memory pages?
WebGL mainly involves writing GLSL scripts that act on buffers & arrays of data. You are limited to what data types you can upload into the video card.
GLSL is a Turing-complete language, so no, it is an unlimited data type. It is provably impossible to put limits on its behavior before you run it. The best you can do is run it with limited access to other memory space. But if that were implemented perfectly, video drivers would never crash.
Perhaps there are more vectors for vulnerability, but I'd rather see WebGL take off as a transparent standard, than rely on Adobe.
Well anything looks good if your standard of comparison is low enough. :-)
I admit I'd like to see more granularity as far as control. Last thing I want is WebGL ads eating up my GPU/CPU/Battery rendering stupid ragdolls physics simulations.
Noscript is your friend!
http://www.computerworld.com/s/article/9216542/Security_firm...
http://www.zdnet.co.uk/news/security-threats/2009/06/09/virt...
Yes, I often decline to run software. Even if I do "trust" the software and its author, running it may unnecessarily increase my attack surface.
Honestly your best bet is to turn your computer off & unplug it from the Internet. Sandboxes, VMs, BIOSes are all subject to bugs which could lead to security vulnerabilities. Wrapping yourself in pillows may make you think you're safer, but it's not necessarily true & often is too much effort for most normal users.
That's the same FUD I hear every time I tell someone "that particular feature set is beyond my risk/reward threshhold".
Here's the counter-argument to that:
Would you have your surgery at a hospital where the staff were playing Flash or WebGL Facebook games on the intravenous drug pump management console?
We all have decisions to make about risk vs. benefit. I make mine with different considerations than yours. This is OK.
This is quite a bit of FUD in it's own right. Would you use a computer who's only ability was to manage an intravenous pump? Apples vs Oranges. Even with medical devices being highly audited, not running Flash or WebGL & not being on the Internet, they still can end up with life threatening bugs.
Yes, I would prefer exactly such a thing for my IV pump.
> Even with medical devices being highly audited, not running Flash or WebGL & not being on the Internet,
There absolutely are many many medical devices running Windows XP embedded plugged in to hospital LANs around the country as we speak.
In fact, I used to work on such a system. It was a total PoS from a security perspective. The mechanical engineers who designed it seriously didn't expect anyone would plug it into the LAN. Maybe they've improved it since then, I don't know. I no longer associate with those people.
> they still can end up with life threatening bugs. http://en.wikipedia.org/wiki/Therac-25
Bonus points for knowing your history, but note that you're going back 25 years for such a bug.
Adobe, lets see ... http://web.nvd.nist.gov/view/vuln/search-results?query=adobe...
There we go CVE-2011-2460:
Summary: Adobe Flash Player before 10.3.183.11 and 11.x before 11.1.102.55 on Windows, Mac OS X, Linux, and Solaris and before 11.1.102.59 on Android, and Adobe AIR before 3.1.0.4880, allows attackers to execute arbitrary code or cause a denial of service (memory corruption) via unspecified vectors, a different vulnerability than CVE-2011-2445, CVE-2011-2451, CVE-2011-2452, CVE-2011-2453, CVE-2011-2454, CVE-2011-2455, and CVE-2011-2459. CVSS Severity: 10.0 (HIGH) Published: 11/11/2011
So I had to look back only 11 days for my example, and that's not to mention the 15 other Adobe bugs with CVEs published that week.
Still medical devices, like SCADA systems, do usually tend to suck because they're computer systems designed by people who've never secured a computer before. Guess what! Did you know you can kill someone wirelessly if they have a Medtronic insulin pump? http://www.reuters.com/article/2011/10/26/us-medtronic-idUST...
But that doesn't have much to do with my reasons for being wary of allowing WebGL.
How new? My MBA is only 2 or 3 months old and WebGL runs fine (and fast) under Chrome 17. Indeed, under the test on the parent link, I get about 500 objects on Canvas, 6000+ on WebGL.
This is a 11" MBA with 1.6GHz i5 and Intel HD Graphics 3000 384 MB. Browser is Chrome 17. OS is Lion.
The cynic says: App store! App store! Money money money!
Also Apple HAS added WebGL support to iOS, only hasn't enabled it yet. Tons of reasons exist: battery issues, the implementation not being ironed out yet, and the security risk of WebGL.
Apple cares about the experience of using an iPhone, and if WebGL games on mobile safari suck the batter dry, stall or even crash, they aint gonna enable it just for to add one for mark on the iPhone spec sheet.
Well, if you're going to evaluate it that way then surely you also have to account for the game logic running much slower in JavaScript than in C++.
The bottom line is: If you care about graphics rendering, C++ wins by a large margin. And if you care about game logic, C++ wins by an even bigger margin.
For more complex rendering, JS will be faster, relatively, because the GPU will have to do more work for each batch of vertices you send over.
For game logic, JS will be faster, relatively, because going through the foreign function interface is actually somewhat slow. You have to pin GC'ed objects onto the stack, etc. Moreover, calling into code the JIT can't see can disable certain optimizations.
If you look at certain FP micro benchmarks, like nbody and spectrum-norm in the Shootout, V8 is within 4x of gcc -O3 on nbody, within 3x on spectral-norm when using gcc's vector extensions, and dead-even with gcc on spectral-norm when not using gcc's vector extensions.
The places where V8 still falls down are property access (the prototype OO system forces some overhead) and lack of vectorization. The former can be addressed by using an SoA layout rather than an AoS layout, and if your code is amenable to vectorization you should break out the assembler anyway.
I've just always held out hope that once the hardware is fast enough, side-scrollers would look just as crisp scrolling as they do standing still.
And interestingly, most newer IPS panels have a gray-to-gray time similar to TN, around 4ms (60fps = 16ms/frame). You can't tell anymore if a display is good or not just by it's specs, each one is different.
They seem like a slightly better TN, but still far away from IPS quality, specially for photography/video.
I also notice your "C++" version used Direct3D. Shouldn't you have used OpenGL with the exact same calls as the JS to get a fair comparison with the WebGL versions?
Edit: also our intent was to compare real game engines, which necessarily add some performance overhead, rather than a thin experiment.
That said, the webgl performance in chrome is indeed (relatively) impressive. Am also a macbook air user and have no trouble getting good performance on it.
One point to note is that the webgl engine in chrome runs all graphics in a single process, atleast partly for reasons of security (shader code gets checked by the browser before being set to the gpu, for example). That is sort of like a driver on top of a driver and so there will always be like a 2x performance gap between webgl and c++ i think.
Still ... the future looks bright.
The game logic will likely be faster by a similar factor, so you could quite possibly get an 18x increase in game logic capacity.
The development Opera does great, but the development versions of the other browsers are likely also much faster than their stable versions.
A test that stayed within JS would optimize better and perform better relative to C++.
It would be a good time to do it, too, because otherwise Mozilla and Opera will do it anyway next year.