Comment by asolove | Hacker News Reader

Comment by asolove | Hacker News Reader

Let's make an n-dimensional map of the available javascript testing environments. The dimensions are something like:

- headless v. javascript driving browser v. full browser-level

- js-only v. test-specific html page v. load full app html

- local v. cloud

And the current contenders go something like:

- Selenium: browser, full app, local

- Sauce: browser, full app, cloud

- FuncUnit: js-browser, full app in iframe, local

- qUnit: js-browser, sparse html page, local

- JS Test Driver: js-browser, js-only, local

- ad infinitum...

The result is that you need (at least!) three different tools to do three different but required tasks:

- unit tests that run in a quick-feedback, autotest style on the command line

- module-level tests that require a small bit of DOM access and cross-browser testing

- integration tests that require loading the full app and triggering events on it.

It would be better if we could come up with a convenient, simple abstraction that allowed doing all three, and sharing code for mocks across the three, and configuring in what environment and what machine they each run.

Ideally, this could be done with AMD and wire.js. Just write a test in javascript, declare its dependencies, and they get injected.

- Unit tests depend on just one or two js files. They get loaded and then the tests run.

- Functional tests depend on "dom," provided either by a browser or a zombie.js-style headless dom, and a variety of js and html snippets. They all get loaded and run.

- Integration tests depend on "app", provided by actually running against one of your servers. It spins up, the tests run, maybe there is an API for telling the test server to reset itself to a known state between tests.

We pretty much have what you're describing already, except we use node.js-style require()s with browserify instead of AMD. You can write really simple unit tests and just execute them with a curl one-liner or you can write more integration-style tests for an existing app with t.createWindow() and t.submitForm().

Interesting. But how do I run tests with t.createWindow locally? Do I need a publicly-accessible IP? We are currently looking at Sauce because we can write tests against Selenium and run them locally or in the cloud, plus they make it easy to tunnel in and run against things in our internal network.

Right now, you do need a public IP for t.createWindow(). We have ssh tunnels in browserling already but they don't work with testling just yet. Stay tuned.

I think one of the biggest benefits of testling right now is that you don't have to install anything outside of curl to start using it.

Fantastic, I look forward to it. I started this post mostly frustrated by the idea of someone creating "yet another" js testing tool, but now I'm legitimately interested in seeing where this goes. Thanks for the fast and helpful feedback.

As an aside, the tunnel scenario is currently something that has been solved by a few other startups including:

  * Browserling - http://www.catonmat.net/blog/announcing-ssh-tunnels-for-browserling/
  * BrowserStack - http://www.browserstack.com

Clickable:

http://www.catonmat.net/blog/announcing-ssh-tunnels-for-brow...

http://www.browserstack.com

IMO if you're going to be publishing some form of cloud SaaS for testing, this feature is a must.

I personally don't like my broken code leaving the confines of the office (or opening up any outside ports).

Testling is our newest product here at browserling, so we definitely know how to make tunnels work. We just wanted to get testling launched quickly is all; tunnels are coming soon.

I think you can get closed to accomplishing all your testing needs by relying on qt-webkit now, alhtough for unit you may still like a simple js interpreter. phantom.js is the main qt-webkit tool.

"Close" is not the same as "actual". qt-webkit tests have their place (they're FAST!), but real tests on real browsers which real users use (aka "Last Mile" testing) has its place, too. It's all part of a balanced breakfast.

Reply on news.ycombinator.com