Announcing Testling - Automated Cross-Browser JavaScript Testing | Hacker News Reader

Announcing Testling - Automated Cross-Browser JavaScript Testing

catonmat.net

125 pointscleverjake28 comments

Open articleView on HN

Let's make an n-dimensional map of the available javascript testing environments. The dimensions are something like:

- headless v. javascript driving browser v. full browser-level

- js-only v. test-specific html page v. load full app html

- local v. cloud

And the current contenders go something like:

- Selenium: browser, full app, local

- Sauce: browser, full app, cloud

- FuncUnit: js-browser, full app in iframe, local

- qUnit: js-browser, sparse html page, local

- JS Test Driver: js-browser, js-only, local

- ad infinitum...

The result is that you need (at least!) three different tools to do three different but required tasks:

- unit tests that run in a quick-feedback, autotest style on the command line

- module-level tests that require a small bit of DOM access and cross-browser testing

- integration tests that require loading the full app and triggering events on it.

It would be better if we could come up with a convenient, simple abstraction that allowed doing all three, and sharing code for mocks across the three, and configuring in what environment and what machine they each run.

Ideally, this could be done with AMD and wire.js. Just write a test in javascript, declare its dependencies, and they get injected.

- Unit tests depend on just one or two js files. They get loaded and then the tests run.

- Functional tests depend on "dom," provided either by a browser or a zombie.js-style headless dom, and a variety of js and html snippets. They all get loaded and run.

- Integration tests depend on "app", provided by actually running against one of your servers. It spins up, the tests run, maybe there is an API for telling the test server to reset itself to a known state between tests.

We pretty much have what you're describing already, except we use node.js-style require()s with browserify instead of AMD. You can write really simple unit tests and just execute them with a curl one-liner or you can write more integration-style tests for an existing app with t.createWindow() and t.submitForm().

Interesting. But how do I run tests with t.createWindow locally? Do I need a publicly-accessible IP? We are currently looking at Sauce because we can write tests against Selenium and run them locally or in the cloud, plus they make it easy to tunnel in and run against things in our internal network.

Right now, you do need a public IP for t.createWindow(). We have ssh tunnels in browserling already but they don't work with testling just yet. Stay tuned.

I think one of the biggest benefits of testling right now is that you don't have to install anything outside of curl to start using it.

Fantastic, I look forward to it. I started this post mostly frustrated by the idea of someone creating "yet another" js testing tool, but now I'm legitimately interested in seeing where this goes. Thanks for the fast and helpful feedback.

As an aside, the tunnel scenario is currently something that has been solved by a few other startups including:

  * Browserling - http://www.catonmat.net/blog/announcing-ssh-tunnels-for-browserling/
  * BrowserStack - http://www.browserstack.com

Clickable:

http://www.catonmat.net/blog/announcing-ssh-tunnels-for-brow...

http://www.browserstack.com

IMO if you're going to be publishing some form of cloud SaaS for testing, this feature is a must.

I personally don't like my broken code leaving the confines of the office (or opening up any outside ports).

Testling is our newest product here at browserling, so we definitely know how to make tunnels work. We just wanted to get testling launched quickly is all; tunnels are coming soon.

I think you can get closed to accomplishing all your testing needs by relying on qt-webkit now, alhtough for unit you may still like a simple js interpreter. phantom.js is the main qt-webkit tool.

"Close" is not the same as "actual". qt-webkit tests have their place (they're FAST!), but real tests on real browsers which real users use (aka "Last Mile" testing) has its place, too. It's all part of a balanced breakfast.

Looks very useful as a github post-commit hook for catching browser-specific JavaScript errors.

Is this also meant to be used for testing usability across browsers? Like browserling but automated? If so, some specific examples on the docs page would be very helpful.

Yep you can use it both ways. To drive an existing page around you can use t.createWindow() and t.submitForm(), which give you a new window object and a jquery handle bound to that window object. http://testling.com/docs/#an-example-test

Woot! Congrats on the launch. Time to add Testling to the StackVM site, eh? :D

Thank you! Yep, got to add it there.

Edit: added! http://stackvm.com

Great products, Poor designs. The last part takes a lot away from the first, making it almost look as if this is a side project. Just saying.

I strongly disagree, and find the hand illustrated approach incredibly refreshing amongst the sea of cookie cutter web apps.

I've heard people say this about browserling, and I can kinda see the point, but I thought testling's landing page was kinda pretty. Maybe I'm just weird.

You are weird. Just kidding. It just kind of hurts me a little bit to see a great product come short in design.

Edit: Best HN username goes to you.

> Best HN username goes to you.

> It just kind of hurts me a little bit to see a great product come short in design.

I do have to wonder what the design would look like were a "pro designer" to get their hands on it. I think SubStizzy did a pretty admirable job, but man design is hard. I know I'm pretty terrible at it anyway.

Can I run jasmine on testling?

The plan is to support other test styles like jasmine and qunit through npm modules: http://testling.com/docs/#using-npm

How do I run tests locally without using this service? If I could run these same tests most of the time with qt-webkit (phantom.js) and then run on testling before deploying I would use this.

For other NodeJS JavaScript testing options, there's also Jellyfish (http://jelly.io/)

Congrats on launching! Welcome to the arena. :-)

Hey thanks! The business of making the web a little less terrible and a little more reliable is a great space to be in.

Nice work.

What I would like to see is automated testing on different mobile browsers (on real devices).

Anyone have pointers for such tools? Thanks.

Great news guys! I'm going to make sure my team finds out about Testling.

Node.JS!

You bet!

Reply on news.ycombinator.com