It's here now: https://preflight.com/pricing
36 karma · joined March 1, 2015
Which competitors? Where are they running the tests on? Do they adapt to changes and save your valuable time? Or charge small and run tests with selectors like: .wudfj289hf .a9dfh82hf .sadf98h which breaks each time you deploy?
We want to make sure we help you. Testing is not easy even though we try everything we can to make it easy.
We don't obfuscate, we care deeply about our clients. I always say this. If Preflight doesn't add value you don't have to pay for it nor use it.
It should be no problem with MetaMask.
Edit: Also listened to everyone and put the pricing on the web page: https://preflight.com/pricing
We can access to environments behind the VPNs. But it's hassle-free if it's accessible through the internet.
Actually we also implemented Context awareness for SPAs with CSS Module to adapt to the changes. We are agnostic to HTML changes and if we can find the element we will update the selectors after the test is successful.
Thank you very much for the kind words!
Absolutely, integrating with CI/CD is super easy. We have some integrations already with major CI/CD pipelines. You can find it right here after signing up: https://app.preflight.com/account/integrations
I was the first engineer at ShipBob and ended up hiring over 40 engineers. We had a lot of UI issues after we grew beyond a team of 5 or so. With all the engineers adding new code to the existing code, everything started to break.
That's when we started writing Selenium tests. I and several other engineers wrote 100 Selenium test cases in 2 months. But we were not able to maintain them in 3- 4 weeks because we had other responsibilities, and our Selenium test cases soon went to trash.
The problem with automated UI tests with Selenium or Cypress is that they depend on the internals of the design (e.g. via CSS and XPath selectors), resulting in tests that break frequently because of unrelated changes. When that happens, you either have to fix all the tests and run them again, or replace the broken automated tests with manual testing.
Then I hired 2 QA engineers, they took 5 months to come up with 5% test coverage and we needed more testing so we couldn't add more to it and maintain those tests as well. These issues pushed me towards creating a tool for automated testing that could understand the context of the page—the way humans do, not the way browser engines do—and can adapt to changes based on each steps' context. So if the UI changes completely and the test is looking for a Password Input in a sign-up form or an Orders menu item in the menu, we will find it without any related selectors.
As humans, we understand the page in terms of buttons, menus, headings, borders, and so on. I wanted to mimic that. On top of that we have implemented a way to understand navigations through these different elements with their context, again, as humans do. We call this Context-Awareness. It’s our secret sauce to adapt to the changes and also calculate what your test coverage is. Since we understand the context of each step, it’s pretty easy to see if, say, a given button is in the tests.
Here’s a video that shows the Context Awareness aspect I just mentioned: https://www.loom.com/share/50426244253943e49906dd881e0fb7d7.
You record your tests using our browser extension, and then we run them on a bunch of Windows 10 machines that access your web app over the internet and execute the tests you recorded. We’re working on adding Macs. (Fun fact: Selenium cannot handle more than 40 nodes. The first version of Preflight was depending on Selenium Grid and it stopped working when we wanted to add more than 40 machines to it.)
We charge money based on how many tests you are planning to run. We don't want to limit you based on server time because it's not easy to predict how much server time your tests will take. So everything is based on test results.
Anyone can sign up and start using Preflight in minutes after signing up at https://app.preflight.com/get-started.
Thank you for reading. I'd love to hear your thoughts and help all of you with any of your QA needs!
We also build no-code testing tool to help people get testing in place in minutes vs months of selenium or cypress. Check it out: https://preflight.com
I also recommend using Preflight with Bubble to test your web application. Since it's no-code there is no way of automating unit tests.
Hope this is useful. Cheers!
At ShipBob, we had a lot of UI issues when we scaled from 5 engineers to 30+ engineers. With the new engineers, we released bugs. Because even though we tested our parts the other parts of the software were breaking.
So, that's why we were so close with customer support. It was a little frustrating to develop and fix at the same time. Because we had to fix the issues right away.
And then I thought of building a tool where everyone can create tests by simply recording actions once. So people who are experiencing the same issues that I had would have peace of mind by testing the existing functionalities.
So, I built Preflight to save time who are suffering from UI that breaks or the server that stops responding. It can be a Product Manager or Manual QA who wants to create the tests to save time for themselves and developers. And with Preflight it's easy to communicate with developers what's wrong and you would not release bugs to your users.
When something is wrong, we detect console log errors and network errors on the page.
The problem with Selenium, Puppeteer, etc is they all push UIEvents on browsers to hack it to work as actual mouse and keyboard interactions. That's why most of these frameworks feel awkward and cannot mimic the actual user's behavior. We now are working on how your users use your product with programmatically controlled actual keyboard and mouse interaction.
We are also working on understanding the context of a test and we are going to be able to run the tests based on context not only based on selectors. For example: add a product to the cart, then go to the checkout page.
Thank you for reading. I'd love to hear your thoughts and help all of you with any of your QA needs.