r0s

I'm primarily focused on automated testing. Which is great but all revolves around test data "fixtures", standardized test data that is integrated with the system under test to be sure all the tests are using correct data.

My company is large, and coordinating all this data was typically done over email or jira.

I designed and built a test data API service with a web UI, now in heavy use across many departments and three business units.

Good stuff. I especially appreciate the entreaty to limit assertions and corner cases to one test concern.

The only thing I didn't like was the Given When Then recommendation. BDD style story language is really meant for key specification examples, which can be automated in functional tests, but not really appropriate for unit testing.

I appreciate the conversation! That's interesting about RSpec, I'll read more about it, thanks.

> it's hard to properly define when the system under test behaves unexpectedly...

I feel your pain. I think at that point it's good to look deeper and ask if exceptions are being properly thrown, as I try to call for in the final section on Exceptions.

You can't plan for everything.

> You could design assertions in a very sophisticated way to minimize that problem, and optimize amount of information... but that's complex and brittle and time consuming.

My point here is this optimization is really pretty simple. One target state per test, and soft assertions for the rest. It's actually a simplification rather than adding complexity.

Soft asserts make your tests more robust by definition, lots of halting assertions break more often, they are brittle by design.

> If you have a decent test coverage with well written tests ... you're golden

Hey if your tests catch all bugs, it ain't broke, don't fix it!

Totally understandable if you aren't looking for optimizations like this. For the massive test suites I work on, it's necessary to at least consider these concepts.

Short answer: Not all failed test coverage is invalid.

Granted, unit tests benefit less from soft assertions, but I'll take a shot at an example anyway:

Consider a complex object returned from a unit test result (this would be a single target state). That structure could have a missing field that should be reported, but the rest of the fields are evaluated fine, and so that information would be lost if the first assert halted the test.

For functional tests, the value is usually easier to discern, so for another contrived example:

A login form has a failed assert on it's UI structure, say a button is the wrong color, but the login is successful, and the following assertion post-login is also successful.

Ideally, you would assert post-login tests in separate target state, perhaps mocking the login as a pre-condition to another test. For functional tests that could dramatically increase the testing time and complexity, and so this makes a non-halting assertion valuable.

Interesting, it sounds like you would benefit from breaking each test out into one that focuses on a single target state.

Another layer would be conditional test execution hierarchy where one early failure would skip a set of tests that depend on that state.

> If the test is in a invalid state then you can't trust soft asserts that pass.

Agreed, which is why I suggest designing each test for a single target state.

That target may then legitimately benefit from multiple assertions, and you have the same problem.

If one assertion fails early in a test, you lose any subsequent coverage from further assertions.

Maybe you can build your tests such that there's only one assert for each, and if that meets your needs, fine.

If not, or if you're in the process of refactoring your tests into single assertions, this way we can reclaim that coverage.

Yes! I'm stoking the flames for these ideas because they were quite beneficial in my experience.

In particular, the code reuse aspect, which I haven't seen any advocacy for previously.

Thanks, I'll check it out.

If you architect each test to have a single assertion, yes, I agree it's essentially the same effect.

Hey Hacker News, I'm Ross Radford from Austin Texas.

I've been a Senior Engineer in Test for long enough to care about this subject, and I think we'd all be better off using soft assertions, and less assertion in tests generally.

Hey Hacker News, I'm Ross Radford from Austin Texas.

I've been a Senior Engineer in Test for long enough to care about this subject, and I think we'd all be better off using soft assertions, and less assertion in tests generally.

This feels like a Wired article from 2003.

Supply has never been the problem with feeding the world.

More supply solves nothing.

Talk to your potential customers. Maybe you have doubts that are very real reasons you shouldn't start.

On the contrary, there's not a lot of money to be made, and people constantly question it.

Would-be spoilers get educated about their own unrealized bias, racism continues to be a huge problem in this country, activists are vindicated and the world moves on.

> Suppose you're right.

Easy to verify, why suppose anything?

> censorship?

On a privately owned site? Try again.

Try gitlab.

Such as parenthood?

For me the hardest part of management is letting go of control, especially when the habit of asking the lead for advice is necessarily ingrained in junior developers.

What works for me is getting in tune to my annoyance and identifying opportunities for eliminating some task I'm doing that the junior dev should be in charge of. (peer code reviews, for example)

Like a transporter on Star Trek, I'm afraid there would be no upload without a copy. Could we really then delete the original?

Cognitive science and human computer interface research.

1) Every problem you're working on will be improved by better learning systems.

2) The direct challenger to AI's potential is human augmentation. There's an evolving hardware component to that, but the software needs to keep up, and we can greatly benefit from that now.

We're still reading, thinking and learning with computers as if they were books. That has to change. I want to fully realize that potential, I consider it the most valuable contribution I can make in my lifetime. I would jump at the opportunity to fully invest myself in that goal. Please contact me for a deeper conversation.

The smugness thing really hits home. Sometimes clueless, overconfident or downright negligent people are so embedded in a culture that an outside expert opinion is exactly what's needed.

Thanks for the reply.

I think the arguments against penetration testing are interesting.

As a software test engineer, we do external audits like this, but I wonder: Documented or "known" vulnerabilities are not worth testing until there are systems in place that expect to cover them.

It seems that all vulnerabilities that are not intentionally addressed should be considered dangerous. If they are penetration tested but vulnerability is unknown, the best you could hope for is "Not vulnerable for unknown reasons" which is just as bad as vulnerable in my perspective.

With some admitted trepidation, I assert that all software should be tested this way, with expected behavior being a primary dependency.

Edit: hiring outside penetration testers is still totally valid and desirable, since development and testing are two totally different domains. I'm only pondering methodology.

I use a mix of stylish themes to achieve something similar, and hide Slack's grating UI and default colors.

Payday loans are a major target for identity theft. This kind of fraud is insidious, it can screw up your credit for years.

From the article: "Almost a third of online payday borrowers surveyed by Pew reported that their personal or financial data was sold without their consent.78 Nearly as many reported unauthorized bank withdrawals in connection with an online payday loan.79"

Payday loans have to register new accounts in national databases. My company monitors for payday loans taken in your name: https://qidreports.com

> The move potentially puts Mexico at the forefront of an international movement to decriminalize drugs – despite a decade-long militarized crackdown on drug cartels which has cost the lives of around 100,000 people.

This is a sane and direct response to exactly that.

It was. On the other hand I feel like I know exactly my skill level and limitations.