Hacker News Reader

Call me crazy, but I go even more verbose than that. I go full Rspec/BDD mode:

  class TestAuthenticateMethod:
      
      class TestWhenUserExists:

          class TestWhenPasswordIsValid:

              def test_it_returns_200(self, app, user):
                  assert ...

          class TestWhenPasswordIsInvalid:

              def test_it_returns_403(self, app, user):
                  assert ...

      class TestWhenUserDoesNotExist:

          def test_it_returns_403(self, app, user):
              assert ...

It is so worth it. I can look back at my tests 3 months later and know exactly what was intended wthout duplicating explanations with comments. Yes some methods can get a bit long and indented, but it's worth it for me. The other downside is you have to start all classes/methods with "test" but I got used to that very quickly.

When you run pytest it's very clear what's being tested and what the expected outcome is since it gives you the full class hierarchy in the output.

This idea is both crazy and genius. Thanks for sharing.

I’ve been working on a codebase like this, and my major complaint is that I’ll see a deeply-nested test and have to scroll waaaaay up to understand the context. I prefer ‘foo_bar_baz’, ‘foo_bar_qux’ to

    foo
        <snip 1000 lines>
        bar
            <snip 553 lines>
            baz
            <snip 243 lines>
            qux

This is neat, but pytest allows for similar granulation using the right fixtures:

E.g. in test_authenticate_method.py:

    def test_returns_200_on_valid_password(app, user_exists):
        ...

    def test_returns_403_on_valid_password(app, user_exists):
        ...

    def test_returns_403(app, user_not_exist):
        ...

We don't need to put this much responsibility on test names. If there is more to explain, write a few words in a comment.

You don’t see comments in a test report though — maybe have an optional description as part of the framework for more detail along with -v

I think this is one of those conventions where your team agrees on one convention, and you just try to follow it consistently. If that's looking at the test in code to find descriptive comments, great. If that's long test names, cool. Just do the same thing consistently.

True - I think it depends on who uses your test outputs. If PMs and BAs care, having a more verbose output (descriptions, pretty graphs, etc) helps a lot. If it’s just for devs then they can more happily go through the code base.

I know that the docstring in Unittest is part of the reported output, and I was pretty sure that it's the same in Pytest.

I would've thought so but no, pytest will show the docstring in `--collect-only -v` (badly), but it doesn't show any sort of description when running the tests, even in verbose mode (see issue #7005 which doesn't seem to be very popular)

`unittest` prints the docstring alongside the name of the test in verbose mode (e.g. failure).

pytest does not though.

I've just never seen the "this belongs in comments" approach work out in practice. Maybe it's something about human psychology. Perhaps things might seem obvious when you've just written the test, so you don't think to comment them. Perhaps code reviewers feel less comfortable saying, "I don't understand this, could you please comment it?" Perhaps it's the simple fact that optional means optional, which means, "You don't have to do it if you aren't in the mood." Regardless of the reason, though, it's a thing I've seen play out so many times that I've become convinced that asking people to do otherwise is spitting into the wind.

I really hate, but I can see its virtue, the pylint rule that complains about lacking docstrings.

In tests, however, I prefer not to have them as my favorite test runners replace the full name of the test method with its docstring, which makes it a lot harder to find the test in the code.

My only issue is that the rule doesn't give you a good way to annotate that the object is documented by its signature. Sometimes devs are lazy, but the rule doesn't make them not lazy so you get pointless docs like:

    def matriculate_flange(via: Worble):
        "Matriculates flange via a Worble."

That's more or less why I hate this feature - it encourages lazy docstrings. Right after you named something is probably the worst possible time for you to explain it.

All this is true, but doesn't it apply just as much to writing descriptive test function names?

Test function names are less sensitive than regular functions, since they're not explicitly called, but I still don't want to read a_sentence_with_spaces_replaced_by_underscores.

I don't want to either, but until Python gives us backtick symbols or we something like Kotlin's kotest that lets the test name just be an actual string, that's sort of the choice we're left with. And I'm inclined to take the option that I've known to lead to more maintainable tests over the long run over the option that I've known to engender problems, even if it is harder on the eyes. Form after function.

As far as whether or not people do a better job with descriptive test function names, what I've seen is that they do? I of course can't share any data because this is all in private codebases, so I guess this could quickly devolve into a game of dueling anecdotes. But what I've observed is that people tend to take function names - and, by extension, function naming conventions - seriously, and they are more likely to think of comments as expendable clutter. (Probably because they usually are.) Which means that they'll think harder about function names in the first place, and also means that code reviewers are more likely to mention if they think a function name isn't quite up to snuff.

And I just don't like to cut those sorts of human behavior factors out of the picture, even when they're annoying or hard to understand. Because, at the end of the day, it's all about human factors.

I don't disagree with much of this.

I was talking specifically about a `def test_displays_error_message_when_refresh_times_out()` function.

That's too big a name for me to keep in my head, so I'd look for other solutions.

Why would you want to keep test function names in your head? I would rather use that space for regular functions which are in use throughout my program. Test functions are usually single-use anyways so being greedy with characters makes much more sense elsewhere. The most important thing to remember about a test function is what property it tests (and how). Putting that in the function name as a reminder makes a lot of sense imo.

I would read that any day over a function called test() where I have to figure out what is actually being tested.

Me too!

I've been speculating that the right place for the "why" of tests to live is in documentation, where references should be hung like citations (which may or may not be included in any given rendering of the documentation in question) supporting claims of fact. Then we can surface that any time a test fails, and find tests that no longer check anything we care about.

It's a choice of course, but I look at test functions like other functions. If you can name a function such that it doesn't need a comment (but also doesn't use 100 characters to do that), I'll gladly take that over a comment. Same like in all other code: if you need comments to explain what it does, it's likely not good enough and/or needs to be put in a function which tells what it does. Comments on why it does stuff the way it does are of course where it's at.

s/comment/docstring/

I have yet to understand the point of docstrings.

How are they, in practical reality, better than comments?

The difference is in tooling. Docstrings are collected and displayed in many contexts. The intended purpose is writing “documentation” in the same place as code. Comments are only seen if you open the source file.

OK, I can see how that's useful for a certain workflow.

The way we work, it's just a different comment syntax. I do like have a dedicated place for it.

Python's builtins, the standard library, and many popular community libraries and tools use docstrings to generate tooling and documentation websites. Describing it as "useful for a certain workflow" is so understated it borders on false. The reverse is more true: docstrings are not useful for a certain workflow, but most Python programmers will find them quite helpful.

All Python programmers already benefit from the docstrings written in the libraries they use, which generate tooltips and documentation websites.

you can access it from object.__doc__, and there is tooling in ide's like pycharm to quick view them to see what a function does, auto generated documentation

prior to mypy/type hints it allowed you to document the types of a function

Sometimes there are practical reasons to avoid this approach and have fewer test methods that test multiple behaviors of a single thing. For example a lot of our projects setup and teardown a database between test methods so a few fatter test methods run a lot faster than a large number of small test methods. We rely on good commenting within the methods to understand what exactly is being checked.

In that case, why not use some auxiliary function to load the resources (in your case, the database) and decorate it with functools.cache[0] to avoid the function from getting executed multiple times? Sure, this means re-using resources between multiple tests (which is discouraged for good reasons) but your current test effectively does the same thing, the only difference being that everything is being tested inside one single test function.

PS: How come your setup and teardown operations are so expensive in the first place? Why do you even need to set up an entire database? Can't you mock out the database, set up only a few tables or use a lighter database layer?

[0]: https://docs.python.org/3/library/functools.html#functools.c...

The difference is that it's very obvious in our current tests when the database is being reset. And to be clear it's not like we're testing the entire app in a single test function - our test functions just tend to cover more than one feature.

We've also found over the years that if you want to be sure everything works on version X of database system Y, you run your CI tests on that.

One of the most common test modules in R is called “test that” and you invoke a test by calling a function (rather than defining one) called “test_that”, the first argument is a string containing a description of what you want to test and the second argument is the code you want to test.

That way, all your unit tests reads: “test that error message is displayed when refresh times out” etc.

I think it’s a really nice way to lay things out and it avoids all the “magic” of some functions being executed by virtue of their name.

This. Like probably most programmers, I tend to read code more than I write code and when I read tests, I want to know immediately what a test does and checks. I don't want to read the test itself, unless I already know it's of interest to me.

In my experience, when a unit test has no clear name, this is usually a sign that the test does too much (meaning that it's actually not a unit test anymore) or even comprises multiple tests, leading to all the known bad consequences this has (leaking state between tests, unclear test assumptions etc.)

I think that behave (https://behave.readthedocs.io/en/stable/) is more useful for testing these more real-life usecases. I tend to have a `test_my_function` in pytest tests and the more integration and functionality-related testing in the behave tests.

I usually put that info in the asset:

Assert foo(), "foo should bar"

So that each failure has a clear meaning.

Indeed, i may have several assert in a test.

have you considered using "test_refresh__failure" instead? makes clear "refresh" is the component being tested and "failure" is a description of the behavior