variedthoughts

Python makes it easy to build simple tools for all kinds of tasks. And it's great to be able to share small projects with others on your team, in your company, or with the world.

When you want to take a script from "just a script" to maintainable package, there are a few steps, but none of it's hard.

Also, the structure of the code layout changes to help with the growth and support.

Instead of just talking about this from memory, I thought it'd be fun to create a new project and walk through the steps, and report back in a kind of time lapse episode. It should be fun.

Here are the steps we walk through:

0.1 Initial script and tests

0.2 build wheel with flit

0.3 build and test with tox

0.4 move source module into a package directory

0.5 move tests into tests directory

Some developers either don't write tests, or don't like writing tests. Why not? I love writing tests. In this episode we examine lots of roadblocks to testing, and start coming up with solutions for these.

* quick overview of what test instruments are * discussion of API and communication with instruments * techniques for shifting complexity out of test cases

These techniques should apply to all test suites dealing with complex systems:

* Creating test cases that are easy to read and debug and tell a story about what is being tested. * Pushing setup complexity into fixtures. * Pushing lengthy repetitive API call sets into helper functions. * Using stable, documented, interfaces.

Some of what we discuss:

- overcoming the fear of public speaking - breathing and pausing during talks - planning your talk as well as planning your time to get ready for the talk - writing proposals and getting feedback on proposals - Nina's talk in PyCascades on programming Adafruit chips - types of talks that are often rejected - pre-recording demos to avoid live demo problems - why you should speak, even if you are an introvert - benefits of public speaking

The Pragmatic Bookshelf published it's most important book, in my opinion, in 2017 with the first pytest book available from any publisher.

Topics:

The Pragmatic Programmer, the book The Manifesto for Agile Software Development Agile methodologies and lightweight methods Some issues with "Agile" as it is now. The GROWS Method Pragmatic Bookshelf, the publishing company How Pragmatic Bookshelf is different, and what it's like to be an author with them. Reading and writing sci-fi novels, including Conglommora, Andy's novels. Playing music.

I talk about: - statistics for the show, including download counts, etc. - how many episodes are about testing, how many about non-testing topics. - Python vs language agnostic topics - solo vs guest episodes - the top 5 episodes - stuff like that

And then a retrospective like:

- what went well - what was lacking - what's next

I'd like feedback on what you think went well, was lacking, and where to take the podcast.

Discussed:

flit Poetry tox Continuous Integration setup.py, MANIFEST.in, etc. pipenv what's with lock files applications (doesn't go on PyPI) vs libraries (goes on PyPI) workflows dependency resolution deployment dependencies vs development dependencies will lock files be standarized multiple lock files requirements.txt

tests for large systems, like kubernetes Testing in real world scenarios with all the configuration and everything Becoming a complete engineer by thinking about the end to end flow from the users perspective Learning from other roles, and the value of roles to allow focus and expertise We even get into Chaos Engineering and testing live systems.

New ways to teach programming include automated tests from the beginning.

Trey Hunner is a former PSF director, a Python and Django team trainer, and he has been using automated tests to help people learn Python.

We cover:

- My approach to testing existing systems. - Put names to strategies so we can refer to them later. - Explain the strategies in general terms and explain why they are useful. - Discuss how these strategies are used in an example project. (The code is available on github).

Strategies covered today:

- Dog Fooding - Exploratory Testing - Tracer Bullet Tests - Act Like A Customer (ALAC) Tests - Manual Procedures - Initial automated tests at 2 levels, API and UI.

Some of the topics we discuss:

* experimentation vs testing * testing pipelines and pipeline changes * automating data validation * property based testing * schema validation and detecting schema changes * using unit test techniques to test data pipeline stages * testing nodes and transitions in DAGs * testing expected and unexpected data * missing data and non-signals * corrupting a dataset with noise * fuzz testing for both data pipelines and web APIs * datafuzz * hypothesis * testing internal interfaces * documenting and sharing domain expertise to build good reasonableness * intermediary data and stages * neural networks * speaking at conferences

* the Testing Pyramid * the Testing Column * TDD * unit testing * balancing unit with system tests, functional tests * API testing * subcutaneous testing * customer facing tests

Topics include:

* How M. Scott Ford got into forming a company that works on legacy code. * Technical debt * Process debt * Software testing * The testing pyramid * iterative development * kanban * readable code and readable test code

* Python Functions aren’t what you think: Functions are objects. The name of a function is just a variable name referring to the object.

* The Tao of Python: Jupiter notebook discussion of the relationship between objects, classes, and metaclasses.

* Why Learn Python? Here Are 8 Data-Driven Reasons

* Testing Python Applications with Pytest

* A million requests per second with Python

* RethinkDB is alive and well

* Python Top 10 Articles for the Past Year (v.2017)

* Red Hat Software Collections for Python 3

* Pipfile, pip, and virtualenv * Django 2.0 dropping Python 2.7 support * attrs for cleaner classes * Python benchmarking and profiling * Getting Python 3 into more distributions * Home automation with Home Assistant

Some of the topics covered

- How did Robert become the maintainer of unittest? - unittest2 as a rolling backport of unittest - test and class parametrization with subtest and testscenarios - Which extension to unittest most closely resembles Pytest fixtures? - Comparing Pytest and unittest - Will unittest ever get assert rewriting? - Future changes to unittest

Joe has spent his career in startups. He’s also been involved with hiring and talent acquisition for several startups.

We talk about testing, continuous integration, code reviews, deployment, tolerance to defects, and how some of those differ between large companies and small companies and startups.

Then we get into hiring. Specifically, finding and evaluating good engineers, and then getting them to be interested in working for you.

If you ever want to grow your team size, you need to listen to this.

But be sure to listen to the last 5 minutes. I’ve got some bonus audio that didn’t quite fit into the rest of the interview that I tacked on at the end.

How pytest, unittest, and nose deal with assertions.

The job of the test framework to tell developers how and why their tests failed is a difficult job.

In this episode I talk about assert helper functions and the 3 methods pytest uses to get around having users need to use assert helper functions.

- Communicate the purpose of your test more clearly - Focus your thinking while writing the test - Make test writing faster - Make it easier to re-use parts of your test - Highlight the assumptions you are making about the test preconditions - Highlight what outcomes you are expecting and testing against.

We discuss:

* testing django with unittest and pytest * layers of testing (test pyramid) including functional testing with Selenium, integration tests, and unit test. * where tests should live * test isolation * testing by development team without a QA group * double loop TDD * having Harry's book available for free

I know the main thread is a very serious topic and there are passionate opinions. However, let's make sure we keep it light and civil.