DRMacIver

Cool! I've added a comment to your commit as I think you've made a mistake in the strategy setup, but it looks good as a test.

Sorry about the lack of 2.6 support. I looked into it and I could probably do it but I just don't care enough about 2.6 to put in the work and make everything uglier. :-)

Have you seen http://hypothesis.readthedocs.org/en/latest/examples.html#fu... ? It doesn't test very much - mostly just that the API doesn't produce a 500 error - but it's a decent example of how you can generate structured data with some constraints. http://hypothesis.readthedocs.org/en/latest/examples.html#co... is also a decent example where the data is more uniform but requires a bunch of massaging to satisfy the constraints.

> Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions between strings and floating point numbers has a bunch of weird issues. There are some obvious ones, but then also things like certain numbers convert differently if you add a trailing zero (so 0.7362856270 is turned into a different number to 0.736285627 and 0.73628562700). So my little "decode(encode(x)) == x" test example broke!

I have in fact had to deal with exactly this problem in Hypothesis internals. The example saving code didn't work correctly with floats in the first edition, because it was serializing them as JSON, which loses some information encoded in the actual float. In the end I serialize floats (which are actually doubles) by converting them to a 64-bit integer with the same bitwise representation first.

The bug aryx is talking about though actually illustrates an amusing feature of Hypothesis, which is that the code in it is just weird enough that it tends to do unexpected things that trigger bugs which have nothing to do with the properties being tested. :-) In this case it was putting unicode objects onto sys.path, which is 100% allowed but causes problems for certain code running on python 2.7 on windows that previously appeared to work.

This is also on the "at some point" list. :-)

Concolic testing is quite hard in Python because of its extremely flexible semantics. I will probably look into doing something with z3 at some point when I need an interesting problem to entertain me, but I don't hold out a massive amount of hope for it being useful.

Thanks so much for the kind words!

The best part of this message is that if you were someone I knew and thus could trust to only say nice things about me you would offer to buy me gin instead of beer. ;-)

No, it's much closer to classic fuzz testing, where the test is held constant and the data fed to it is varied. I've been meaning to see if I can figure out a way to use mutation testing to feed into Hypothesis, but it's on the long list of things I'll try "at some point"

This looks super interesting and I'll check it out when I'm not doing release management stuff, thanks!

If you haven't seen it, http://hypothesis.readthedocs.org/en/latest/internals.html#p... describes the current algorithm for generation. It's smarter than is typical for quickcheck, but could definitely be improved.

Yeah, the google ranking of Hypothesis when you search for python quickcheck is a thing that's been bugging me. There's a blog post from 2013 about it on the front page but none of the various places the project exists appear. I'm not sure what to do about it. I'm planning to do a bunch of shopping around and promoting of the library when the 1.0 release happpens (probably next week), so hopefully that should help.

The CLA is for a mix of weird personal reasons around open source and cynically practical ones. Hypothesis is 100% always going to remain free, but I'm trying to figure out ways that I can make money out of working on it (I've put a ton of work into it, so it would be a shame not to) and that's a lot easier if I retain the copyright. I may decide it's not useful and drop it later, but it's a lot easier to have it now and drop it later than it is to try and retroactively get one if it turns out I need it.

No, the infinite precision integer type in Haskell is called Integer. Int is a platform dependent fixed size int.

Realised I should have linked to http://hypothesis.readthedocs.org/en/master/internals.html to back up that "a whole bunch of things even quickcheck doesn't" claim. :-)

Yep. This is why I wrote Hypothesis: http://hypothesis.readthedocs.org/ / https://pypi.python.org/pypi/hypothesis

It's a lot more fully featured than the alternatives. It still lacks a few features of quickcheck (coarbitrary for function generation, labelling of examples), but it does example minimization and a whole bunch of things even quickcheck doesn't.

Um. Encapsulating data and operations is exactly what libraries do. It's kinda what they're for.

It is unclear to me what the intended mechanism of services helping different teams work together is that is not also provided by just writing libraries. Could you elaborate?

For context, my normal desktop environment is wmii running on Linux. I also find Windows physically painful. The problem is that the reason this works for me is that I'm able and willing to put up with an awful lot of bullshit and have a reasonably good ability to debug things when they're not working properly.

Yes, I know. I had it but my config was badly broken so it wasn't doing anything. Whoops.

TBF, I've had a self-hosted blog for about 6 years. I just chose a platform that I didn't understand in order to prevent me from wasting time yak shaving on the means of hosting and just write content so have never actually performed the learning part of step 1.

I've mostly got away with it in the past - it's survived much heavier hackernewsings - but I think I installed a plugin recently that was rather more costly than it should have been, and then the fact that I hadn't previously looked under the hood bit me.

Edit: It's also worth noting I don't think principle 1 is always a good idea. This advice is very specifically optimised for the case of joining an existing company with an existing product and an existing team. If you're starting from scratch you should do more up front investigation and thinking things through. though maybe not orders of magnitude more.

I think "best solutions come from new people" is an over-simplification of what actually happens. New people force you to re-examine your assumptions and blind spots. Sometimes this is useful, sometimes it isn't. This doesn't mean that new people solve more problems, but sometimes an influx of new people can help the experts to solve more problems because they realise something they assumed to be true/OK wasn't.

If your sole measure of advantage is promotion then yeah, feel free to kick back your heels and do whatever you want.

Personally, I quite like to be doing useful work, and normal ramp-up times for people seem to be about 3-6 months, which if you work at a company for < 3 years can be as much as 10-20% of your time. More if you're switching between projects within a company. It's pleasant to not have to waste that large a percentage of your time getting up to speed.

Ah. That's a good idea which I totally failed to follow.

Instead I flailed around blindly in the usual manner of trying to get PHP applications to work and seem to have fixed the problem. :-)

I should really move my blog to an environment I actually understand and have control over.

Thanks. I'm currently paying the price for my foolish decision to self-host wordpress. In the process of trying to fix.

Yeah, I'm not wild about those labels either. I confess they were thought up on the spur of the moment to try to explain the idea.

The reasoning for the naming is essentially that an explicit mental model is one where you can write down all the steps, whileas an implicit model is one that just arises naturally from the things already in your brain and you can't really point to it and say what it is. I'm not sure that behavioural model quite captures that, but I'm definitely up for a better naming convention.

I'm not sure I would call the steps "reasonable" per se. They're much more obviously bad ideas in retrospect than they were at the time, but a lot of them should have been pretty obviously bad ideas at the time as well.

On the HTTP is slow thing: I think I was pretty clear in the write up that we had failed to performance test and it wasn't at all clear that our replacement was faster. I think these are sufficient disclaimers that I was pretty obviously labelling it "HEY DON'T DO IT THIS WAY" :-)

Did you miss the part where I explicitly said in multiple places that there was an obvious simple solution and that this post was an example of what not to do?

May I extol the virtues of quitting and finding a better job to you? :-)

Did you miss the sarcasm even in retrospect? Because I don't really see how it can be possible to read this article and not understand that I'm pointing out that all of these things we did are terrible ideas and you shouldn't do them.

For our cousins across the pond, this is an instance of a type of humour called "Sarcasm", in which you state something directly in order to imply that it is obviously wrong.

What, you expect us to read the article to derive the lessons contained therein? What lunacy is this?!

Yes, indeed, we have the same, but it becomes much more useful if the exceptions being thrown by your dependencies are of high quality.

So it's legitimate to use exceptions to terminate early with an HTTP status code such as 304, but it's not legitimate to use exceptions to terminate early to say that there's a cached version of the page available and no further processing is needed. Have I got that right?

Given that the article was a response to criticism of a previous proposal of mine on grounds that it used exceptions for control flow, and given that I linked to that discussion (on the page for the proposal in question) in the opening paragraphs of this one I probably felt that the reasons for using exceptions for control had been adequately covered at that point.

But I was probably not trying to suggest you should regularly do this (certainly I wouldn't suggest that now), only that if you find you have a good reason to do it you shouldn't be afraid of it.