Ask HN: How to generate random test data (valid and noise too)? | Hacker News Reader

Ask HN: How to generate random test data (valid and noise too)?

5 pointsazatom5 comments

I need random data in block of 700 bytes, what is full with valid common binary/plaintext data structures with common data. Sometimes included with common errors making them invalid. In a range from simple bit patterns (eg.: 0x55aa) to just uncompressible noise.

(So.. just everything, I hope it is not too much to ask:)

What won't work:

- /dev/urandom: it is just not random.. always noisy :)

- A corpus like dumped wikipedia: even if with added some random biterror, it is just too specific, besides impractically big.

- markovchain of /dev/sda of freshly installed system: still not enough real world data/errors.

- markovchain of my /dev/sda: although it contains enough real data and errors :) there is sensitive data

- I collect painstakingly common patterns: the whole point is finding something what I don't expect

As always, there has to be existing solution, I just can't find it.

It might be that I don't understand your problem and if so I apologize, but from what I gather this is a classic XY problem.

Use property-based tests instead. Like quickcheck/jqwik/hypothesis.

The emphasis is more on the brute force. These testing tools are for a concrete method with concrete data structure (if not just one property) - so the "finding something I don't expect" does not apply. I'd like to stress test the a system on highly unreliable data, from the too witty parses to the to the electrical thingies, like memtests bitpatterns. Yes, I can unit test what I know, but in addition I want to know what I don't know:)

Okay, but property-based testing libraries have a "controlled randomizing" component. You could define what you want in those 700 bytes and then just feed them to the method or function that you're trying to fuzz.

Why is this different than what you are asking?

Can you show me an example of this "controlled randomizing"? What I know about it either just too much space for randomizing, or I have to add some pattern/rules, how to generate it.. so it is the base of the problem. But if there is an implementation what already have that some kind of "dictionary" for fuzzing, that is exactly what I am looking for.

I'm most familiar with the Java version; this is from the docs:

https://jqwik.net/docs/current/user-guide.html#customized-pa...

Yes, of course you do have to define some patterns, but you can do that for arbitrary types and the internal randomizer takes care of the rest. You could have multiple cases, say: one that's fully random, one where you have some integers, one where you have strings, etc.

Reply on news.ycombinator.com