The Smart Fuzzer Revolution [video]

> An open-source alternative of Microsoft's SAGE/Springfield would be cool

We're working on one (!) and I hope we can offer it for free to non-commercial projects. For now, there is Microsoft Springfield [1] for Windows software and Google OSS-Fuzz [2] for open-source software. It is extraordinarily hard to not only get the tech for something like that working but bring it to market.

As noted in the video, nearly all the individual pieces of our CRS are open-source but you actually do not want a "CRS." The competition DARPA designed for them involved more than what is necessary to provide value to a development team, e.g., you don't want something that writes IDS signatures, considers "gameplay" or resource contention, or attempts to write automatic patches. You want something that accurately finds and reproduces bugs. We open-sourced the tools we wrote to do that or used tools that were already open-source, like Grr, Manticore, Radamsa, KLEE, and Z3.

[1] https://www.microsoft.com/en-us/springfield/

[2] https://github.com/google/oss-fuzz

I'm not the original poster, but:

> Is fuzzing something for me to improve code quality,

Is it C/C++ and does it include parsers? Then definitely yes. If not it depends.

> or are tests the lower hanging fruit?

Again: Is it C/C++? Then familiarize yourself with the sanitizer features of gcc and clang, primarily address sanitizer. Lots of "rotting" code shows memory safety errors just by running them with asan.

Disclaimer: I have not looked at your codebase, so this should only be taken as my 2 cents. I might have a different recommendation if I had greater familiarity with your exact problem.

I'm biased, but I would start with a fuzzer. Fuzzers have two key differences for your scenario that makes me lean towards them:

They provide the will to act. You mentioned the codebase is old and rotting, so a normal bug may not rise to enough attention to fix. Finding security bugs may give greater justification to undertake an effort to start maintaining it properly.

1 fuzzer exercises more than 1 test. I think you'll get more bang for your buck by integrating a fuzzer, whether it is libFuzzer, AFL, Radamsa, or anything else. Fuzzers are not targeted to a single unit and will, ideally, find bugs all over the place from a single, simple starting point.

That said, there are good arguments for diving into the codebase and writing tests too. In fact, writing tests may make your fuzzer more effective.

Big thanks for this reply. I was looking to cut through all the noise you mentioned. I'm finding all I was looking for in the Radamsa readme for starting and next steps. Use cases expressed in unix pipes > clear_as_day.txt. Fuzzing newb at this point but I'll be using your original and other linked presentations for context while working through basics.

Burp suite does not do a good job at fuzzing APIs - not biased but true. APIs require more structured fuzzers that expose application level problems - not like burp's fuzzers which are working on raw HTTP requests which was useful sometime ago when you had to find bugs in the actual server implementation. This is not relevant in the web application security space anymore apart from the fields of research which is exactly what most web shops not interested to do. You can still use Burp for that but the user needs to do all the heavy lifting by hand. How does Burp do recursive XML or JSON fuzzing? It doesn't. You can write a plugin for that but that defeats the purpose of using an off the shelf tool.

Thanks for clarifying this and your info on Burp's capabilities above. What are some of the commercial options you just mentioned? I'm dealing with a large app not written with C but does have API endpoints over http. For a start on some black-box fuzzing, I'm thinking I could use Radamsa's client/server capabilities.