One of the truly great advantages of writing C or C++ code is the (multiple) possibilities for subjecting that code to fuzz testing, which is an outstanding way to find bugs in code:
Some people think that fuzzing is just for catching out-of-bounds accesses and the like, but it’s really much more useful than just that. If one has multiple independent implementations of a single algorithm with well defined input and output, it’s possible to generate random input for them with the fuzzer and then check if the outputs match. Solutions like the one used by Clang also attempt to maximize coverage during the fuzzing process. This and similar features make fuzzing really useful.
Even though I haven’t yet written any Julia code that could have benefited from a little fuzzing (I’m a newbie), I’m interested in whether there are options available, or if there is interest in making some progress on that front?
I suppose that Julia code is actually superbly suited to fuzzing, as Julia programs can modify their own syntax trees and (AFAIK) influence compilation in other powerful ways; and also perhaps because Julia already uses LLVM for code generation, so perhaps it could use libfuzzer just like Clang does?
The efforts in the Golang world to make fuzzing a first class citizen (like unit testing already is) are probably also worth taking a look at:
I don’t know of any such tools/libraries for Julia off the top of my head. A Google search would seem to suggest the ConcolicFuzzing.jl (JuliaCon talk) is probably the furthest anyone has gone with fuzzing for Julia.
I definitely think it’s something people would be interested in, though! (I know I would be interested, anyways ) I agree that with Julia’s metaprogramming facilities (plus libraries like Cassette.jl) it would probably be very well-suited to fuzzing.
) I agree that with Julia’s metaprogramming facilities (plus libraries like Cassette.jl) it would probably be very well-suited to fuzzing.