BEben509·Apr 25, 2019
Congrats on the release!
I'd recommend mentioning "fuzz testing" and "fuzzing" in the description to help people find it.
Simple and Sample are very close in spelling.
The sample makes sense, but a big use case for this would be testing APIs, and in that case, a JSON object is commonly modelled after the attributes of a class. So in that scenario, simply randomizing the keys would just blow up, which is not useful for testing. The kinds of errors you usually want to catch are small glitches.
I needed to describe the patterns that JSON could take when converting to and from native types, and I did that here [1] so it might be a useful structure to consider. But, roughly, you have several common cases:
1. atoms
2. homogenous arrays (same type of all elements)
3. heterogeneous arrays (element X has type Y, bounded length)
4. homogenous objec
Read the rest
ts (same value type for all elements)
5. heterogeneous objects (key X has type Y, required + optional keys)
6. alternatives
Implement those and you probably cover any structure someone could throw at it. You'll also want to make it easy to distinguish between required and optional keys.
If you take a page from QuickCheck[2], it's nice to be able to implement a "reduce" feature. The idea is you produce something complex that breaks it, and simplify it to find a minimal test case.
You might go both ways... generate a valid pattern and add crap to it to see how tolerant it is. Or possibly start with reasonable length strings and make it possible to make them longer and longer.
[1]: https://github.com/UnitedIncome/json-syntax/blob/master/json...
[2]: https://hackage.haskell.org/package/QuickCheck
Versions of QuickCheck exist in many languages, e.g. https://github.com/HypothesisWorks/hypothesis-java