CrossHair

Parquery/icontract: https://github.com/Parquery/icontract mristin/icontract-hypothesis: https://github.com/mristin/icontract-hypothesis : > The result is a powerful combination that allows you to automatically test your code. Instead of writing manually the Hypothesis search strategies for a function, icontract-hypothesis infers them based on the function's precondition. This makes automatic testing as effortless as it goes. pschanely/CrossHair: An analysis tool for Python that blurs the line between testing and type systems https://github.com/pschanely/CrossHair : > If you have a function with type annotations and add a contract in a supported syntax, CrossHair will attempt to find counterexamples for you: [gif] > CrossHair works by repeatedly calling your functions with symbolic inputs. It uses an SMT solver (a kind of theorem prover)

This is exactly what the CrossHair library does: https://github.com/pschanely/CrossHair It also overloads other methods to provide kind of symbolic objects.

> I believe your argument on the performance penalty is right, and as a corollary, this implies that contracts are mostly useful if associated with a formal proof system. Contracts in production are probably a bad idea, in most cases. Agree! There is a sliding scale between testing contracts and proving them too. My labor of love for the past several years has been a tool for checking Python contracts using an SMT solver (using symbolic inputs over concrete paths): https://github.com/pschanely/CrossHair That said, these days I think property-based testing gets you the ~same benefits as contracts, and is easier for most people to apply.

Just skimmed through the paper: https://arxiv.org/pdf/2007.14049.pdf, it seems like they are using some sort of heuristically mutating a test suite until it's fully branch covered, based on calculating `branch distance` for each predicate. Why isn't a SMT solver like Z3-solver being used here to solve for the predicate (generating inputs to evaluate to true/false)? Since it's getting so powerful, and python container/dict/string(regex) operation can also be modeled conveniently. And I'm also wondering, whether there is a return based automatic test generation, that start from the return value, and resolve all variables used and gather all possible return values with its constraint, and feed those to z3 to generate inputs to cover. It seems like it will help with branch explosion by eliminating unused branch, and only focus on branch that

Just in case you are looking for an alternative approach: if you write contracts in your code, you might also consider crosshair [1] or icontract-hypothesis [2]. If your function/method does not need any pre-conditions then the the type annotations can be directly used. (I'm one of the authors of icontract-hypothesis.) [1] https://github.com/pschanely/CrossHair [2] https://github.com/mristin/icontract-hypothesis

There's a tool for verification of Python programs based on contracts which uses Z3: https://github.com/pschanely/CrossHair You can use it as part of your CI or during the development (there's even a neat "watch" mode, akin to auto-correct).

Looking at some of your SMT-based projects, I'd love to compare your SMT solver notes with my mine from working on https://github.com/pschanely/CrossHair Sadly, there aren't a lot of resources on how to use SMT solvers well.

How else are you going to check the contracts at compile time? I honestly don't know if that's how DrNim works or if there are other ways to achieve that. But a similar tool for Python does it exactly this way: https://github.com/pschanely/CrossHair "CrossHair works by repeatedly calling your functions with symbolic inputs. It uses an SMT solver (a kind of theorem prover) to explore viable execution paths and find counterexamples for you." ...so that's where I got the idea that DrNim is probably doing the same