wycheproof

<shameless plug> If you enjoy these challenges you might want to join my team at Google, to analyze, break, design and implement real-world crypto solutions for products used by billions of people. Aside from short-term projects like analyzing and improving state-of-the-art password hashing or end-to-end encryption schemes, we've started two long-term that could use your help: Wycheproof [1] and Tink [2]. The former is a revolutionary testing framework that has discovered 40+ bugs [1] in many popular crypto libraries. The later provides a safe, simple, agile and fast way to accomplish common crypto tasks. If this sounds exciting, please send your resumes to thaidn@google.com. Cheers! [1] https://github.com/google/wycheproof [2] https://github.com/google/tink [3] https://github.com/google/wycheproof/blob/master/doc/bugs.md </shameless plug

Why is there a lack of free public standardized test suites for exercising common algorithmic code? Some domains, such as data structures, math, stats, string processing, cryptography, etc have many well-defined problems. But whenever one writes code to either replicate (for study) or improve upon existing algorithms, each developer has to write their own test cases, reinventing and duplicating test suites for similar problems over and over and over. In contrast, a standardized test bank can save massive amounts of each developers time and contribute to improved software quality across the board. It should be easy to write an utility that reads simple text files hosted online containing input/output test cases to exercise the code. Project Wycheproof for testing crypto code (https://github.com/google/wycheproof) comes close to what I'm ta

See e.g. https://github.com/google/wycheproof Fuzzing tests can also uncover weird bugs and edge cases that can lead to vulnerabilities.

the problem with testing encryption in an integration test is that the same input should encrypt to different values each time. this makes testing without some kind of stubbing harder. in terms of automated tests that could have caught this error: 1). you can create a second implementation and check that it properly decrypts the output from the first implementation and check the first implementation will properly decrypt the output from the second implementation. 2) you can create output from what you believe is a known good program. then write an automated test that would decrypt this output. then write an automated test that would perform an encryption and then a decryption. this would catch a problem where the encryption and decryption routines switch to using a hardcoded string because of a bug. warning: 2) only works if the bug doesn