joss-reviews

The reviews for JOSS happen on github[0] but the journal's not prescriptive about how you develop your package as long as the code is public. The criteria for the JOSS review are very clear[1]. I don't want to oversell the depth of the code review possible; not all of the reviewers will be fully expert in whatever tiny cutting-edge area the package is for (making correctness checks difficult beyond the test suite), and most of us are academics-who-code rather than research software engineers. But the fact it's happening at all is a great step forward. [0]: https://github.com/openjournals/joss-reviews/issues [1]: https://joss.readthedocs.io/en/latest/review_criteria.html

I feel it may be a problem for them since it looks like they try to reproduce the program's setup/workflow, which is (in my experience) more than bioinformatics reviewers usually do: >Authors are strongly encouraged to include an automated test suite covering the core functionality of their software. > OK: Documented manual steps that can be followed to check the expected functionality of the software (e.g. a sample input file to assert behaviour) You can see the checklist reviewers go through in any of the issues here: https://github.com/openjournals/joss-reviews/issues