Therefore, the codechecker’s activity should be recorded, ideally in the published paper. The value of performing a CODECHECK is comparable to that of a peer review, and it may require a similar amount of time. Allowing the codechecker to communicate directly and openly with the author make this process as constructive as possible routing this conversation (possibly anonymously) through a publisher would introduce delays and inhibit community building.

Some code may work without any interaction but often there are hidden dependencies that need adjusting for a particular system. Codecheckers take the pictures at a crime scene, they do not hunt the criminal. The conversation with a codechecker fixes this problem. The level of documentation required for third parties to reproduce a workflow is hard to get right, and too often this uncertainty leads researchers to give up and not document it at all. We believe that the job of the codechecker is not to fix these problems but simply to report them to the author and await a fix. If instructions are unclear, or if code does not run, the codechecker tells the author. The codechecker follows the author’s instructions to run the code.

Codecheckers record but don’t investigate or fix.See the CODECHECK paper for a full description of problems, solutions, and goals and take a look at the GitHub organisation for examples of codechecks and the CODECHECK infrastructure and tools. The independent time-stamped runs conducted by codecheckers will award a “certificate of executable computation” and increase availability, discovery and reproducibility of crucial artefacts for computational sciences. See the CODECHECK register for all completed certificates.ĬODECHECK tackles one of the main challenges of computational research by supporting codecheckers with a workflow, guidelines and tools to evaluate computer programs underlying scientific papers.

Chk org