Third-party certification
Tier 3 · Reproducible - Gold recommended
This criterion is recommended in general. For packages that do not contain all datasets needed because some cannot be shared, it becomes a core criterion: without a third-party certification, only the Tier 1 criteria are assessed, so the package can get at most Bronze. With a certification, the core criteria that need the data are rated from the certificate, so Silver and Gold become possible. See rules 3 and 4 in Rules of the badge system.
An independent party is someone who is not an author of the paper and had no part in its data collection or analysis. The certificate or signed letter must state the reproducer’s name, affiliation and relationship to the authors, what was reproduced, and how.
Rating
The package includes a certification that an independent, disinterested party reproduced all results (or specified tables and figures) from the package, e.g. a signed letter by the reproducer or a certificate from a certification agency, stating what was reproduced and how.
A certification is provided, but it covers only some of the results, does not state what was reproduced, or comes from a party that is not independent of the authors.
No third-party certification is provided when the data are confidential and the reproducer cannot access them.
Based on: #131 only (ACRE, adjusting levels for confidential/proprietary data); see also #52 CODECHECK certificate, #56 ROR-R badge.
All datasets needed for reproduction are included the package (C2)
Prerequisites
Check order: 1
Builds on
- Data provenance (1 source)
Prerequisite for
No criterion builds on this one.
In the corpus
| Sources | Injunctive | Descriptive | Tacit | Automatable | Manual check | Automation unclear | LLM sources |
|---|---|---|---|---|---|---|---|
| 1 | 100% | 0% | 0% | 0% | 100% | 0% | 0 |