Using analysis code from Mallard
Use Mallard code as a scoped starting point, reading the status of each language and checking every adaptation.
Fit
When it matters, and when it does not
It matters when
- A Learn method has a code entry whose scope matches your design and target estimate.
- You can inspect variable definitions, missing values, model settings and the intended outputs.
It matters less when
- The code fits a different estimand or sampling structure. Choose an appropriate method before translating it. Read: Choosing an analysis from your estimand
- You need help judging the adaptation. Obtain statistical or programming support before using the result. Read: What your resources change, and what they don't
Before you start
What to check in your study
- The entry scope and your intended analysisMatch the outcome, effect scale, sampling, covariates and missing-data handling. Similar syntax does not establish the same estimand.
- The status of each language and its checksRead executed, not executed or not applicable separately. Preserve package versions, seeds, warnings and failures.
- A seeded fixture checks specified quantities only.How to check: Known inputs and expected outputs help detect mistakes, but passing them cannot validate every dataset or assumption.
- Your adaptation needs its own checks.How to check: Verify row counts, coding, missingness and the model call. Reproduce known answers before interpreting your study result.
Plain language
How it works
Executed means a language ran against the library fixture and passed its stated recovery checks. Not executed means a reference implementation that nobody ran. Not applicable means the entry deliberately provides no implementation in that language. An unknown status is not rendered.
Cross-language agreement is a separate claim with entry-specific tolerances. It is not a guarantee about your data. Generated or adapted code is not made executed by borrowing from an executed reference.
Save the original entry, your changes, software versions and checks. Compare the fitted estimand with the plan, inspect warnings, and check a known answer. Running without an error is not evidence that the analysis supports the scientific claim.
Worked example
A worked example
- Setting
- You build a small synthetic fixture before adapting an analysis.
- Question
- Does the event coding produce the expected fraction?
- Approach
- The fixture has 10 rows; 8 contain an event and 2 do not.
- Result
- The expected count is 8 of 10, or 80 in 100. The expected event fraction is 0.8.
- What it means
- Matching that answer checks this coding task only. It does not establish causal validity, interval coverage or correctness of the whole adapted analysis.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A language status is never a blanket verified label. Fixture checks do not validate your data, the adapted code or the scientific assumptions.
- Calling every language verified because one was run.Instead: Keep each language status separate. Not applicable means no implementation, rather than a successful run.
- Claiming agreement when only one language ran.Instead: Agreement needs at least two executed languages and recorded tolerances. One run can support only the stated recovery checks.
Inside Mallard
How Mallard uses this
Mallard displays code from its pinned library with separate language statuses. Agreement is shown only with at least two executed languages and recorded tolerances.
concept.using_mallard_code, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine. (opens in a new tab) · BMJ · doi 10.1136/bmj.k400
- Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. (opens in a new tab) · Proc Natl Acad Sci U S A · doi 10.1073/pnas.1602413113