Ask Mallard
Core concept

Using analysis code from Mallard

4 min read

Use Mallard code as a scoped starting point, reading the status of each language and checking every adaptation.

Fit

When it matters, and when it does not

It matters when

  • A Learn method has a code entry whose scope matches your design and target estimate.
  • You can inspect variable definitions, missing values, model settings and the intended outputs.

It matters less when

Before you start

What to check in your study

Decide these first
  1. The entry scope and your intended analysisMatch the outcome, effect scale, sampling, covariates and missing-data handling. Similar syntax does not establish the same estimand.
  2. The status of each language and its checksRead executed, not executed or not applicable separately. Preserve package versions, seeds, warnings and failures.
It rests on these
  1. A seeded fixture checks specified quantities only.How to check: Known inputs and expected outputs help detect mistakes, but passing them cannot validate every dataset or assumption.
  2. Your adaptation needs its own checks.How to check: Verify row counts, coding, missingness and the model call. Reproduce known answers before interpreting your study result.

Plain language

How it works

Executed means a language ran against the library fixture and passed its stated recovery checks. Not executed means a reference implementation that nobody ran. Not applicable means the entry deliberately provides no implementation in that language. An unknown status is not rendered.

How it works

Cross-language agreement is a separate claim with entry-specific tolerances. It is not a guarantee about your data. Generated or adapted code is not made executed by borrowing from an executed reference.

How it works

Save the original entry, your changes, software versions and checks. Compare the fitted estimand with the plan, inspect warnings, and check a known answer. Running without an error is not evidence that the analysis supports the scientific claim.

The decision in view
Each language has its own status. Executed supports stated fixture recovery checks; not executed is a reference; not applicable has no implementation. Agreement needs two or more executed languages and recorded tolerances.Read the status for each languageExecutedStated fixture recovery checksNot executedReference onlyNot applicableNo implementationAgreement is separate: at least two executed languages and recorded tolerances. No status validates anadaptation to new data.
Conceptual diagram. It shows the structure of the idea, not data from any study.

Worked example

A worked example

Invented for illustration
Setting
You build a small synthetic fixture before adapting an analysis.
Question
Does the event coding produce the expected fraction?
Approach
The fixture has 10 rows; 8 contain an event and 2 do not.
Result
The expected count is 8 of 10, or 80 in 100. The expected event fraction is 0.8.
What it means
Matching that answer checks this coding task only. It does not establish causal validity, interval coverage or correctness of the whole adapted analysis.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

A language status is never a blanket verified label. Fixture checks do not validate your data, the adapted code or the scientific assumptions.

  • Calling every language verified because one was run.Instead: Keep each language status separate. Not applicable means no implementation, rather than a successful run.
  • Claiming agreement when only one language ran.Instead: Agreement needs at least two executed languages and recorded tolerances. One run can support only the stated recovery checks.

Inside Mallard

How Mallard uses this

Mallard displays code from its pinned library with separate language statuses. Agreement is shown only with at least two executed languages and recorded tolerances.

No Mallard code example points to this page.
Atlas node concept.using_mallard_code, the id a plan cue records.

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine. (opens in a new tab) · BMJ · doi 10.1136/bmj.k400
  2. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. (opens in a new tab) · Proc Natl Acad Sci U S A · doi 10.1073/pnas.1602413113
Prepared with AI assistance. Primary methodological and empirical source records were checked by PMID for title, authors, year and DOI on 2026-11-06. These sources support the general methods; tests pin the separate claims about Mallard. No independent professional approval is claimed.
Last source review: 2026-11-06
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.