ASK MALLARDbeta
Describe a study
ASK MALLARDHow the flock builds the app

How the Flock
Builds Ask Mallard

The story: a flock builds a nest

This is a story about how a flock of mallards builds a nest. At its heart are two mallards, a drake and a hen, who build it together, each leading a flock of helpers. Before any twig goes into the nest, the other bird checks that it is strong. Watching over them is the mallard shepherd, crook in hand, who decides where the nest goes and how it should be built.

The mallard shepherd, the founder: a bald man with glasses and a short salt-and-pepper beard, in a green chore jacket, charcoal trousers and brown boots, between Claude the drake and Codex the hen
The mallard shepherd: the founder, who decides where the nest goes and how it should be built

Ask Mallard is built the same way. The founder is the mallard shepherd, deciding what gets built and making the important choices along the way. The drake is Claude Code, made by Anthropic. The hen is Codex, made by OpenAI. Both are AI coding assistants that read, write and test software from instructions in plain English. The normal release process requires automated tests before a change reaches the live website, and an independent review by the other mallard for changes to how the app behaves. Both run on tokens, the small pieces of text an AI reads and writes. Each subscription has usage limits, the way a bird has only so much energy to flap its wings before it must rest.

Claude the drake and Codex the hen beside the nest, with three eggs
ClaudeMade by AnthropicThe nestThe code, kept on GitHubCodexMade by OpenAI

A nest exists for its eggs. In Ask Mallard, the eggs are the study plans the app generates for clinicians. When a plan hatches, it becomes a duckling: a new research idea on its way to becoming a real study. Everything in this story exists to keep those eggs safe.

Study plans are the eggs; they hatch into ducklings, research ideas realized
Study plansThe eggsA plan put to workHatchingResearch ideas realisedThe ducklings

Ask Mallard is a learning research system. A learning health system feeds what it learns from each patient back into the care of the next. In the same way, Ask Mallard uses validated findings from its own code and the plans it writes to strengthen checks for later work. Each lesson can make the nest a little stronger for the next egg.

To see how it works, follow one illustrative request through the story. One evening, the founder types on a phone: "Add a calculator that tells a clinician how many months of data they need for a before-and-after study."

Three more pieces complete the story:

GitHub Copilot, the heron, inspects the twigs on the bank and flags one
GitHub Copilot, the heron

Part 1. Building the nest: how the app is developed

The flock takes flight: one instruction, many helpers

Figure 1 · Building Ask Mallard

One instruction sends out a flock of AI helpers, and every result is checked

The mallard shepherd (the founder) sends the request to one of the two mallards, Claude the drake or Codex the hen: whichever has more energy to flap its wings that evening (more of its AI usage allowance left). This time it goes to the drake (Claude's lead session), one of a few coordinators the founder talks to. He breaks the job into pieces and sends out his flock (smaller AI helpers). One helper studies the calculators Ask Mallard already has, one builds the new one, one writes tests, and one drafts the Learn page that explains it. Meanwhile, the lookout (a release coordinator, another AI session) keeps watch over what is ready to go live, and in what order.

Two flocks, one nest: how Claude and Codex work together

While the drake's flock builds the calculator, the hen's flock (Codex's helpers) may be at work elsewhere in the app. A few simple rules keep them from getting in each other's way.

The drake's flock and the hen's flock
Four classes of change

The riskier the change, the more checks it must pass

Every change is sorted into a class by how much harm a mistake could do.

The calculator touches the eggs (Class C), so it needs every check: tests, a deep review by the hen, a written plan from the founder with a replay of test questions, and the founder's sign-off.

Why the nest holds: speed and fewer errors

Speed. Like a flock in formation, several AI sessions work at once, each on its own copy of the code (parallel sessions in separate git worktrees). The calculator, a sign-in fix and a new Learn article can all move forward on the same evening.

Fewer errors. AI tools from different companies may miss different things, the way two birds scan different patches of sky. That is why the independent development review is a cross-lab review. In our story, the hen questions one formula in the drake's calculator, and the answer is checked against R, widely used statistics software (the project calls these oracle tests), before work continues.

The hen questions a calculation while the drake checks it against R: qt(0.975, 10) is approximately 2.228139.

Validated findings become lasting safeguards. A validated finding leads to a new check or automated test (a regression test). It must catch the error and stay quiet on a real example that is sound. The system then deliberately puts the error back to prove the test notices, like knocking a twig loose to check that the alarm call sounds (a sabotage run). If the hen's question about the formula turns up a real mistake, that test stays in the nest, guarding the behaviour it tests in later versions. The same idea runs through the public Mallard Code Library, the R and Python reference code a plan's analysis starts from when the library holds a compatible method. Its execution record applies to the reference example; code adapted for a study has not been run. A library fix can help later plans once the app uses the updated reference.

The nest and the heron: GitHub and Copilot

In the normal development process, each proposed change starts as a pull request on GitHub, showing exactly what would change. Test results, review comments and sign-offs are recorded beside it, so the change can later be traced. The calculator arrives marked Class C, because it changes statistical advice. It is accepted only after it passes every check below.

The nest is built: the calculator goes live

Back to the founder. Once every check passes, a short summary of the calculator arrives for sign-off, because it gives statistical advice. The founder approves. The change waits its turn, is retested alongside everything else released that day, and goes live on askmallard.com. It joins the calculators in Mallard Learn, beside articles such as Sample size and statistical power and Interrupted time series, and the glossary that explains each term.

The nest is built and the first ducklings hatch

The calculator is new, but most of what made it safe was already in the nest: the tests, the reviews, and the safeguards that earlier mistakes left behind. Validated lessons add to them, so later work can be built on firmer ground. The aim is a sturdier nest for the eggs it holds: the plans Ask Mallard generates, and the ducklings that hatch from them into real studies. That is the learning research system at work.

Part 2. Laying the eggs: how a study plan is made

Part 1 showed how the flock builds and guards the nest (the app). Now a clinician comes to the pond as the village healer, hoping for a duckling of their own. This part follows their question into a study plan, and then a study. The full method, including how each number is checked, appears in the Methods and validation reference.

From a healer's wish to a duckling: the app's six steps

Figure 2 · Six steps in the app

From a healer’s wish to a duckling

Before the app: the healer's wish. The clinician types that wish as a study question in plain words on the study plan page, or uploads a document. Before submitted study text leaves their browser, Ask Mallard strips out the patient identifiers it recognizes (protected health information, or PHI). It cannot recognize every identifier, so clinicians must keep patient-identifying information out of what they submit.

Then the app's six steps begin:

  1. The mallards listen (Triage). An AI restates the question in clearer terms, proposes research aims and testable hypotheses (with alternatives), and checks that they fit together. An early literature scan then searches PubMed, the public index of medical research. It summarizes related studies, points out any gap the new study could fill, and suggests questions to take to a medical librarian.
  2. Surveying the pond (Feasibility). The clinician records funding, team, data sources and deadline. These shape what is reachable, but they never make a weaker method acceptable. Answers that contradict each other are flagged before going further.
  3. Choosing the nest site (Design). A design brief proposes a study design and alternatives, each with the conditions it depends on, and the clinician chooses. Harder designs, such as cluster randomized trials, are routed to a stronger drafting model. Then, just as the drake and the hen check each other's twigs, two AI reviewers from different companies (Anthropic and OpenAI) are asked to read the brief, and the drafter gets a corrective pass when needed. The record distinguishes reviews that ran, issues raised and revisions applied; a failed review is not a pass.
  4. Forming the egg (Analysis). The plan states its estimand: exactly which effect, in whom, measured how. Every number must agree with it. Where a supported calculation fits, the sample size is calculated with formulas tested against R. A sample size that contradicts the estimand is withheld. A design beyond the supported calculations, including some clustered designs, may need simulation; the app gives a sizing plan, and a statistician must still run that calculation. The same two reviewers are asked to read this draft too. When they raise issues, a final reader is asked to check what remains unresolved after revision. The reviewers are asked seven safety questions; missing answers remain unknown.
  5. Readying the nest (Execution). The plan sets out what running the study takes, and what to check before the first participant enrolls.
  6. The egg is laid (Finalize). The finished study plan includes an abstract and a protocol synopsis, and a background section with PubMed references, each marked as a record to confirm, not a proven claim. For each aim it gives methods, a sizing plan and a statistical analysis plan. It adds a proposed causal diagram where appropriate, results-table templates and illustrative figures, and analysis code in R and Python, which starts from the Mallard Code Library when it holds a compatible method. The illustrations are not study results, and the adapted code has not been run. It closes with open questions for the clinician's statistician or institution, Word and PDF exports, and links to the Mallard Learn pages that explain the design, where a calculator like the one the drake built in Part 1 may sit beside the article.

After the app: the duckling hatches. The duckling goes home with the healer: the clinician shares the plan with their team (the healer's helpers), their statistician (the vet, who gives the duckling a checkup) and their review board (the village council, which decides what oversight is required before it is raised). Once the required determinations and approvals are in place, they run the study, raising the duckling until it grows into a full-grown duck (the study's findings). The duck takes flight and is seen across the valley (the findings are published and shared). Healers in other villages consider what it means for the care of their villagers (clinicians assess whether the findings should change patient care). Those healers come back to the pond with new wishes, and the cycle begins again (new questions that start new studies).

Conclusion: a flock, a nest, and ducklings

Ask Mallard is built the way our mallards build their nest. The mallard shepherd decides what to build. Two flocks, the drake's and the hen's, do the building, and each inspects the other's twigs before they are woven into the nest. The heron and the lookout keep watch over the nest too, so it stays strong enough to nurture its eggs.

Inside that nest, every healer's wish follows the same six steps. The mallards listen, survey the pond, choose the nest site and form the egg with care, then ready the nest before the egg is laid, and the healer looks it over at every turn. When the egg hatches, the duckling the healer wished for goes home to be raised: a research idea on its way to becoming a real study. The shepherd watches from a distance.

The founder watches from the pond as three village healers in contemporary outdoor clothes carry herb satchels toward the sun, each with a duckling beside them.