Reporting a statistical methods study

Connect each claim to a target, assumptions, uncertainty, and executable evidence

Before reading: Basic estimates, standard errors, and intervals. Evaluating a statistical method supports the performance discussion. The executable teaching example reuses the estimator from From an influence function to an estimator; you can read the reporting principles before completing that derivation.

1 Make the claim traceable

A methods report connects a scientific or methodological claim to a target, assumptions, procedure, and evidence. A reader should be able to reconstruct why the reported number supports the sentence written about it.

Start with the problem: what quantity matters, in which population and on what scale, and why an existing procedure may be inadequate. Define the independent observation and the observed data. Distinguish assumptions identifying the target from assumptions supporting an estimator or uncertainty calculation. A causal claim requires a causal identification argument; a descriptive association does not become causal through adjustment alone.

Then describe the estimator precisely enough to reproduce: estimating equation or objective, nuisance models and tuning, fold construction, preprocessing, fitting failures/retries, and confidence-interval calculation. State whether the result is an estimate of the scientific target, a model projection, or an approximation to another procedure.

A claim has several components that can be checked separately. “We estimated an adjusted difference” needs a definition of adjustment and the population over which it is averaged. “With 95% confidence intervals” needs an interval construction and its conditions. “More accurate” needs a target, a loss or performance measure, a comparator, and an evaluation setting. Start from those concrete statements rather than describing a method as universally robust or superior.

2 Match evidence to the claim

Claim	Evidence to report	What it does not establish alone
The target is identified	Observed-data functional and explicit assumptions	That assumptions hold in the application
An estimator has a stated limit	Argument and conditions, including remainder control	Finite-sample accuracy everywhere
Intervals attain nominal coverage in a scenario	Repeated-sampling coverage, MCSE, failures, scenario definition	Coverage in all populations
A method is efficient in a model	Relevant efficiency bound and an attainment argument	A small SE in one dataset
An omics analysis is scientifically informative	Effect scale, uncertainty, design, diagnostics, biological context	Ground-truth validation of every feature
A result is reproducible	Inputs, environment, configuration, code, execution record	Scientific correctness

Use Evaluating a statistical method to define performance measures before writing a comparison. In a real-data example, truth is usually unknown; call it an application or illustration unless a validation target is actually available.

2.1 State a theorem at the scope it supports

For an influence-function estimator, a useful theoretical statement identifies the target, statistical model, sampling assumptions, nuisance controls, and conclusion. For example, a first-order expansion has the form

\hat\mu-\mu=\frac1n\sum_{i=1}^n\phi_P(W_i)+o_p(n^{-1/2}),

where W_i is one person’s complete observed record and the contribution is mean zero with finite positive variance. For i.i.d. people, together with variance consistency, it supports asymptotic normal inference. It does not assert exact unbiasedness or 95% coverage at every finite n. If efficiency is claimed, specify the model’s canonical gradient and explain why the estimator attains its variance bound. A small reported SE alone is not that argument.

For the missing-outcome example below, identification needs an outcome-observation assumption and support. The estimator additionally needs suitable nuisance consistency, a negligible product remainder, moment control, and a variance argument. Cross-fitting addresses a particular empirical term, not every source of error. Put the precise theorem or a clear reference near the claim; do not make a reader infer its conditions from an algorithm’s name.

3 A worked synthetic reporting example

Reuse the estimator from From an influence function to an estimator so that the report and estimator share one implementation. Its downloadable R functions generate independent synthetic people; the true full-population mean is 7/3. Covariates are available for everyone. Outcomes are missing at random given Z in this generator, with observation probabilities bounded away from zero. These are known features of a teaching simulation, not verified assumptions about a clinical dataset.

Code

source("../examples/missing-mean.R")
example <- SimulateMissingMean(n = 800, seed = 51)
fit <- CrossFitMissingMean(example$data, K = 5, seed = 52)
s <- fit$summary
s

  estimate         se    lower    upper observed   n probability_clipped
1 2.354531 0.04003445 2.276063 2.432998      329 800                   0
  mean_clipped
1            0

The next paragraph is generated from that result. It names the target, units, method, uncertainty, and observation count. It does not claim that a single covering interval establishes 95% coverage.

Code

cat(sprintf(paste0(
  "Among %d independently generated people, %d had observed outcomes. ",
  "For the full-population mean outcome in the synthetic measurement units, ",
  "the five-fold cross-fitted augmented estimator was %.3f ",
  "(estimated SE %.3f; approximate 95%% Wald interval %.3f to %.3f). ",
  "Inference uses the MAR/overlap and nuisance-rate conditions in the capstone; ",
  "the generator and fitted model classes satisfy its teaching setup. ",
  "Fitted probabilities were clipped for %.1f%% of people and fitted means for %.1f%%.\n\n"),
  s$n, s$observed, s$estimate, s$se, s$lower, s$upper,
  100 * s$probability_clipped, 100 * s$mean_clipped))

Among 800 independently generated people, 329 had observed outcomes. For the full-population mean outcome in the synthetic measurement units, the five-fold cross-fitted augmented estimator was 2.355 (estimated SE 0.040; approximate 95% Wald interval 2.276 to 2.433). Inference uses the MAR/overlap and nuisance-rate conditions in the capstone; the generator and fitted model classes satisfy its teaching setup. Fitted probabilities were clipped for 0.0% of people and fitted means for 0.0%.

A real-data paragraph would name the assay scale, actual sampling design, population, acquisition and exclusion rules, and justification for MAR and overlap; it would not have access to the unobserved population truth. If the target were a log-scale contrast, interpret it on that scale or explicitly state the transformation used for a ratio interpretation.

3.1 Walk from the result to the sentence

The person and observation counts come from s$n and s$observed. The point estimate, SE, and endpoints come from the same fit$summary. The five-fold choice comes from the call to the estimator. The MAR and nuisance-rate assertions come from the generator and theorem, not from the numeric result table. Keep that distinction when auditing a paragraph: some words are computed facts, while others state the assumptions under which those facts are interpreted.

Because this is synthetic, we can also inspect its realized estimation error:

Code

error <- s$estimate - example$truth
cat(sprintf(paste0(
  "The generator's true mean is %.3f. This estimate differs from it by %.3f, ",
  "or %.2f estimated SEs. This is one realized error; ",
  "it is not an estimate of the procedure's bias or coverage.\n\n"),
  example$truth, error, error/s$se))

The generator’s true mean is 2.333. This estimate differs from it by 0.021, or 0.53 estimated SEs. This is one realized error; it is not an estimate of the procedure’s bias or coverage.

Bias averages signed error across repeated datasets. Coverage averages whether the target is included by their intervals. A single interval contributes one zero-or-one coverage outcome, not evidence that its construction achieves 95%. The stated 95% level describes a repeated-sampling property under the relevant conditions; it is not a probability assigned to the fixed mean after seeing this interval.

For a real analysis, retain the output object or table and generate the paragraph’s numbers from it when rendering the manuscript. If an exclusion changes the analysis population, regenerate the estimate, denominator, diagnostics, and associated text together. Updating the displayed number while leaving the old population description is a substantive reporting error even if the number itself is correct.

4 Report an evaluation with its uncertainty

A simulation table should identify scenarios, target values, attempted replicate counts, successful counts, bias, empirical SD, mean SE, interval coverage, and MCSEs for the summaries. Retain method pairing and report runtime or resource requirements if they are part of the claim. Explain whether a failure is counted as an unusable interval or omitted from a conditional summary. Avoid declaring one method superior from a difference smaller than its simulation uncertainty.

Separate absolute calibration from relative performance. “Coverage was near 95%” requires its estimate and MCSE, sample size, scenario, and failure denominator. “Coverage improved” additionally needs a paired difference and its uncertainty when methods used the same datasets. A difference of 0.01 with MCSE 0.012 is an imprecise estimate of improvement, not strong evidence of a gain. Conversely, lack of a clear difference does not establish equivalence; a practical equivalence claim needs its own criterion and precision.

Report the regimes in which the theory is being checked, those near a boundary, and deliberate failure cases. A good working-model scenario shows that an implementation can behave as predicted there. It does not establish robustness to every misspecification. If a method fails on difficult datasets, report those attempts alongside any summaries conditional on success.

A figure caption should be understandable without reconstructing the entire analysis: state what is plotted, units and target, independent-unit and replicate counts as relevant, what points/lines/bands mean, and whether a reference line is known truth, a null, or an estimate. Standard error bars, confidence intervals, empirical SDs, and MCSE bars need different labels.

For feature-wise omics results, name the hypothesis family and multiplicity procedure, effect scale, adjustment variables, and selection/filtering decisions. A small adjusted p-value does not imply a large effect, a causal effect, or independent biological replication.

More discoveries can reflect stronger effects, a different family, a changed target, a more liberal test, or better power. State which comparison is intended and how error control was assessed. A pointwise interval displayed for a selected feature does not become simultaneous or selection-adjusted because the p-values received an FDR adjustment. A comparison of network significance labels likewise does not replace a direct test of an edge contrast.

5 Choose displays that answer a question

A single estimate and interval can be reported clearly in a sentence. Use a plot when its structure reveals something the reader needs to assess: heterogeneity across features, bias as n grows, failure under a changed assumption, or a tradeoff across methods. A graphic that only repeats two numbers from the surrounding sentence adds little.

For example, the balance-by-mean-shift plot in Simulation studies for methods research reveals an interaction that a single rejection rate conceals. Its caption must distinguish the true equal-correlation null, the simulated datasets, the changing mean shifts, and the binomial reference band. The curve in Sensitivity analysis for assumptions and research decisions instead varies an assumption on one observed dataset; its shading represents conditional sampling uncertainty. Calling both shaded regions simply “95% uncertainty” would hide different meanings.

Label a reference line as known simulation truth, a hypothesized null, or an estimated benchmark. A full-data or external estimate is not an oracle; it has its own sampling variation, target, and possible measurement error. Captions should also state whether features were prespecified for display or selected after seeing results. A reader should be able to tell what comparison the figure supports before reading a conclusion about it.

6 Keep a compact reporting contract

A complete first draft can follow this sequence:

Question and target: population, measurement scale, independent unit, estimand.
Identification and design: observed data, missingness/selection, assumptions, support and confounding limitations.
Procedure and uncertainty: exact algorithm, nuisance fitting/folds/tuning, variance/interval formula, failure policy.
Evaluation: scenarios and truth where known, comparisons, replicate counts, metrics and MCSE.
Application and interpretation: data provenance, effect estimates, diagnostics, limits of real-data evidence.
Reproduction: code revision, input manifest, configuration, environment, run instructions, and retained outputs.

Separate prespecified choices from exploratory revisions and discuss where conclusions are sensitive to them. Link each displayed result to the code or retained output that produced it. Use A reproducible research project and Data provenance and an analysis-ready dataset to make this contract executable.

Before sharing a draft, choose one central sentence and follow it backwards: the displayed value, retained result, run configuration, data version and exclusions, then the target and assumptions. If a link is missing, repair that evidence chain. Then read forwards: does the sentence claim only what that chain supports? This small audit is more useful than checking that every methods section contains the same fashionable technique names.

7 Exercises

Try each checkpoint before opening its hint or solution.

Checkpoint · WRITE-01 — Rewrite a claim to match its evidence

Task type: Interpretation and reporting.

A draft says: “Our new method is unbiased and efficient because its confidence interval contains the true effect in one simulated dataset. In the omics application it discovers more significant genes, proving better accuracy.” Rewrite this as a defensible description and list the evidence still needed for the stronger claims. Include the distinction between an estimate’s SE and a simulation summary’s MCSE.

For the generated paragraph in this page, identify which statements come from result fields, which come from configuration, and which depend on assumptions or theory. State what needs to be regenerated if an exclusion changes the target population.

Ready to continue when: you can state what the reported evidence supports and identify the missing target, assumptions, uncertainty, or validation needed for each stronger claim.

Transfer: How would the caption change if the dashed reference line were a full-data estimate rather than known simulation truth?

Hint

One interval is one random outcome; discovery counts mix signal, calibration, filtering, and power.

Solution outline — open after trying

Report the one estimate, its stated target/scale, SE, interval, and the known synthetic truth as an illustration. Assess bias and coverage over independent replicates with MCSE and failures; efficiency needs a relevant bound and an attainment argument. In the application report effect sizes, uncertainty, multiplicity family/procedure, and assumptions. More discoveries alone do not establish accuracy when truth is unknown. SE concerns estimator variation across samples; MCSE concerns uncertainty in a simulation summary from finitely many replicates.

The counts and estimates are result fields; K=5 is a fitting choice; MAR, support, and nuisance-rate controls belong to the identification and inferential argument. A changed exclusion can alter the population, weights, fitted nuisances, estimates, diagnostics, and report text. Trace and rebuild their dependencies together.

Additional practice (optional)

Write a self-contained caption for an empirical omics contrast without an oracle truth line.
Draft a methods paragraph and a reproducibility appendix for one checkpoint analysis; have a reader identify the target and independent unit.

8 Reading

Morris, White, and Crowther, Using simulation studies to evaluate statistical methods.
Wilson et al., Good enough practices in scientific computing.

9 Continue through the guide

Return to the Research practice and reproducibility overview or use Completion checkpoints to assess the completion targets.

For assumption-dependent conclusions, use Sensitivity analysis for assumptions and research decisions. Practice reconstructing the reporting argument in the group’s published manuscripts.

--- title: "Reporting a statistical methods study" subtitle: "Connect each claim to a target, assumptions, uncertainty, and executable evidence" guide-status: "Available" --- **Before reading:** Basic estimates, standard errors, and intervals. [Evaluating a statistical method](../method-development/evaluating-methods.qmd) supports the performance discussion. The executable teaching example reuses the estimator from [From an influence function to an estimator](../semiparametric-methods/influence-function-to-estimator.qmd); you can read the reporting principles before completing that derivation. ## Make the claim traceable A methods report connects a scientific or methodological claim to a target, assumptions, procedure, and evidence. A reader should be able to reconstruct why the reported number supports the sentence written about it. Start with the problem: what quantity matters, in which population and on what scale, and why an existing procedure may be inadequate. Define the independent observation and the observed data. Distinguish assumptions identifying the target from assumptions supporting an estimator or uncertainty calculation. A causal claim requires a causal identification argument; a descriptive association does not become causal through adjustment alone. Then describe the estimator precisely enough to reproduce: estimating equation or objective, nuisance models and tuning, fold construction, preprocessing, fitting failures/retries, and confidence-interval calculation. State whether the result is an estimate of the scientific target, a model projection, or an approximation to another procedure. A claim has several components that can be checked separately. “We estimated an adjusted difference” needs a definition of adjustment and the population over which it is averaged. “With 95% confidence intervals” needs an interval construction and its conditions. “More accurate” needs a target, a loss or performance measure, a comparator, and an evaluation setting. Start from those concrete statements rather than describing a method as universally robust or superior. ## Match evidence to the claim | Claim | Evidence to report | What it does not establish alone | |---|---|---| | The target is identified | Observed-data functional and explicit assumptions | That assumptions hold in the application | | An estimator has a stated limit | Argument and conditions, including remainder control | Finite-sample accuracy everywhere | | Intervals attain nominal coverage in a scenario | Repeated-sampling coverage, MCSE, failures, scenario definition | Coverage in all populations | | A method is efficient in a model | Relevant efficiency bound and an attainment argument | A small SE in one dataset | | An omics analysis is scientifically informative | Effect scale, uncertainty, design, diagnostics, biological context | Ground-truth validation of every feature | | A result is reproducible | Inputs, environment, configuration, code, execution record | Scientific correctness | Use [Evaluating a statistical method](../method-development/evaluating-methods.qmd) to define performance measures before writing a comparison. In a real-data example, truth is usually unknown; call it an application or illustration unless a validation target is actually available. ### State a theorem at the scope it supports For an influence-function estimator, a useful theoretical statement identifies the target, statistical model, sampling assumptions, nuisance controls, and conclusion. For example, a first-order expansion has the form $$ \hat\mu-\mu=\frac1n\sum_{i=1}^n\phi_P(W_i)+o_p(n^{-1/2}), $$ where $W_i$ is one person's complete observed record and the contribution is mean zero with finite positive variance. For i.i.d. people, together with variance consistency, it supports asymptotic normal inference. It does not assert exact unbiasedness or 95% coverage at every finite n. If efficiency is claimed, specify the model's canonical gradient and explain why the estimator attains its variance bound. A small reported SE alone is not that argument. For the missing-outcome example below, identification needs an outcome-observation assumption and support. The estimator additionally needs suitable nuisance consistency, a negligible product remainder, moment control, and a variance argument. Cross-fitting addresses a particular empirical term, not every source of error. Put the precise theorem or a clear reference near the claim; do not make a reader infer its conditions from an algorithm's name. ## A worked synthetic reporting example Reuse the estimator from [From an influence function to an estimator](../semiparametric-methods/influence-function-to-estimator.qmd) so that the report and estimator share one implementation. Its [downloadable R functions](../examples/missing-mean.R) generate independent synthetic people; the true full-population mean is $7/3$. Covariates are available for everyone. Outcomes are missing at random given $Z$ in this generator, with observation probabilities bounded away from zero. These are known features of a teaching simulation, not verified assumptions about a clinical dataset. ```{r} source("../examples/missing-mean.R") example <- SimulateMissingMean(n = 800, seed = 51) fit <- CrossFitMissingMean(example$data, K = 5, seed = 52) s <- fit$summary s ``` The next paragraph is generated from that result. It names the target, units, method, uncertainty, and observation count. It does not claim that a single covering interval establishes 95% coverage. ```{r} #| results: asis cat(sprintf(paste0( "Among %d independently generated people, %d had observed outcomes. ", "For the full-population mean outcome in the synthetic measurement units, ", "the five-fold cross-fitted augmented estimator was %.3f ", "(estimated SE %.3f; approximate 95%% Wald interval %.3f to %.3f). ", "Inference uses the MAR/overlap and nuisance-rate conditions in the capstone; ", "the generator and fitted model classes satisfy its teaching setup. ", "Fitted probabilities were clipped for %.1f%% of people and fitted means for %.1f%%.\n\n"), s$n, s$observed, s$estimate, s$se, s$lower, s$upper, 100 * s$probability_clipped, 100 * s$mean_clipped)) ``` A real-data paragraph would name the assay scale, actual sampling design, population, acquisition and exclusion rules, and justification for MAR and overlap; it would not have access to the unobserved population truth. If the target were a log-scale contrast, interpret it on that scale or explicitly state the transformation used for a ratio interpretation. <span id="fig-report-interval"></span> ### Walk from the result to the sentence The person and observation counts come from `s$n` and `s$observed`. The point estimate, SE, and endpoints come from the same `fit$summary`. The five-fold choice comes from the call to the estimator. The MAR and nuisance-rate assertions come from the generator and theorem, not from the numeric result table. Keep that distinction when auditing a paragraph: some words are computed facts, while others state the assumptions under which those facts are interpreted. Because this is synthetic, we can also inspect its realized estimation error: ```{r} #| results: asis error <- s$estimate - example$truth cat(sprintf(paste0( "The generator's true mean is %.3f. This estimate differs from it by %.3f, ", "or %.2f estimated SEs. This is one realized error; ", "it is not an estimate of the procedure's bias or coverage.\n\n"), example$truth, error, error/s$se)) ``` Bias averages signed error across repeated datasets. Coverage averages whether the target is included by their intervals. A single interval contributes one zero-or-one coverage outcome, not evidence that its construction achieves 95%. The stated 95% level describes a repeated-sampling property under the relevant conditions; it is not a probability assigned to the fixed mean after seeing this interval. For a real analysis, retain the output object or table and generate the paragraph's numbers from it when rendering the manuscript. If an exclusion changes the analysis population, regenerate the estimate, denominator, diagnostics, and associated text together. Updating the displayed number while leaving the old population description is a substantive reporting error even if the number itself is correct. ## Report an evaluation with its uncertainty A simulation table should identify scenarios, target values, attempted replicate counts, successful counts, bias, empirical SD, mean SE, interval coverage, and MCSEs for the summaries. Retain method pairing and report runtime or resource requirements if they are part of the claim. Explain whether a failure is counted as an unusable interval or omitted from a conditional summary. Avoid declaring one method superior from a difference smaller than its simulation uncertainty. Separate absolute calibration from relative performance. “Coverage was near 95%” requires its estimate and MCSE, sample size, scenario, and failure denominator. “Coverage improved” additionally needs a paired difference and its uncertainty when methods used the same datasets. A difference of 0.01 with MCSE 0.012 is an imprecise estimate of improvement, not strong evidence of a gain. Conversely, lack of a clear difference does not establish equivalence; a practical equivalence claim needs its own criterion and precision. Report the regimes in which the theory is being checked, those near a boundary, and deliberate failure cases. A good working-model scenario shows that an implementation can behave as predicted there. It does not establish robustness to every misspecification. If a method fails on difficult datasets, report those attempts alongside any summaries conditional on success. A figure caption should be understandable without reconstructing the entire analysis: state what is plotted, units and target, independent-unit and replicate counts as relevant, what points/lines/bands mean, and whether a reference line is known truth, a null, or an estimate. Standard error bars, confidence intervals, empirical SDs, and MCSE bars need different labels. For feature-wise omics results, name the hypothesis family and multiplicity procedure, effect scale, adjustment variables, and selection/filtering decisions. A small adjusted p-value does not imply a large effect, a causal effect, or independent biological replication. More discoveries can reflect stronger effects, a different family, a changed target, a more liberal test, or better power. State which comparison is intended and how error control was assessed. A pointwise interval displayed for a selected feature does not become simultaneous or selection-adjusted because the p-values received an FDR adjustment. A comparison of network significance labels likewise does not replace a direct test of an edge contrast. ## Choose displays that answer a question A single estimate and interval can be reported clearly in a sentence. Use a plot when its structure reveals something the reader needs to assess: heterogeneity across features, bias as n grows, failure under a changed assumption, or a tradeoff across methods. A graphic that only repeats two numbers from the surrounding sentence adds little. For example, the balance-by-mean-shift plot in [Simulation studies for methods research](../method-development/simulation-studies.qmd) reveals an interaction that a single rejection rate conceals. Its caption must distinguish the true equal-correlation null, the simulated datasets, the changing mean shifts, and the binomial reference band. The curve in [Sensitivity analysis for assumptions and research decisions](../method-development/sensitivity-analysis.qmd) instead varies an assumption on one observed dataset; its shading represents conditional sampling uncertainty. Calling both shaded regions simply “95% uncertainty” would hide different meanings. Label a reference line as known simulation truth, a hypothesized null, or an estimated benchmark. A full-data or external estimate is not an oracle; it has its own sampling variation, target, and possible measurement error. Captions should also state whether features were prespecified for display or selected after seeing results. A reader should be able to tell what comparison the figure supports before reading a conclusion about it. ## Keep a compact reporting contract A complete first draft can follow this sequence: 1. **Question and target:** population, measurement scale, independent unit, estimand. 2. **Identification and design:** observed data, missingness/selection, assumptions, support and confounding limitations. 3. **Procedure and uncertainty:** exact algorithm, nuisance fitting/folds/tuning, variance/interval formula, failure policy. 4. **Evaluation:** scenarios and truth where known, comparisons, replicate counts, metrics and MCSE. 5. **Application and interpretation:** data provenance, effect estimates, diagnostics, limits of real-data evidence. 6. **Reproduction:** code revision, input manifest, configuration, environment, run instructions, and retained outputs. Separate prespecified choices from exploratory revisions and discuss where conclusions are sensitive to them. Link each displayed result to the code or retained output that produced it. Use [A reproducible research project](reproducible-projects.qmd) and [Data provenance and an analysis-ready dataset](data-provenance.qmd) to make this contract executable. Before sharing a draft, choose one central sentence and follow it backwards: the displayed value, retained result, run configuration, data version and exclusions, then the target and assumptions. If a link is missing, repair that evidence chain. Then read forwards: does the sentence claim only what that chain supports? This small audit is more useful than checking that every methods section contains the same fashionable technique names. ## Exercises Try each checkpoint before opening its hint or solution. ::: {#checkpoint-write-01 .checkpoint} ### Checkpoint · WRITE-01 — Rewrite a claim to match its evidence {.unnumbered} **Task type:** Interpretation and reporting. A draft says: “Our new method is unbiased and efficient because its confidence interval contains the true effect in one simulated dataset. In the omics application it discovers more significant genes, proving better accuracy.” Rewrite this as a defensible description and list the evidence still needed for the stronger claims. Include the distinction between an estimate’s SE and a simulation summary’s MCSE. For the generated paragraph in this page, identify which statements come from result fields, which come from configuration, and which depend on assumptions or theory. State what needs to be regenerated if an exclusion changes the target population. **Ready to continue when:** you can state what the reported evidence supports and identify the missing target, assumptions, uncertainty, or validation needed for each stronger claim. **Transfer:** How would the caption change if the dashed reference line were a full-data estimate rather than known simulation truth? <details class="checkpoint-help"> <summary>Hint</summary> One interval is one random outcome; discovery counts mix signal, calibration, filtering, and power. </details> <details class="checkpoint-help"> <summary>Solution outline — open after trying</summary> Report the one estimate, its stated target/scale, SE, interval, and the known synthetic truth as an illustration. Assess bias and coverage over independent replicates with MCSE and failures; efficiency needs a relevant bound and an attainment argument. In the application report effect sizes, uncertainty, multiplicity family/procedure, and assumptions. More discoveries alone do not establish accuracy when truth is unknown. SE concerns estimator variation across samples; MCSE concerns uncertainty in a simulation summary from finitely many replicates. The counts and estimates are result fields; K=5 is a fitting choice; MAR, support, and nuisance-rate controls belong to the identification and inferential argument. A changed exclusion can alter the population, weights, fitted nuisances, estimates, diagnostics, and report text. Trace and rebuild their dependencies together. </details> ::: ### Additional practice (optional) {.unnumbered} 1. Write a self-contained caption for an empirical omics contrast without an oracle truth line. 2. Draft a methods paragraph and a reproducibility appendix for one checkpoint analysis; have a reader identify the target and independent unit. ## Reading - Morris, White, and Crowther, [Using simulation studies to evaluate statistical methods](https://pmc.ncbi.nlm.nih.gov/articles/PMC6492164/). - Wilson et al., [Good enough practices in scientific computing](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005510). ## Continue through the guide {#continue-through-the-collection} Return to the [Research practice and reproducibility](index.qmd) overview or use [Completion checkpoints](../estimation-and-inference/checkpoints.qmd) to assess the completion targets. For assumption-dependent conclusions, use [Sensitivity analysis for assumptions and research decisions](../method-development/sensitivity-analysis.qmd). Practice reconstructing the reporting argument in the group’s [published manuscripts](../../publications.qmd).