Population, target, support, and interpretation.
Before reading: Elementary means, conditional means, and basic regression. The observation and sampling unit are introduced here; no assay-specific biology is assumed.
“Compare the patients and controls” does not specify a unique estimand. You still need a feature scale, population, time, contrast, adjustment, and weighting rule. A coefficient returned by software is a candidate answer only after those choices are connected to its population meaning.
A useful first statement is: among the eligible people in this cohort, compare the mean log-expression of feature g between groups, standardized to a specified covariate distribution. A different question might concern prediction for a new person, a within-person trajectory, or an intervention effect. Those questions can use related data and algorithms while requiring different assumptions and evaluation criteria.
Make the statement operational. “Eligible” needs inclusion criteria and a time window. “Mean expression” needs the measured scale and a rule for repeated specimens. “Adjusted” needs both the covariates and the population over which comparisons are averaged. These are parts of the question, not details to leave to the software defaults.
For a first example, Y is one measured feature level per eligible person on a stated scale. Z is recorded for everyone, and R says whether Y was observed. The target is the eligible-population mean \mu=E[Y], and the observed record is W=(Z,R,RY). Retain R so that an unrecorded outcome is not confused with an observed zero.
Here the eligibility rule defines the population from which people are sampled. The expectation is over that population’s distribution, not just an average of the outcomes that happened to be observed. A mean for a fixed finite enrolled cohort is another possible target, but it needs that finite-population interpretation and its sampling/observation argument stated explicitly.
A study can contain repeated specimens from a person. The independent sampling unit for a person-level question is the person, not automatically the specimen, cell, or matrix column. Units of analysis, replication, and study design develops the weighting and replication choices. An assay’s detailed interpretation comes later in Omics data and measurement.
A difference between two population means is a subsequent extension. Standardization then asks which covariate distribution should be used for each mean; the binary-stratum example below keeps those weights visible.
A descriptive target summarizes a specified population. An associational target relates variables within that population, possibly after adjustment. A predictive target concerns a loss or accuracy measure in a stated deployment population, for a learning rule with a defined training procedure. A causal target concerns outcomes under interventions and needs assumptions linking those potential outcomes to observations.
The categories describe the interpretation being claimed, not whether a method is sophisticated. Flexible regression can estimate an associational target. A simple mean can identify a causal contrast in an appropriately randomized design.
For a target \psi, definition says what quantity is wanted. Identification expresses it as a unique functional of the observable distribution under assumptions. Estimation constructs a finite-sample procedure for that functional. Inference studies uncertainty and approximation errors.
For the missing mean, definition names all eligible people’s Y values as the quantity of interest, even though some are unobserved. Identification must explain how those missing outcomes relate to what was recorded. Estimation then replaces the identified population averages or regressions by sample quantities. Inference asks how much that estimate varies and which approximation errors remain. A precise estimate at the last step cannot supply an assumption missing at the second.
For example, E[Y\mid A=1]-E[Y\mid A=0] is an observed-data group contrast if both groups occur and the means exist. A standardized contrast
\begin{aligned} \psi_Q&=\int\{m_1(z)-m_0(z)\}\,dQ(z),\\ m_a(z)&=E[Y\mid A=a,Z=z], \end{aligned}
also specifies a covariate distribution Q. Identifying it from the observed data requires that the needed group–covariate combinations are supported, or additional extrapolation restrictions. Choosing Q as the overall cohort covariate distribution differs from choosing the control or patient distribution.
This displayed contrast initially assumes fully observed outcomes and group conditional means defined on the support used by Q. With missing outcomes, the full-group means additionally need an observation-process argument. If a group never occurs in a target stratum, its conditional mean there is not uniquely defined by that observed joint law. An extrapolated response model or a potential-outcome formulation must define what is intended there before asking how to identify it.
This separate numerical example concerns a group contrast. Consider a binary covariate Z with population probability P(Z=1)=1/2. Group membership is associated with Z: P(A=1\mid Z=0)=0.2 and P(A=1\mid Z=1)=0.8. The conditional means are m_0(z)=1+2z and m_1(z)=2+3z.
The within-Z contrasts are 1 and 2. Predict the contrast averaged equally over Z, and compare it with the crude group difference.
crude standardized at_Z0 at_Z1
3.0 1.5 1.0 2.0 The answers are 3, 1.5, 1, and 2. Work through the weights: group 1 has stratum probabilities 0.2 and 0.8, so its mean is 0.2(2)+0.8(5)=4.4. Group 0 has probabilities 0.8 and 0.2, so its mean is 0.8(1)+0.2(3)=1.4. Subtracting gives 3.
Equal-Z standardization instead compares the means within each stratum, then uses the same weights: 0.5(2-1)+0.5(5-3)=1.5. Using the patient covariate distribution gives 0.2(1)+0.8(2)=1.8; using the control distribution gives 1.2. None of these answers is intrinsically wrong. They summarize different comparisons. Calling one “corrected” is insufficient without explaining why its adjustment and target population answer the question.
If the target is E[Y^1-Y^0], a common identification argument uses consistency, conditional exchangeability Y^a\perp A\mid Z, and treatment positivity over the target covariate support. These are distinct from choosing a flexible regression learner. Under them, standardization to the target population’s Z distribution identifies the causal mean contrast.
Here consistency means Y=Y^A: the observed outcome agrees with the specified intervention received. This causal linking assumption is different from statistical consistency, which later describes an estimator approaching its target. Conditional exchangeability lets us replace an intervention-specific conditional mean by an observed group mean at the same Z. Positivity supplies observations for each intervention comparison required by the target population. These are separate roles; fitting an accurate prediction model addresses none of them by itself.
The assumptions must match the intervention and study. A group label such as disease status may not define a sufficiently specified intervention. Adjusting for a variable affected by the exposure can change the causal question. Unmeasured common causes are not removed by a good prediction score. This collection introduces the identification distinction; deeper causal designs are a separate branch.
In Y=\beta X+g(Z)+\varepsilon with E[\varepsilon\mid X,Z]=0, residualizing X and Y identifies \beta when residual exposure variation is positive. That coefficient has a model-based interpretation. A causal effect interpretation additionally needs an intervention and identifying assumptions. Outside the model, the same sample equation may approach a projection-like functional rather than the original scientific contrast.
Use the What is your estimator actually estimating? page to study that limit. Do not use model flexibility as a substitute for identifying the quantity in the first place.
No observations in a sample cell need not mean its population probability is zero. With positive population support, an empty finite cell is an estimation difficulty: more suitable observations may supply information. A structural zero is different. If outcomes are never observed in a needed population stratum, even an infinite sample from that process leaves its outcome mean unknown without additional restrictions.
For the exercise below, both groups still occur in both Z strata. What becomes zero is the probability of observing an outcome in group 0 at Z=1. Its full-population conditional mean is therefore a defined quantity, but the observed record contains no outcomes that reveal it. Keeping group membership and outcome observation separate makes the nonidentification argument precise.
Record the eligible population and sampling frame; independent unit; assay feature and measurement scale; time or visit summary; contrast and weighting; covariate distribution; missingness assumptions; and interpretation. Then write the observable functional and list the support needed to evaluate it. If a required combination is absent, state whether the analysis changes population, introduces extrapolation, or cannot identify the target.
A short specification should allow another analyst to disagree about the assumptions while still knowing exactly which quantity you intend to estimate. This is more useful than choosing an algorithm name as the analysis plan.
When identification depends on an untestable restriction, use Sensitivity analysis for assumptions and research decisions to distinguish a changed assumption from a changed target.
Try each checkpoint before opening its hint or solution.
Task type: Interpretation and derivation.
Use the binary-Z example to define the equal-Z standardized contrast and derive 1.5. Suppose both groups still occur in both strata, but P(R=1\mid A=0,Z=1)=0 in the population, while outcomes are observed in the other cells and the target gives Z=1 weight 1/2. Which conditional mean is unsupported? Describe two observed-data-compatible choices for it that give different targets. Would a flexible learner or a sandwich standard error resolve this?
Ready to continue when: you can show nonidentification by constructing different compatible targets, rather than describing it only as a software difficulty.
Transfer: If you restrict the target to Z=0, what is gained and what population claim is lost?
The target still needs m_0(1) even when that conditional mean cannot be observed.
The contrast is \{m_1(0)-m_0(0)+m_1(1)-m_0(1)\}/2. With all means supplied it is (1+2)/2=1.5. If group 0 still occurs at Z=1 but none of its outcomes are observed, changing its full-group conditional mean from 3 to 7 leaves the law of (Z,A,R,RY) unchanged but changes the contrast from 1.5 to -0.5. Additional restrictions, a new population target, or new observations are needed. Learning and variance estimation cannot identify an unrestricted mean in an unsupported stratum.
---
title: "From a research question to an identified estimand"
subtitle: "Population, target, support, and interpretation."
guide-status: "Available"
---
**Before reading:** Elementary means, conditional means, and basic regression. The observation and sampling unit are introduced here; no assay-specific biology is assumed.
## Write the question in population language
“Compare the patients and controls” does not specify a unique estimand. You still need a feature scale, population, time, contrast, adjustment, and weighting rule. A coefficient returned by software is a candidate answer only after those choices are connected to its population meaning.
A useful first statement is: among the eligible people in this cohort, compare the mean log-expression of feature $g$ between groups, standardized to a specified covariate distribution. A different question might concern prediction for a new person, a within-person trajectory, or an intervention effect. Those questions can use related data and algorithms while requiring different assumptions and evaluation criteria.
Make the statement operational. “Eligible” needs inclusion criteria and a time window. “Mean expression” needs the measured scale and a rule for repeated specimens. “Adjusted” needs both the covariates and the population over which comparisons are averaged. These are parts of the question, not details to leave to the software defaults.
## A running population mean
For a first example, Y is one measured feature level per eligible person on a stated scale. Z is recorded for everyone, and R says whether Y was observed. The target is the eligible-population mean $\mu=E[Y]$, and the observed record is $W=(Z,R,RY)$. Retain R so that an unrecorded outcome is not confused with an observed zero.
Here the eligibility rule defines the population from which people are sampled. The expectation is over that population's distribution, not just an average of the outcomes that happened to be observed. A mean for a fixed finite enrolled cohort is another possible target, but it needs that finite-population interpretation and its sampling/observation argument stated explicitly.
A study can contain repeated specimens from a person. The independent sampling unit for a person-level question is the person, not automatically the specimen, cell, or matrix column. [Units of analysis, replication, and study design](units-and-study-design.qmd) develops the weighting and replication choices. An assay’s detailed interpretation comes later in [Omics data and measurement](../data-and-measurement/index.qmd).
A difference between two population means is a subsequent extension. Standardization then asks which covariate distribution should be used for each mean; the binary-stratum example below keeps those weights visible.
## Four kinds of question
A descriptive target summarizes a specified population. An associational target relates variables within that population, possibly after adjustment. A predictive target concerns a loss or accuracy measure in a stated deployment population, for a learning rule with a defined training procedure. A causal target concerns outcomes under interventions and needs assumptions linking those potential outcomes to observations.
The categories describe the interpretation being claimed, not whether a method is sophisticated. Flexible regression can estimate an associational target. A simple mean can identify a causal contrast in an appropriately randomized design.
## Definition and identification are different steps
For a target $\psi$, **definition** says what quantity is wanted. **Identification** expresses it as a unique functional of the observable distribution under assumptions. **Estimation** constructs a finite-sample procedure for that functional. **Inference** studies uncertainty and approximation errors.
For the missing mean, definition names all eligible people's Y values as the quantity of interest, even though some are unobserved. Identification must explain how those missing outcomes relate to what was recorded. Estimation then replaces the identified population averages or regressions by sample quantities. Inference asks how much that estimate varies and which approximation errors remain. A precise estimate at the last step cannot supply an assumption missing at the second.
For example, $E[Y\mid A=1]-E[Y\mid A=0]$ is an observed-data group contrast if both groups occur and the means exist. A standardized contrast
$$
\begin{aligned}
\psi_Q&=\int\{m_1(z)-m_0(z)\}\,dQ(z),\\
m_a(z)&=E[Y\mid A=a,Z=z],
\end{aligned}
$$
also specifies a covariate distribution $Q$. Identifying it from the observed data requires that the needed group–covariate combinations are supported, or additional extrapolation restrictions. Choosing $Q$ as the overall cohort covariate distribution differs from choosing the control or patient distribution.
This displayed contrast initially assumes fully observed outcomes and group conditional means defined on the support used by Q. With missing outcomes, the full-group means additionally need an observation-process argument. If a group never occurs in a target stratum, its conditional mean there is not uniquely defined by that observed joint law. An extrapolated response model or a potential-outcome formulation must define what is intended there before asking how to identify it.
<span id="fig-question-roadmap"></span>
<span id="fig-question-roadmap-title"></span>
## A contrast can change after standardization
This separate numerical example concerns a group contrast. Consider a binary covariate $Z$ with population probability $P(Z=1)=1/2$. Group membership is associated with $Z$: $P(A=1\mid Z=0)=0.2$ and $P(A=1\mid Z=1)=0.8$. The conditional means are $m_0(z)=1+2z$ and $m_1(z)=2+3z$.
The within-$Z$ contrasts are 1 and 2. Predict the contrast averaged equally over $Z$, and compare it with the crude group difference.
```{r}
pz <- c(0.5, 0.5); pa <- c(0.2, 0.8)
m0 <- c(1, 3); m1 <- c(2, 5)
pz_a1 <- pz * pa / sum(pz * pa)
pz_a0 <- pz * (1 - pa) / sum(pz * (1 - pa))
c(crude = sum(pz_a1 * m1) - sum(pz_a0 * m0),
standardized = sum(pz * (m1 - m0)),
at_Z0 = m1[1] - m0[1], at_Z1 = m1[2] - m0[2])
```
The answers are 3, 1.5, 1, and 2. Work through the weights: group 1 has stratum probabilities 0.2 and 0.8, so its mean is $0.2(2)+0.8(5)=4.4$. Group 0 has probabilities 0.8 and 0.2, so its mean is $0.8(1)+0.2(3)=1.4$. Subtracting gives 3.
Equal-Z standardization instead compares the means **within** each stratum, then uses the same weights: $0.5(2-1)+0.5(5-3)=1.5$. Using the patient covariate distribution gives $0.2(1)+0.8(2)=1.8$; using the control distribution gives 1.2. None of these answers is intrinsically wrong. They summarize different comparisons. Calling one “corrected” is insufficient without explaining why its adjustment and target population answer the question.
## A causal interpretation needs an additional argument
If the target is $E[Y^1-Y^0]$, a common identification argument uses consistency, conditional exchangeability $Y^a\perp A\mid Z$, and treatment positivity over the target covariate support. These are distinct from choosing a flexible regression learner. Under them, standardization to the target population's $Z$ distribution identifies the causal mean contrast.
Here consistency means $Y=Y^A$: the observed outcome agrees with the specified intervention received. This causal linking assumption is different from statistical consistency, which later describes an estimator approaching its target. Conditional exchangeability lets us replace an intervention-specific conditional mean by an observed group mean at the same Z. Positivity supplies observations for each intervention comparison required by the target population. These are separate roles; fitting an accurate prediction model addresses none of them by itself.
The assumptions must match the intervention and study. A group label such as disease status may not define a sufficiently specified intervention. Adjusting for a variable affected by the exposure can change the causal question. Unmeasured common causes are not removed by a good prediction score. This collection introduces the identification distinction; deeper causal designs are a separate branch.
## The partially linear coefficient is another target
In $Y=\beta X+g(Z)+\varepsilon$ with $E[\varepsilon\mid X,Z]=0$, residualizing $X$ and $Y$ identifies $\beta$ when residual exposure variation is positive. That coefficient has a model-based interpretation. A causal effect interpretation additionally needs an intervention and identifying assumptions. Outside the model, the same sample equation may approach a projection-like functional rather than the original scientific contrast.
Use the [What is your estimator actually estimating?](estimands-and-misspecification.qmd) page to study that limit. Do not use model flexibility as a substitute for identifying the quantity in the first place.
## Distinguish structural absence from a small sample
No observations in a sample cell need not mean its population probability is zero. With positive population support, an empty finite cell is an estimation difficulty: more suitable observations may supply information. A structural zero is different. If outcomes are never observed in a needed population stratum, even an infinite sample from that process leaves its outcome mean unknown without additional restrictions.
For the exercise below, both groups still occur in both Z strata. What becomes zero is the probability of **observing an outcome** in group 0 at Z=1. Its full-population conditional mean is therefore a defined quantity, but the observed record contains no outcomes that reveal it. Keeping group membership and outcome observation separate makes the nonidentification argument precise.
## Write an estimand specification
Record the eligible population and sampling frame; independent unit; assay feature and measurement scale; time or visit summary; contrast and weighting; covariate distribution; missingness assumptions; and interpretation. Then write the observable functional and list the support needed to evaluate it. If a required combination is absent, state whether the analysis changes population, introduces extrapolation, or cannot identify the target.
A short specification should allow another analyst to disagree about the assumptions while still knowing exactly which quantity you intend to estimate. This is more useful than choosing an algorithm name as the analysis plan.
When identification depends on an untestable restriction, use [Sensitivity analysis for assumptions and research decisions](../method-development/sensitivity-analysis.qmd) to distinguish a changed assumption from a changed target.
## Exercises
Try each checkpoint before opening its hint or solution.
::: {#checkpoint-id-01 .checkpoint}
### Checkpoint · ID-01 — Separate definition, identification, and estimation {.unnumbered}
**Task type:** Interpretation and derivation.
Use the binary-$Z$ example to define the equal-$Z$ standardized contrast and derive 1.5. Suppose both groups still occur in both strata, but $P(R=1\mid A=0,Z=1)=0$ in the population, while outcomes are observed in the other cells and the target gives Z=1 weight 1/2. Which conditional mean is unsupported? Describe two observed-data-compatible choices for it that give different targets. Would a flexible learner or a sandwich standard error resolve this?
**Ready to continue when:** you can show nonidentification by constructing different compatible targets, rather than describing it only as a software difficulty.
**Transfer:** If you restrict the target to $Z=0$, what is gained and what population claim is lost?
<details class="checkpoint-help">
<summary>Hint</summary>
The target still needs $m_0(1)$ even when that conditional mean cannot be observed.
</details>
<details class="checkpoint-help">
<summary>Solution outline — open after trying</summary>
The contrast is $\{m_1(0)-m_0(0)+m_1(1)-m_0(1)\}/2$. With all means supplied it is $(1+2)/2=1.5$. If group 0 still occurs at $Z=1$ but none of its outcomes are observed, changing its full-group conditional mean from 3 to 7 leaves the law of $(Z,A,R,RY)$ unchanged but changes the contrast from 1.5 to -0.5. Additional restrictions, a new population target, or new observations are needed. Learning and variance estimation cannot identify an unrestricted mean in an unsupported stratum.
</details>
:::
### Additional practice (optional) {.unnumbered}
1. Replace the overall covariate distribution by the patient distribution and compute the standardized contrast.
2. Write separate estimand specifications for a bulk-expression contrast and a composition-adjusted contrast.
3. For a prediction study, specify deployment population, loss, training procedure, and validation unit.
## Reading
- Kennedy, [Semiparametric theory](https://arxiv.org/abs/1709.06418), for an estimation framework with explicitly defined target functionals.
<span id="continue-through-the-collection"></span>