From an influence function to an estimator

A missing-outcome mean capstone: identification, construction, and proof.

Before reading: Missingness, selection, and the observed-data target, Influence functions, Orthogonality and cross-fitting, Estimating nuisance functions for inference, and Large-sample reasoning for methods research.

1 One target, a complete estimation argument

The target is the eligible-population mean \mu=E[Y] when some outcomes are missing. We observe i.i.d. records W=(Z,R,RY) with covariates recorded for everyone. Assume E[|Y|]<\infty, missing at random (MAR), R\perp Y\mid Z, and \pi_0(Z)>0 almost surely in the target population. The Missingness, selection, and the observed-data target page explains why these conditions identify the mean as E[m_0(Z)], where m_0(z)=E[Y\mid Z=z,R=1] and \pi_0(z)=P(R=1\mid Z=z).

Write Pf=E[f(W)] under the true observed-data law and P_nf=n^{-1}\sum_i f(W_i). For fitted functions, P means expectation over a fresh record while holding the training fit fixed. The L2 norm is \|f\|_2=\sqrt{P f^2}. When f depends only on Z, this averages over all eligible people’s covariates. Positivity and MAR identify the target; stronger bounds and learning conditions will be imposed separately for the inference proof.

This is an extended capstone. First follow the derivation, then inspect the implementation, and finally reconstruct the proof through the checkpoints. The same pattern transfers to residual covariance and partially linear coefficients, but their gradients and remainders must be derived for their own targets.

2 Derive the observed-data influence function

An observed-data density factors into the distribution of Z, the observation mechanism given Z, and the outcome distribution among observed units. Writing w=(z,r,ry) for an observed record value and p_1(y\mid z) for the last density, its form is

p(w)=p_Z(z)\pi(z)^r\{1-\pi(z)\}^{1-r} \times p_1(y\mid z)^r.

The outcome factor appears only when r=1; the notation does not evaluate an unrecorded y. A smooth path has total score

s(W)=s_Z(Z)+s_{R\mid Z}(R,Z) +Rs_Y(Y,Z),

where s_Y abbreviates the observed conditional-outcome score. Density normalization gives E[s_Z]=0, E[s_{R\mid Z}\mid Z]=0, and E[s_Y\mid Z,R=1]=0. These are the three sets of permitted local directions whose target derivatives must be represented.

Differentiate the identified functional \mu(P)=E_P[m_P(Z)]. Changing the covariate distribution contributes E[(m_0-\mu)s_Z], because its score is centered. At a fixed z, differentiating the outcome mean contributes E[(Y-m_0(z))s_Y\mid Z=z,R=1]. Averaging over target covariates gives

E_Z\!\left[E[(Y-m_0(Z))s_Y\mid Z,R=1]\right].

Changing the observation probability alone does not change the identified functional: its expression uses p_Z and p_1, not \pi. This does not make missingness irrelevant to precision; fewer observed outcomes can still mean less information. A candidate gradient representing these derivatives is

\phi_0(W)=m_0(Z)-\mu +\frac{R}{\pi_0(Z)}\{Y-m_0(Z)\}.

2.1 Verify each score component

Conditioning first on (Z,R) gives E[\phi_0\mid Z,R]=m_0(Z)-\mu: when R=1, the observed residual has mean zero; when R=0, its weighted term is absent. Averaging gives E[\phi_0]=0.

For covariate scores, the same centering gives E[\phi_0s_Z]=E[(m_0-\mu)s_Z]. For observation-mechanism scores,

\begin{aligned} E[\phi_0s_{R\mid Z}] &=E[(m_0-\mu)E[s_{R\mid Z}\mid Z]]\\ &=0. \end{aligned}

For outcome scores, the m_0-\mu term vanishes because the conditional score is centered. The remaining product has R^2=R, so

\begin{aligned} E[\phi_0 R s_Y]&=E\left[\frac{R}{\pi_0}(Y-m_0)s_Y\right]\\ &=E_Z\!\left[E[(Y-m_0)s_Y\mid Z,R=1]\right]. \end{aligned}

The inner conditioning contributes a factor \pi_0(Z) that cancels the inverse weight. Thus the same \phi_0 represents the derivative for each score component, and hence their sums and limits under the required differentiability and integrability conditions.

2.2 Interpret the efficiency bound

In the unrestricted observed-data model the tangent space is L_2^0(P). When the candidate has finite second moment, its representation of all these derivatives makes it the unique gradient, the efficient influence function (EIF). To compute its variance, note that E[\phi_0\mid Z]=m_0(Z)-\mu and

E\left[\left.\left\{\frac{R}{\pi_0}(Y-m_0)\right\}^2\right|Z\right]=\frac{\operatorname{Var}(Y\mid Z,R=1)}{\pi_0(Z)}.

One factor of \pi_0 cancels the squared inverse weight, leaving one in the denominator. Conditional centering makes the cross term zero. The total-variance calculation gives

V_0=\operatorname{Var}(m_0(Z)) +E\left[\frac{\operatorname{Var}(Y\mid Z,R=1)}{\pi_0(Z)}\right].

V_0 is the variance of one influence contribution, or the limiting variance of the root-n scaled estimator error. The estimator-variance bound is V_0/n, and its SE bound is \sqrt{V_0/n}. Dividing by n accounts for averaging; taking the square root expresses uncertainty on the outcome scale.

The first component reflects between-covariate differences in the mean. The second reflects within-covariate outcome noise, inflated by incomplete observation. For example, if \pi_0=1/2, \operatorname{Var}(m_0)=1, and every conditional outcome variance is 4, then V_0=1+4/(1/2)=9. If every outcome were observed, the variance bound would instead be (1+4)/n=5/n. Missingness doubles the conditional-noise contribution here, rather than doubling every part of the variance. These compare two different observation experiments at the same full-data law.

3 Construct the augmented estimator

For fitted nuisances \hat\eta=(\hat m,\hat\pi), define

D_{\hat\eta}(W)=\hat m(Z) +\frac R{\hat\pi(Z)}\{Y-\hat m(Z)\}.

A plug-in starts with an average prediction P_n\hat m. Adding the empirical residual correction gives the augmented inverse-probability estimator. With fixed fits, the equation P_n(D_{\hat\eta}-\mu)=0 has the explicit solution \widehat\mu=P_nD_{\hat\eta}. This is why the EIF suggests the augmentation. Solving a sample equation does not itself show that the resulting estimator is centered at the population target.

For cross-fitting, split independent units into a fixed number K of folds independently of their data. In fold k, learn both functions on the other folds and evaluate D_{\hat\eta_k} on held-out units. The label k names the excluded validation fold. Average all held-out contributions with fold-size weights. Every row is evaluated once using a fit that did not train on that row. Repeated observations of one person need grouped folds and an appropriate cluster-level variance; the implementation below assumes independent people with one row each.

Only observed-outcome rows receive a residual correction. Missing-outcome rows still contribute their predicted means. In code, multiplying 0 * NA gives NA; the implementation constructs the correction on observed rows rather than treating the unobserved outcome as zero.

4 Compute the exact bias remainder

Hold a training fit fixed and take expectation over a fresh observation. The definition of m_0 gives E[R(Y-\widehat m)\mid Z]=\pi_0(m_0-\widehat m). Therefore

\begin{aligned} E[D_{\widehat\eta}\mid Z]-m_0&=\widehat m-m_0+\frac{\pi_0}{\widehat\pi}(m_0-\widehat m)\\ &=\frac{\widehat\pi-\pi_0}{\widehat\pi}(\widehat m-m_0). \end{aligned}

Averaging over Z gives

\begin{aligned} P D_{\hat\eta}-\mu &=E\left[\frac{\hat\pi-\pi_0}{\hat\pi} (\hat m-m_0)\right]. \end{aligned}

If \hat\pi\ge c>0, taking absolute values and applying Cauchy–Schwarz bounds the bias magnitude by

c^{-1}\|\hat\pi-\pi_0\|_2\|\hat m-m_0\|_2.

The denominator bound controls amplification of probability-estimation errors. If either supplied function equals the truth, the exact bias is zero. If one learned function converges to truth and the other’s error remains controlled, the product can vanish even when the latter has an incorrect limit. With a vanishing empirical error this yields double-robust consistency. It does not remove MAR or prove efficiency. For efficient inference using \phi_0, the proof below assumes both nuisances approach truth, a sufficiently small product, and the empirical and variance conditions stated there.

5 Decompose the full estimation error

Let I_k be fold k, n_k its size, w_k=n_k/n, and P_{n,k} its empirical average. The estimator is \widehat\mu=\sum_k w_kP_{n,k}D_{\hat\eta_k}. Put D_0=\mu+\phi_0 and H_k=D_{\hat\eta_k}-D_0. Add and subtract D_0 inside each validation average, then add and subtract PH_k. Because the folds partition the sample, \sum_k w_kP_{n,k}D_0=P_nD_0. Exactly,

\hat\mu-\mu=(P_n-P)\phi_0 +\sum_k w_k(P_{n,k}-P)H_k +\sum_k w_kPH_k.

The first term has a fixed-function CLT. The last is the population bias remainder just calculated. The middle is the centered empirical error from evaluating a learned function. A fixed-function CLT cannot simply be applied to it as if the fits were prespecified, because those fits are random and use part of the same overall dataset.

5.1 Bound one validation fold, then combine folds

Define T_k=\sqrt{n_k}(P_{n,k}-P)H_k. Conditional on fold k’s training data, H_k is fixed and its validation observations are independent. Thus E[T_k\mid\text{training}_k]=0 and

\operatorname{Var}(T_k\mid\text{training}_k)\le PH_k^2.

If PH_k^2=o_p(1), the variance bound is small with high probability. This does not require its training-sample expectation to converge to zero. For fixed \epsilon,\delta>0, separate the event where the bound exceeds \delta, and apply conditional Chebyshev on its complement:

\Pr(|T_k|>\epsilon) \le\Pr(PH_k^2>\delta)+\frac{\delta}{\epsilon^2}.

First let n grow; the first probability tends to zero. Then let \delta decrease to zero. This proves T_k=o_p(1). For the overall empirical remainder,

\begin{aligned} \sqrt n\sum_k w_k(P_{n,k}-P)H_k&=\sum_k\sqrt{w_k}\,T_k\\ &=o_p(1). \end{aligned}

There are only finitely many folds, and every weight is bounded. Their training sets overlap, so joint conditional independence of all contributions is not assumed. A fixed finite sum of negligible terms is enough. A growing number of folds, including a leave-one-out scheme, would need a further argument rather than automatic reuse of this proof.

6 A tractable sufficient set of conditions

One sufficient teaching setup uses the following conditions:

The units are i.i.d.; K is fixed; the split is independent of the data; and every fold’s size fraction stays bounded away from zero.
MAR holds, and true and fitted observation probabilities are at least a fixed c>0.
Outcomes and fitted means are bounded in absolute value by a fixed B<\infty.
Both nuisances are L2-consistent, and their L2 error product is o_p(n^{-1/2}) in every fold.
The influence variance satisfies 0<V_0<\infty.

To see why nuisance consistency controls H_k, suppress the fold label and expand exactly:

H=(\widehat m-m_0)\left(1-\frac R{\widehat\pi}\right) +R(Y-m_0)\left(\frac1{\widehat\pi}-\frac1{\pi_0}\right).

The first multiplier is bounded by 1+c^{-1}. For the second, |Y-m_0|\le2B and the inverse-probability difference is at most c^{-2}|\widehat\pi-\pi_0|. The triangle inequality therefore gives

\|H\|_2\le(1+c^{-1})\|\widehat m-m_0\|_2 +2Bc^{-2}\|\widehat\pi-\pi_0\|_2.

Both errors tending to zero makes H L2-small. The conditional-variance argument now controls the empirical remainder, while the product rate controls the population remainder. The fixed-function CLT and Slutsky’s theorem give

\begin{aligned} \widehat\mu-\mu&=P_n\phi_0+o_p(n^{-1/2}),\\ \sqrt n(\widehat\mu-\mu)&\xrightarrow{d}N(0,V_0). \end{aligned}

6.1 Justify the estimated variance

For i\in I_k, write \widehat D_i=D_{\widehat\eta_k}(W_i) and \widehat\phi_i=\widehat D_i-\widehat\mu. Their sample mean is exactly zero. The sample variance and SE are

\begin{aligned} \widehat V&=\frac1{n-1}\sum_i\widehat\phi_i^2,\\ \widehat{\operatorname{SE}}&=\sqrt{\widehat V/n}. \end{aligned}

The learned contributions share training data, so the justification is not that they are all independent. For each fold, conditioning on training gives E[P_{n,k}H_k^2\mid\text{training}_k]=PH_k^2. Conditional Markov and the same small-probability-event argument above show that the average squared difference between \widehat D_i and D_0(W_i) tends to zero. Cauchy–Schwarz then controls the difference of their empirical second moments. The LLN for the fixed true contributions and consistency of \widehat\mu give \widehat V\xrightarrow{p}V_0.

Consequently (\widehat\mu-\mu)/\widehat{\operatorname{SE}} converges in distribution to a standard normal, and \widehat\mu\pm1.96\widehat{\operatorname{SE}} is an asymptotic 95% interval. Coverage concerns repeated datasets under the stated conditions. Bounded outcomes simplify this proof; unbounded outcomes need appropriate moment and uniform-integrability controls.

7 Implement the complete teaching example

Here Z is uniform on [-1,1], m_0(Z)=2+Z+Z^2, \pi_0(Z)=\operatorname{expit}(-0.3+Z), and the outcome error is independently uniform on [-1,1]. The simulator draws R using only Z, so MAR holds by construction. Since E[Z]=0 and E[Z^2]=1/3, the population mean is \mu=2+0+1/3=7/3. Conditional outcome variance is 1/3.

The regression and observation models below are correctly specified: the outcome fit includes both Z and Z^2, and the observation fit uses a linear logit. Under their regular finite-dimensional conditions, both can attain root-n rates, giving a product of order n^{-1}, negligible on the root-n scale. This is a transparent verification case; flexible learners would need their own accuracy conditions.

The observed dataset does not retain missing outcome values. The simulator constructs them only to generate the observation process and then returns NA for missing rows. Prespecified probability bounds (0.1,0.9) contain the true probability range, approximately (0.21,0.67); the outcome/prediction bounds (0,6) also contain the truth. These teaching constraints are not learned from unavailable outcomes. Clipping to bounds containing the truth does not increase the distance from that truth. All reported inference concerns the cross-fitted augmented estimator.

Within each training set, lm uses only observed outcomes and glm uses everyone’s observed R. Predictions are made for the excluded fold, then assembled in original row order. score stores \widehat D_i; phi stores \widehat\phi_i. These names denote contributions, not likelihood scores. Centering and dividing their SD by \sqrt n implements the variance formula just justified.

The functions are also available as a downloadable R script.

Code

# Self-contained bounded-outcome teaching example; independent rows are people.
SimulateMissingMean <- function(n = 800, seed = 51) {
  set.seed(seed)
  z <- runif(n, -1, 1)
  prob <- plogis(-0.3 + z)
  full_y <- 2 + z + z^2 + runif(n, -1, 1)
  r <- rbinom(n, 1, prob)
  list(data = data.frame(id = seq_len(n), z = z, r = r,
                         y = ifelse(r == 1, full_y, NA_real_)), truth = 7 / 3)
}

CrossFitMissingMean <- function(d, K = 5, seed = 52,
                                probability_bounds = c(0.1, 0.9),
                                outcome_bounds = c(0, 6)) {
  n <- nrow(d)
  stopifnot(n >= 20, K >= 2, K <= n, !anyDuplicated(d$id),
            all(is.finite(d$z)), all(d$r %in% c(0, 1)),
            all(is.finite(d$y[d$r == 1])),
            all(is.na(d$y[d$r == 0])),
            length(probability_bounds) == 2,
            0 < probability_bounds[1], probability_bounds[1] < probability_bounds[2],
            probability_bounds[2] < 1, length(outcome_bounds) == 2,
            outcome_bounds[1] < outcome_bounds[2],
            all(d$y[d$r == 1] >= outcome_bounds[1]),
            all(d$y[d$r == 1] <= outcome_bounds[2]))
  set.seed(seed)
  fold <- sample(rep(seq_len(K), length.out = n))
  mhat <- phat <- raw_p <- raw_m <- numeric(n)
  for (k in seq_len(K)) {
    train <- d[fold != k, ]; test <- d[fold == k, ]
    observed <- train[train$r == 1, ]
    if (nrow(observed) < 6 || length(unique(train$r)) < 2) {
      stop("Training fold has inadequate observed outcomes or observation classes")
    }
    outcome_fit <- lm(y ~ z + I(z^2), data = observed)
    probability_fit <- glm(r ~ z, family = binomial(), data = train)
    if (!isTRUE(probability_fit$converged)) stop("Observation fit did not converge")
    raw_m[fold == k] <- predict(outcome_fit, newdata = test)
    raw_p[fold == k] <- predict(probability_fit, newdata = test, type = "response")
  }
  stopifnot(all(is.finite(raw_m)), all(is.finite(raw_p)))
  mhat <- pmin(outcome_bounds[2], pmax(outcome_bounds[1], raw_m))
  phat <- pmin(probability_bounds[2], pmax(probability_bounds[1], raw_p))
  score <- mhat
  observed <- d$r == 1
  # Do not multiply zero by NA: construct residual corrections on observed rows.
  score[observed] <- mhat[observed] + (d$y[observed] - mhat[observed]) / phat[observed]
  estimate <- mean(score); phi <- score - estimate
  se <- sd(phi) / sqrt(n)
  list(summary = data.frame(estimate = estimate, se = se,
                            lower = estimate - 1.96 * se, upper = estimate + 1.96 * se,
                            observed = sum(observed), n = n,
                            probability_clipped = mean(phat != raw_p),
                            mean_clipped = mean(mhat != raw_m)),
       contributions = data.frame(id = d$id, fold = fold, r = d$r, z = d$z,
                                  mhat = mhat, phat = phat, score = score, phi = phi))
}

Before running the next chunk, predict why the complete-case mean may differ from 7/3 even though observation depends only on recorded Z. Under this mechanism, the complete-case limit is E[\pi_0(Z)m_0(Z)]/E[\pi_0(Z)]: it weights covariates by their observation probabilities. MAR does not make that covariate mix match the full population. The cross-fitted estimate should be close to the truth on the scale of its SE, not identical to it.

Code

example <- SimulateMissingMean()
fit <- CrossFitMissingMean(example$data)
fit$summary

  estimate         se    lower    upper observed   n probability_clipped
1 2.354531 0.04003445 2.276063 2.432998      329 800                   0
  mean_clipped
1            0

Code

c(complete_case = mean(example$data$y[example$data$r == 1]), truth = example$truth)

complete_case         truth 
     2.510341      2.333333

Code

# Exact observed-data bound in this known simulation, not a real-data oracle.
V0 <- integrate(function(z) ((1/3) / plogis(-0.3 + z) +
                            (2 + z + z^2 - 7/3)^2) / 2, -1, 1)$value
c(theoretical_SE = sqrt(V0 / nrow(example$data)), estimated_SE = fit$summary$se)

theoretical_SE   estimated_SE 
    0.04006777     0.04003445

Code

a <- fit$contributions
plot(a$z, a$phi, pch = 16, cex = 0.55,
     col = ifelse(a$r == 1, "#2F80C8", "#D55E00"),
     xlab = "Observed covariate Z", ylab = "Estimated influence contribution")
abline(h = 0, lty = 2)
legend("topright", c("Outcome observed", "Outcome missing"),
       col = c("#2F80C8", "#D55E00"), pch = 16, bty = "n")

Influence values plotted against Z, with observed and unobserved outcomes shown in different colors. — Figure 1: Cross-fitted contributions in one synthetic sample. Large values can arise from weighted observed residuals; missing-outcome units contribute predicted means.

The integral uses the known functions and conditional variance 1/3 to compute V_0, then reports \sqrt{V_0/n}. Compare this theoretical SE with the estimated SE; neither is the SD of individual outcomes. One estimate can be above or below the truth and one estimated SE can be above or below the asymptotic benchmark. Such differences in a single sample do not establish bias, inefficiency, or improved efficiency.

In the figure, missing-outcome units have \widehat\phi_i=\widehat m(Z_i)-\widehat\mu. Observed units also receive the weighted residual, so their contributions can be more spread out, especially where estimated observation probability is small. The plot displays the terms used in the estimator and SE; it is not a check of MAR.

The code records clipping fractions and stops on inadequate training folds or nonfinite fits. A production evaluation should retain those failures rather than silently omit them. One interval containing the truth does not establish coverage; Evaluating a statistical method explains repeated-sampling assessment.

8 An introduction to targeting (optional)

This optional comparison introduces a different construction. It updates an initial regression to solve an empirical EIF equation, then reports the target evaluated at the updated regression. For a Gaussian working squared-error loss, take a one-dimensional fluctuation m_\epsilon=\widehat m+\epsilon a, where a(Z)=1/\widehat\pi(Z) is the specified update direction.

Minimizing observed-outcome squared loss gives the equation P_n[Ra(Y-m_\epsilon)]=0. Its solution, when the denominator is positive, is

\widehat\epsilon =\frac{P_n[Ra(Y-\widehat m)]}{P_n[Ra^2]}.

Now report the plug-in mean \widetilde\mu=P_nm_{\widehat\epsilon}. The mean of its prediction-minus-target term is zero by definition, and the weighted residual term is zero by the update equation. Hence the updated empirical EIF mean is zero. This is the algebra behind targeting the update toward the final parameter.

This illustrates a targeting step. A complete targeted-learning procedure must choose a suitable loss/submodel, preserve relevant bounds, handle fitted nuisances and folds, and establish its own remainder conditions. The proof and interval implementation above are for the cross-fitted augmented estimator; the illustration does not transfer that proof automatically to every targeting algorithm.

9 Diagnose the failed condition

If a positive-probability stratum has \pi_0=0, no outcomes are observed there. Two full-data populations can have the same observed law and different means in that stratum, giving different \mu. The full mean is therefore not identified without extra information. More data from the same observation mechanism do not resolve that ambiguity.

If \pi_0(Z)>0 almost surely but approaches zero, identification can remain while EIF variance diverges. For Z uniform on (0,1), \pi_0(Z)=Z, and conditional outcome variance 1, the variance contains

\begin{aligned} E[1/Z]&=\int_0^1\frac{1}{z}\,dz\\ &=\infty. \end{aligned}

Every stratum has some observation support, but the rare observations near zero carry too much inverse-probability variation for the finite-variance root-n argument. This is a precision/regularity failure, distinct from the previous identification failure.

Slow nuisance learning can leave first-order bias even with perfectly separated folds. Let \pi_0=1/2 and take constant errors \widehat\pi-\pi_0=n^{-1/5} and \widehat m-m_0=n^{-1/5} for sufficiently large n that the probabilities are admissible. The exact bias and its scaling are

\begin{aligned} b_n&=\frac{n^{-2/5}}{1/2+n^{-1/5}}\\ &\longrightarrow0,\\ \sqrt n\,b_n&=\frac{n^{1/10}}{1/2+n^{-1/5}}\\ &\longrightarrow\infty. \end{aligned}

Both learners converge and the estimator can be consistent, yet its bias overwhelms root-n uncertainty. This constructs an actual noncanceling remainder; merely failing a sufficient norm bound would not establish such failure. Cross-fitting leaves this population calculation unchanged.

Optional transfer beyond the missing-mean target. A unique median with F(y)-1/2\approx cy^3, c>0, near zero remains a defined and identified target, but its population derivative cannot be inverted by a positive density. Balancing CDF noise of order n^{-1/2} with c\widehat q^3 suggests \widehat q has order n^{-1/6}. Quantiles and local expansions explains the regular positive-density comparison. This failure belongs to the target’s local behavior, not to a missingness model or a nuisance learner.

Identification, finite influence variance, nuisance-bias control, and local differentiability are separate parts of an inference argument. A denominator cutoff cannot supply missing information about an unobserved stratum, and an SE formula cannot remove persistent bias.

10 Exercises

Checkpoints are completion targets. Additional practice is optional. Try each checkpoint before opening its hint or solution.

Checkpoint · SP-12 — Control the cross-fitted empirical-process term

Task type: Proof.

Assume i.i.d. observations and a fixed number of folds with nonvanishing validation fractions. Let H_k be the fitted-minus-true contribution, fixed conditional on fold k’s training data, with PH_k^2\to_p0. Define T_k=\sqrt{n_k}(P_{n,k}-P)H_k, where n_k is its validation size. Derive the conditional-variance bound for T_k. Use conditional Chebyshev to show the scaled validation average is negligible, then explain how a fixed number of folds are combined without assuming their fitted contributions are jointly independent. Identify the additional empirical-process condition needed if fitting and evaluation use the same sample.

Ready to continue when: you can show exactly which random term cross-fitting controls and what it does not control.

Transfer: Would the same proof work if the number of folds grew without bound? Which finite-sum argument would need replacement?

Hint

For any \delta>0, separate the event PH_k^2>\delta before applying the conditional variance bound.

Solution outline — open after trying

Conditional on the training data, the validation observations are independent fresh draws. Thus E[T_k\mid\mathrm{training}]=0 and \operatorname{Var}(T_k\mid\mathrm{training})=\operatorname{Var}_P(H_k)\leq PH_k^2. For fixed \epsilon>0, the unconditional tail probability is bounded by

P(PH_k^2>\delta)+\frac{\delta}{\epsilon^2}. Take the large-sample limit and then \delta\downarrow0. Multiplying each fold’s o_p(1) term by its bounded root-size weight and summing a fixed number of folds remains o_p(1); no joint fold independence is required. Without splitting, one must justify that the data-dependent empirical term is o_p(1), for example through suitable stochastic-equicontinuity/Donsker conditions and L2 consistency. Cross-fitting leaves the population bias term to be bounded separately.

Checkpoint · SP-13 — Complete the missing-outcome mean capstone

Task type: Derivation, implementation, and inference.

Starting from W=(Z,R,RY) and the full-population mean target, state identification assumptions, derive the EIF through the score components, construct cross-fitted contributions, and derive their exact bias. Give conditions controlling both remainders and a feasible standard error. Run the example without using unobserved outcomes, and explain what a comparison with the observed-data efficiency bound can establish.

Ready to continue when: you can reproduce the chain from target and assumptions through estimator, proof, uncertainty, and interpretation.

Transfer: Replace the outcome regression by a misspecified line. Which consistency and efficient-inference claims remain justified, and which require rechecking?

Hint

Organize the argument as identification, derivative, estimator, error decomposition, and variance.

Solution outline — open after trying

MAR and positivity identify \mu=E[Y] as E[m_0(Z)]. Score decomposition gives the centered EIF m_0-\mu+R(Y-m_0)/\pi_0. Average its estimated uncentered version out of fold. Its training-conditional bias is E[(\widehat\pi-\pi_0)(\widehat m-m_0)/\widehat\pi]. A lower probability bound, L2 consistency, a product o_p(n^{-1/2}), finite nonzero EIF variance, and the stated sampling/moment controls yield an asymptotic linear expansion: SP-12 controls the empirical term and the product controls bias. The SD of estimated influence values divided by \sqrt n gives the SE under those conditions. Double-robust consistency alone does not establish efficiency. An oracle simulation bound benchmarks repeated-sampling variance in this model; one interval or a smaller observed SE does not prove attainment.

Checkpoint · SP-14 — Distinguish identification, variance, and remainder failures

Task type: Diagnosis and derivation.

Diagnose three settings for the missing-outcome mean: (a) observation probability zero in a stratum of positive target mass; (b) \pi_0(Z)=Z for uniform Z on (0,1), with conditional outcome variance 1; and (c) constant positive nuisance errors n^{-1/5} in both models when \pi_0=1/2. State what fails and whether identification can remain.

Ready to continue when: you can name the failed condition or term instead of treating every problem as insufficient sample size.

Transfer: For each setting, propose what additional data, assumptions, changed target, or different asymptotic analysis would be needed.

Hint

Evaluate the unobserved stratum, the inverse-probability variance integral, and the exact bias times \sqrt n.

Solution outline — open after trying

The missing stratum mean is not identified without extra restrictions. (b) Positivity holds almost surely, so MAR identification can hold, but the EIF variance includes the divergent integral of 1/Z; the finite-variance root-n argument fails. (c) The exact bias is n^{-2/5}/(1/2+n^{-1/5}); its \sqrt n scaling grows, so centering at the scientific target is wrong at first order.

Additional practice (optional)

Repeat the synthetic analysis over at least 200 independent datasets; report bias, empirical SD, mean SE, coverage, failures, and MCSE.
Change only the outcome model and then only the observation model; separate consistency experiments from efficient-inference claims.
Derive the variance formula using conditional variance and compare it with the hypothetical complete-data variance.
For the optional median example with F(y)-1/2\approx cy^3, balance the population CDF drift against noise of order n^{-1/2}. Explain the resulting rate and compare the failed condition with the regular case in Quantiles and local expansions.

11 Reading

Kennedy, review of efficient functional estimation: influence-function constructions and remainder analysis.
Chernozhukov et al., Double/debiased machine learning: orthogonal equations, cross-fitting, and inferential conditions.

12 Continue through the guide

Apply the construction in An end-to-end expression analysis, which combines group standardization with missing assays and multiple testing. Continue to Evaluating a statistical method and the Completion checkpoints page.

--- title: "From an influence function to an estimator" subtitle: "A missing-outcome mean capstone: identification, construction, and proof." guide-status: "Available" --- **Before reading:** [Missingness, selection, and the observed-data target](../questions-and-design/missingness-and-selection.qmd), [Influence functions](../statistical-models/influence-functions.qmd), [Orthogonality and cross-fitting](orthogonality-and-cross-fitting.qmd), [Estimating nuisance functions for inference](nuisance-estimation.qmd), and [Large-sample reasoning for methods research](../estimation-and-inference/large-sample-reasoning.qmd). ## One target, a complete estimation argument The target is the eligible-population mean $\mu=E[Y]$ when some outcomes are missing. We observe i.i.d. records $W=(Z,R,RY)$ with covariates recorded for everyone. Assume $E[|Y|]<\infty$, missing at random (MAR), $R\perp Y\mid Z$, and $\pi_0(Z)>0$ almost surely in the target population. The [Missingness, selection, and the observed-data target](../questions-and-design/missingness-and-selection.qmd) page explains why these conditions identify the mean as $E[m_0(Z)]$, where $m_0(z)=E[Y\mid Z=z,R=1]$ and $\pi_0(z)=P(R=1\mid Z=z)$. Write $Pf=E[f(W)]$ under the true observed-data law and $P_nf=n^{-1}\sum_i f(W_i)$. For fitted functions, $P$ means expectation over a fresh record while holding the training fit fixed. The L2 norm is $\|f\|_2=\sqrt{P f^2}$. When $f$ depends only on $Z$, this averages over all eligible people's covariates. Positivity and MAR identify the target; stronger bounds and learning conditions will be imposed separately for the inference proof. This is an extended capstone. First follow the derivation, then inspect the implementation, and finally reconstruct the proof through the checkpoints. The same pattern transfers to residual covariance and partially linear coefficients, but their gradients and remainders must be derived for their own targets. ## Derive the observed-data influence function An observed-data density factors into the distribution of $Z$, the observation mechanism given $Z$, and the outcome distribution among observed units. Writing $w=(z,r,ry)$ for an observed record value and $p_1(y\mid z)$ for the last density, its form is $$ p(w)=p_Z(z)\pi(z)^r\{1-\pi(z)\}^{1-r} \times p_1(y\mid z)^r. $$ The outcome factor appears only when $r=1$; the notation does not evaluate an unrecorded $y$. A smooth path has total score $$ s(W)=s_Z(Z)+s_{R\mid Z}(R,Z) +Rs_Y(Y,Z), $$ where $s_Y$ abbreviates the observed conditional-outcome score. Density normalization gives $E[s_Z]=0$, $E[s_{R\mid Z}\mid Z]=0$, and $E[s_Y\mid Z,R=1]=0$. These are the three sets of permitted local directions whose target derivatives must be represented. Differentiate the identified functional $\mu(P)=E_P[m_P(Z)]$. Changing the covariate distribution contributes $E[(m_0-\mu)s_Z]$, because its score is centered. At a fixed $z$, differentiating the outcome mean contributes $E[(Y-m_0(z))s_Y\mid Z=z,R=1]$. Averaging over target covariates gives $$ E_Z\!\left[E[(Y-m_0(Z))s_Y\mid Z,R=1]\right]. $$ Changing the observation probability alone does not change the identified functional: its expression uses $p_Z$ and $p_1$, not $\pi$. This does not make missingness irrelevant to precision; fewer observed outcomes can still mean less information. A candidate gradient representing these derivatives is $$ \phi_0(W)=m_0(Z)-\mu +\frac{R}{\pi_0(Z)}\{Y-m_0(Z)\}. $$ ### Verify each score component Conditioning first on $(Z,R)$ gives $E[\phi_0\mid Z,R]=m_0(Z)-\mu$: when $R=1$, the observed residual has mean zero; when $R=0$, its weighted term is absent. Averaging gives $E[\phi_0]=0$. For covariate scores, the same centering gives $E[\phi_0s_Z]=E[(m_0-\mu)s_Z]$. For observation-mechanism scores, $$ \begin{aligned} E[\phi_0s_{R\mid Z}] &=E[(m_0-\mu)E[s_{R\mid Z}\mid Z]]\\ &=0. \end{aligned} $$ For outcome scores, the $m_0-\mu$ term vanishes because the conditional score is centered. The remaining product has $R^2=R$, so $$ \begin{aligned} E[\phi_0 R s_Y]&=E\left[\frac{R}{\pi_0}(Y-m_0)s_Y\right]\\ &=E_Z\!\left[E[(Y-m_0)s_Y\mid Z,R=1]\right]. \end{aligned} $$ The inner conditioning contributes a factor $\pi_0(Z)$ that cancels the inverse weight. Thus the same $\phi_0$ represents the derivative for each score component, and hence their sums and limits under the required differentiability and integrability conditions. ### Interpret the efficiency bound In the unrestricted observed-data model the tangent space is $L_2^0(P)$. When the candidate has finite second moment, its representation of all these derivatives makes it the unique gradient, the efficient influence function (EIF). To compute its variance, note that $E[\phi_0\mid Z]=m_0(Z)-\mu$ and $$ E\left[\left.\left\{\frac{R}{\pi_0}(Y-m_0)\right\}^2\right|Z\right]=\frac{\operatorname{Var}(Y\mid Z,R=1)}{\pi_0(Z)}. $$ One factor of $\pi_0$ cancels the squared inverse weight, leaving one in the denominator. Conditional centering makes the cross term zero. The total-variance calculation gives $$ V_0=\operatorname{Var}(m_0(Z)) +E\left[\frac{\operatorname{Var}(Y\mid Z,R=1)}{\pi_0(Z)}\right]. $$ $V_0$ is the variance of one influence contribution, or the limiting variance of the root-$n$ scaled estimator error. The estimator-variance bound is $V_0/n$, and its SE bound is $\sqrt{V_0/n}$. Dividing by $n$ accounts for averaging; taking the square root expresses uncertainty on the outcome scale. The first component reflects between-covariate differences in the mean. The second reflects within-covariate outcome noise, inflated by incomplete observation. For example, if $\pi_0=1/2$, $\operatorname{Var}(m_0)=1$, and every conditional outcome variance is 4, then $V_0=1+4/(1/2)=9$. If every outcome were observed, the variance bound would instead be $(1+4)/n=5/n$. Missingness doubles the conditional-noise contribution here, rather than doubling every part of the variance. These compare two different observation experiments at the same full-data law. ## Construct the augmented estimator For fitted nuisances $\hat\eta=(\hat m,\hat\pi)$, define $$ D_{\hat\eta}(W)=\hat m(Z) +\frac R{\hat\pi(Z)}\{Y-\hat m(Z)\}. $$ A plug-in starts with an average prediction $P_n\hat m$. Adding the empirical residual correction gives the augmented inverse-probability estimator. With fixed fits, the equation $P_n(D_{\hat\eta}-\mu)=0$ has the explicit solution $\widehat\mu=P_nD_{\hat\eta}$. This is why the EIF suggests the augmentation. Solving a sample equation does not itself show that the resulting estimator is centered at the population target. For cross-fitting, split independent units into a fixed number $K$ of folds independently of their data. In fold $k$, learn both functions on the other folds and evaluate $D_{\hat\eta_k}$ on held-out units. The label $k$ names the excluded validation fold. Average all held-out contributions with fold-size weights. Every row is evaluated once using a fit that did not train on that row. Repeated observations of one person need grouped folds and an appropriate cluster-level variance; the implementation below assumes independent people with one row each. Only observed-outcome rows receive a residual correction. Missing-outcome rows still contribute their predicted means. In code, multiplying `0 * NA` gives `NA`; the implementation constructs the correction on observed rows rather than treating the unobserved outcome as zero. ## Compute the exact bias remainder Hold a training fit fixed and take expectation over a fresh observation. The definition of $m_0$ gives $E[R(Y-\widehat m)\mid Z]=\pi_0(m_0-\widehat m)$. Therefore $$ \begin{aligned} E[D_{\widehat\eta}\mid Z]-m_0&=\widehat m-m_0+\frac{\pi_0}{\widehat\pi}(m_0-\widehat m)\\ &=\frac{\widehat\pi-\pi_0}{\widehat\pi}(\widehat m-m_0). \end{aligned} $$ Averaging over $Z$ gives $$ \begin{aligned} P D_{\hat\eta}-\mu &=E\left[\frac{\hat\pi-\pi_0}{\hat\pi} (\hat m-m_0)\right]. \end{aligned} $$ If $\hat\pi\ge c>0$, taking absolute values and applying Cauchy–Schwarz bounds the bias magnitude by $$ c^{-1}\|\hat\pi-\pi_0\|_2\|\hat m-m_0\|_2. $$ The denominator bound controls amplification of probability-estimation errors. If either supplied function equals the truth, the exact bias is zero. If one learned function converges to truth and the other's error remains controlled, the product can vanish even when the latter has an incorrect limit. With a vanishing empirical error this yields double-robust consistency. It does not remove MAR or prove efficiency. For efficient inference using $\phi_0$, the proof below assumes both nuisances approach truth, a sufficiently small product, and the empirical and variance conditions stated there. ## Decompose the full estimation error Let $I_k$ be fold $k$, $n_k$ its size, $w_k=n_k/n$, and $P_{n,k}$ its empirical average. The estimator is $\widehat\mu=\sum_k w_kP_{n,k}D_{\hat\eta_k}$. Put $D_0=\mu+\phi_0$ and $H_k=D_{\hat\eta_k}-D_0$. Add and subtract $D_0$ inside each validation average, then add and subtract $PH_k$. Because the folds partition the sample, $\sum_k w_kP_{n,k}D_0=P_nD_0$. Exactly, $$ \hat\mu-\mu=(P_n-P)\phi_0 +\sum_k w_k(P_{n,k}-P)H_k +\sum_k w_kPH_k. $$ The first term has a fixed-function CLT. The last is the population bias remainder just calculated. The middle is the centered empirical error from evaluating a learned function. A fixed-function CLT cannot simply be applied to it as if the fits were prespecified, because those fits are random and use part of the same overall dataset. ### Bound one validation fold, then combine folds Define $T_k=\sqrt{n_k}(P_{n,k}-P)H_k$. Conditional on fold $k$'s training data, $H_k$ is fixed and its validation observations are independent. Thus $E[T_k\mid\text{training}_k]=0$ and $$ \operatorname{Var}(T_k\mid\text{training}_k)\le PH_k^2. $$ If $PH_k^2=o_p(1)$, the variance bound is small with high probability. This does not require its training-sample expectation to converge to zero. For fixed $\epsilon,\delta>0$, separate the event where the bound exceeds $\delta$, and apply conditional Chebyshev on its complement: $$ \Pr(|T_k|>\epsilon) \le\Pr(PH_k^2>\delta)+\frac{\delta}{\epsilon^2}. $$ First let $n$ grow; the first probability tends to zero. Then let $\delta$ decrease to zero. This proves $T_k=o_p(1)$. For the overall empirical remainder, $$ \begin{aligned} \sqrt n\sum_k w_k(P_{n,k}-P)H_k&=\sum_k\sqrt{w_k}\,T_k\\ &=o_p(1). \end{aligned} $$ There are only finitely many folds, and every weight is bounded. Their training sets overlap, so joint conditional independence of all contributions is not assumed. A fixed finite sum of negligible terms is enough. A growing number of folds, including a leave-one-out scheme, would need a further argument rather than automatic reuse of this proof. ## A tractable sufficient set of conditions One sufficient teaching setup uses the following conditions: - The units are i.i.d.; $K$ is fixed; the split is independent of the data; and every fold's size fraction stays bounded away from zero. - MAR holds, and true and fitted observation probabilities are at least a fixed $c>0$. - Outcomes and fitted means are bounded in absolute value by a fixed $B<\infty$. - Both nuisances are L2-consistent, and their L2 error product is $o_p(n^{-1/2})$ in every fold. - The influence variance satisfies $0<V_0<\infty$. To see why nuisance consistency controls $H_k$, suppress the fold label and expand exactly: $$ H=(\widehat m-m_0)\left(1-\frac R{\widehat\pi}\right) +R(Y-m_0)\left(\frac1{\widehat\pi}-\frac1{\pi_0}\right). $$ The first multiplier is bounded by $1+c^{-1}$. For the second, $|Y-m_0|\le2B$ and the inverse-probability difference is at most $c^{-2}|\widehat\pi-\pi_0|$. The triangle inequality therefore gives $$ \|H\|_2\le(1+c^{-1})\|\widehat m-m_0\|_2 +2Bc^{-2}\|\widehat\pi-\pi_0\|_2. $$ Both errors tending to zero makes $H$ L2-small. The conditional-variance argument now controls the empirical remainder, while the product rate controls the population remainder. The fixed-function CLT and Slutsky's theorem give $$ \begin{aligned} \widehat\mu-\mu&=P_n\phi_0+o_p(n^{-1/2}),\\ \sqrt n(\widehat\mu-\mu)&\xrightarrow{d}N(0,V_0). \end{aligned} $$ ### Justify the estimated variance For $i\in I_k$, write $\widehat D_i=D_{\widehat\eta_k}(W_i)$ and $\widehat\phi_i=\widehat D_i-\widehat\mu$. Their sample mean is exactly zero. The sample variance and SE are $$ \begin{aligned} \widehat V&=\frac1{n-1}\sum_i\widehat\phi_i^2,\\ \widehat{\operatorname{SE}}&=\sqrt{\widehat V/n}. \end{aligned} $$ The learned contributions share training data, so the justification is not that they are all independent. For each fold, conditioning on training gives $E[P_{n,k}H_k^2\mid\text{training}_k]=PH_k^2$. Conditional Markov and the same small-probability-event argument above show that the average squared difference between $\widehat D_i$ and $D_0(W_i)$ tends to zero. Cauchy–Schwarz then controls the difference of their empirical second moments. The LLN for the fixed true contributions and consistency of $\widehat\mu$ give $\widehat V\xrightarrow{p}V_0$. Consequently $(\widehat\mu-\mu)/\widehat{\operatorname{SE}}$ converges in distribution to a standard normal, and $\widehat\mu\pm1.96\widehat{\operatorname{SE}}$ is an asymptotic 95% interval. Coverage concerns repeated datasets under the stated conditions. Bounded outcomes simplify this proof; unbounded outcomes need appropriate moment and uniform-integrability controls. ## Implement the complete teaching example Here $Z$ is uniform on $[-1,1]$, $m_0(Z)=2+Z+Z^2$, $\pi_0(Z)=\operatorname{expit}(-0.3+Z)$, and the outcome error is independently uniform on $[-1,1]$. The simulator draws $R$ using only $Z$, so MAR holds by construction. Since $E[Z]=0$ and $E[Z^2]=1/3$, the population mean is $\mu=2+0+1/3=7/3$. Conditional outcome variance is $1/3$. The regression and observation models below are correctly specified: the outcome fit includes both $Z$ and $Z^2$, and the observation fit uses a linear logit. Under their regular finite-dimensional conditions, both can attain root-$n$ rates, giving a product of order $n^{-1}$, negligible on the root-$n$ scale. This is a transparent verification case; flexible learners would need their own accuracy conditions. The observed dataset does not retain missing outcome values. The simulator constructs them only to generate the observation process and then returns `NA` for missing rows. Prespecified probability bounds $(0.1,0.9)$ contain the true probability range, approximately $(0.21,0.67)$; the outcome/prediction bounds $(0,6)$ also contain the truth. These teaching constraints are not learned from unavailable outcomes. Clipping to bounds containing the truth does not increase the distance from that truth. All reported inference concerns the cross-fitted augmented estimator. Within each training set, `lm` uses only observed outcomes and `glm` uses everyone's observed $R$. Predictions are made for the excluded fold, then assembled in original row order. `score` stores $\widehat D_i$; `phi` stores $\widehat\phi_i$. These names denote contributions, not likelihood scores. Centering and dividing their SD by $\sqrt n$ implements the variance formula just justified. The functions are also available as a [downloadable R script](../examples/missing-mean.R). ```{r} # Self-contained bounded-outcome teaching example; independent rows are people. SimulateMissingMean <- function(n = 800, seed = 51) { set.seed(seed) z <- runif(n, -1, 1) prob <- plogis(-0.3 + z) full_y <- 2 + z + z^2 + runif(n, -1, 1) r <- rbinom(n, 1, prob) list(data = data.frame(id = seq_len(n), z = z, r = r, y = ifelse(r == 1, full_y, NA_real_)), truth = 7 / 3) } CrossFitMissingMean <- function(d, K = 5, seed = 52, probability_bounds = c(0.1, 0.9), outcome_bounds = c(0, 6)) { n <- nrow(d) stopifnot(n >= 20, K >= 2, K <= n, !anyDuplicated(d$id), all(is.finite(d$z)), all(d$r %in% c(0, 1)), all(is.finite(d$y[d$r == 1])), all(is.na(d$y[d$r == 0])), length(probability_bounds) == 2, 0 < probability_bounds[1], probability_bounds[1] < probability_bounds[2], probability_bounds[2] < 1, length(outcome_bounds) == 2, outcome_bounds[1] < outcome_bounds[2], all(d$y[d$r == 1] >= outcome_bounds[1]), all(d$y[d$r == 1] <= outcome_bounds[2])) set.seed(seed) fold <- sample(rep(seq_len(K), length.out = n)) mhat <- phat <- raw_p <- raw_m <- numeric(n) for (k in seq_len(K)) { train <- d[fold != k, ]; test <- d[fold == k, ] observed <- train[train$r == 1, ] if (nrow(observed) < 6 || length(unique(train$r)) < 2) { stop("Training fold has inadequate observed outcomes or observation classes") } outcome_fit <- lm(y ~ z + I(z^2), data = observed) probability_fit <- glm(r ~ z, family = binomial(), data = train) if (!isTRUE(probability_fit$converged)) stop("Observation fit did not converge") raw_m[fold == k] <- predict(outcome_fit, newdata = test) raw_p[fold == k] <- predict(probability_fit, newdata = test, type = "response") } stopifnot(all(is.finite(raw_m)), all(is.finite(raw_p))) mhat <- pmin(outcome_bounds[2], pmax(outcome_bounds[1], raw_m)) phat <- pmin(probability_bounds[2], pmax(probability_bounds[1], raw_p)) score <- mhat observed <- d$r == 1 # Do not multiply zero by NA: construct residual corrections on observed rows. score[observed] <- mhat[observed] + (d$y[observed] - mhat[observed]) / phat[observed] estimate <- mean(score); phi <- score - estimate se <- sd(phi) / sqrt(n) list(summary = data.frame(estimate = estimate, se = se, lower = estimate - 1.96 * se, upper = estimate + 1.96 * se, observed = sum(observed), n = n, probability_clipped = mean(phat != raw_p), mean_clipped = mean(mhat != raw_m)), contributions = data.frame(id = d$id, fold = fold, r = d$r, z = d$z, mhat = mhat, phat = phat, score = score, phi = phi)) } ``` Before running the next chunk, predict why the complete-case mean may differ from $7/3$ even though observation depends only on recorded $Z$. Under this mechanism, the complete-case limit is $E[\pi_0(Z)m_0(Z)]/E[\pi_0(Z)]$: it weights covariates by their observation probabilities. MAR does not make that covariate mix match the full population. The cross-fitted estimate should be close to the truth on the scale of its SE, not identical to it. ```{r} example <- SimulateMissingMean() fit <- CrossFitMissingMean(example$data) fit$summary c(complete_case = mean(example$data$y[example$data$r == 1]), truth = example$truth) # Exact observed-data bound in this known simulation, not a real-data oracle. V0 <- integrate(function(z) ((1/3) / plogis(-0.3 + z) + (2 + z + z^2 - 7/3)^2) / 2, -1, 1)$value c(theoretical_SE = sqrt(V0 / nrow(example$data)), estimated_SE = fit$summary$se) ``` ```{r} #| label: fig-missing-influence #| fig-cap: "Cross-fitted contributions in one synthetic sample. Large values can arise from weighted observed residuals; missing-outcome units contribute predicted means." #| fig-alt: "Influence values plotted against Z, with observed and unobserved outcomes shown in different colors." a <- fit$contributions plot(a$z, a$phi, pch = 16, cex = 0.55, col = ifelse(a$r == 1, "#2F80C8", "#D55E00"), xlab = "Observed covariate Z", ylab = "Estimated influence contribution") abline(h = 0, lty = 2) legend("topright", c("Outcome observed", "Outcome missing"), col = c("#2F80C8", "#D55E00"), pch = 16, bty = "n") ``` The integral uses the known functions and conditional variance $1/3$ to compute $V_0$, then reports $\sqrt{V_0/n}$. Compare this theoretical SE with the estimated SE; neither is the SD of individual outcomes. One estimate can be above or below the truth and one estimated SE can be above or below the asymptotic benchmark. Such differences in a single sample do not establish bias, inefficiency, or improved efficiency. In the figure, missing-outcome units have $\widehat\phi_i=\widehat m(Z_i)-\widehat\mu$. Observed units also receive the weighted residual, so their contributions can be more spread out, especially where estimated observation probability is small. The plot displays the terms used in the estimator and SE; it is not a check of MAR. The code records clipping fractions and stops on inadequate training folds or nonfinite fits. A production evaluation should retain those failures rather than silently omit them. One interval containing the truth does not establish coverage; [Evaluating a statistical method](../method-development/evaluating-methods.qmd) explains repeated-sampling assessment. ## An introduction to targeting (optional) {#an-introduction-to-targeting} This optional comparison introduces a different construction. It updates an initial regression to solve an empirical EIF equation, then reports the target evaluated at the updated regression. For a Gaussian working squared-error loss, take a one-dimensional fluctuation $m_\epsilon=\widehat m+\epsilon a$, where $a(Z)=1/\widehat\pi(Z)$ is the specified update direction. Minimizing observed-outcome squared loss gives the equation $P_n[Ra(Y-m_\epsilon)]=0$. Its solution, when the denominator is positive, is $$ \widehat\epsilon =\frac{P_n[Ra(Y-\widehat m)]}{P_n[Ra^2]}. $$ Now report the plug-in mean $\widetilde\mu=P_nm_{\widehat\epsilon}$. The mean of its prediction-minus-target term is zero by definition, and the weighted residual term is zero by the update equation. Hence the updated empirical EIF mean is zero. This is the algebra behind targeting the update toward the final parameter. This illustrates a targeting step. A complete targeted-learning procedure must choose a suitable loss/submodel, preserve relevant bounds, handle fitted nuisances and folds, and establish its own remainder conditions. The proof and interval implementation above are for the cross-fitted augmented estimator; the illustration does not transfer that proof automatically to every targeting algorithm. ## Diagnose the failed condition If a positive-probability stratum has $\pi_0=0$, no outcomes are observed there. Two full-data populations can have the same observed law and different means in that stratum, giving different $\mu$. The full mean is therefore not identified without extra information. More data from the same observation mechanism do not resolve that ambiguity. If $\pi_0(Z)>0$ almost surely but approaches zero, identification can remain while EIF variance diverges. For $Z$ uniform on $(0,1)$, $\pi_0(Z)=Z$, and conditional outcome variance 1, the variance contains $$ \begin{aligned} E[1/Z]&=\int_0^1\frac{1}{z}\,dz\\ &=\infty. \end{aligned} $$ Every stratum has some observation support, but the rare observations near zero carry too much inverse-probability variation for the finite-variance root-$n$ argument. This is a precision/regularity failure, distinct from the previous identification failure. Slow nuisance learning can leave first-order bias even with perfectly separated folds. Let $\pi_0=1/2$ and take constant errors $\widehat\pi-\pi_0=n^{-1/5}$ and $\widehat m-m_0=n^{-1/5}$ for sufficiently large $n$ that the probabilities are admissible. The exact bias and its scaling are $$ \begin{aligned} b_n&=\frac{n^{-2/5}}{1/2+n^{-1/5}}\\ &\longrightarrow0,\\ \sqrt n\,b_n&=\frac{n^{1/10}}{1/2+n^{-1/5}}\\ &\longrightarrow\infty. \end{aligned} $$ Both learners converge and the estimator can be consistent, yet its bias overwhelms root-$n$ uncertainty. This constructs an actual noncanceling remainder; merely failing a sufficient norm bound would not establish such failure. Cross-fitting leaves this population calculation unchanged. **Optional transfer beyond the missing-mean target.** A unique median with $F(y)-1/2\approx cy^3$, $c>0$, near zero remains a defined and identified target, but its population derivative cannot be inverted by a positive density. Balancing CDF noise of order $n^{-1/2}$ with $c\widehat q^3$ suggests $\widehat q$ has order $n^{-1/6}$. [Quantiles and local expansions](../statistical-models/quantiles-and-local-expansions.qmd) explains the regular positive-density comparison. This failure belongs to the target's local behavior, not to a missingness model or a nuisance learner. Identification, finite influence variance, nuisance-bias control, and local differentiability are separate parts of an inference argument. A denominator cutoff cannot supply missing information about an unobserved stratum, and an SE formula cannot remove persistent bias. ## Exercises Checkpoints are completion targets. Additional practice is optional. Try each checkpoint before opening its hint or solution. ::: {#checkpoint-sp-12 .checkpoint} ### Checkpoint · SP-12 — Control the cross-fitted empirical-process term {.unnumbered} **Task type:** Proof. Assume i.i.d. observations and a fixed number of folds with nonvanishing validation fractions. Let $H_k$ be the fitted-minus-true contribution, fixed conditional on fold $k$'s training data, with $PH_k^2\to_p0$. Define $T_k=\sqrt{n_k}(P_{n,k}-P)H_k$, where $n_k$ is its validation size. Derive the conditional-variance bound for $T_k$. Use conditional Chebyshev to show the scaled validation average is negligible, then explain how a fixed number of folds are combined without assuming their fitted contributions are jointly independent. Identify the additional empirical-process condition needed if fitting and evaluation use the same sample. **Ready to continue when:** you can show exactly which random term cross-fitting controls and what it does not control. **Transfer:** Would the same proof work if the number of folds grew without bound? Which finite-sum argument would need replacement? <details class="checkpoint-help"> <summary>Hint</summary> For any $\delta>0$, separate the event $PH_k^2>\delta$ before applying the conditional variance bound. </details> <details class="checkpoint-help"> <summary>Solution outline — open after trying</summary> Conditional on the training data, the validation observations are independent fresh draws. Thus $E[T_k\mid\mathrm{training}]=0$ and $\operatorname{Var}(T_k\mid\mathrm{training})=\operatorname{Var}_P(H_k)\leq PH_k^2$. For fixed $\epsilon>0$, the unconditional tail probability is bounded by $$ P(PH_k^2>\delta)+\frac{\delta}{\epsilon^2}. $$ Take the large-sample limit and then $\delta\downarrow0$. Multiplying each fold's $o_p(1)$ term by its bounded root-size weight and summing a fixed number of folds remains $o_p(1)$; no joint fold independence is required. Without splitting, one must justify that the data-dependent empirical term is $o_p(1)$, for example through suitable stochastic-equicontinuity/Donsker conditions and L2 consistency. Cross-fitting leaves the population bias term to be bounded separately. </details> ::: ::: {#checkpoint-sp-13 .checkpoint} ### Checkpoint · SP-13 — Complete the missing-outcome mean capstone {.unnumbered} **Task type:** Derivation, implementation, and inference. Starting from $W=(Z,R,RY)$ and the full-population mean target, state identification assumptions, derive the EIF through the score components, construct cross-fitted contributions, and derive their exact bias. Give conditions controlling both remainders and a feasible standard error. Run the example without using unobserved outcomes, and explain what a comparison with the observed-data efficiency bound can establish. **Ready to continue when:** you can reproduce the chain from target and assumptions through estimator, proof, uncertainty, and interpretation. **Transfer:** Replace the outcome regression by a misspecified line. Which consistency and efficient-inference claims remain justified, and which require rechecking? <details class="checkpoint-help"> <summary>Hint</summary> Organize the argument as identification, derivative, estimator, error decomposition, and variance. </details> <details class="checkpoint-help"> <summary>Solution outline — open after trying</summary> MAR and positivity identify $\mu=E[Y]$ as $E[m_0(Z)]$. Score decomposition gives the centered EIF $m_0-\mu+R(Y-m_0)/\pi_0$. Average its estimated uncentered version out of fold. Its training-conditional bias is $E[(\widehat\pi-\pi_0)(\widehat m-m_0)/\widehat\pi]$. A lower probability bound, L2 consistency, a product $o_p(n^{-1/2})$, finite nonzero EIF variance, and the stated sampling/moment controls yield an asymptotic linear expansion: SP-12 controls the empirical term and the product controls bias. The SD of estimated influence values divided by $\sqrt n$ gives the SE under those conditions. Double-robust consistency alone does not establish efficiency. An oracle simulation bound benchmarks repeated-sampling variance in this model; one interval or a smaller observed SE does not prove attainment. </details> ::: ::: {#checkpoint-sp-14 .checkpoint} ### Checkpoint · SP-14 — Distinguish identification, variance, and remainder failures {.unnumbered} **Task type:** Diagnosis and derivation. Diagnose three settings for the missing-outcome mean: (a) observation probability zero in a stratum of positive target mass; (b) $\pi_0(Z)=Z$ for uniform $Z$ on $(0,1)$, with conditional outcome variance 1; and (c) constant positive nuisance errors $n^{-1/5}$ in both models when $\pi_0=1/2$. State what fails and whether identification can remain. **Ready to continue when:** you can name the failed condition or term instead of treating every problem as insufficient sample size. **Transfer:** For each setting, propose what additional data, assumptions, changed target, or different asymptotic analysis would be needed. <details class="checkpoint-help"> <summary>Hint</summary> Evaluate the unobserved stratum, the inverse-probability variance integral, and the exact bias times $\sqrt n$. </details> <details class="checkpoint-help"> <summary>Solution outline — open after trying</summary> (a) The missing stratum mean is not identified without extra restrictions. (b) Positivity holds almost surely, so MAR identification can hold, but the EIF variance includes the divergent integral of $1/Z$; the finite-variance root-$n$ argument fails. (c) The exact bias is $n^{-2/5}/(1/2+n^{-1/5})$; its $\sqrt n$ scaling grows, so centering at the scientific target is wrong at first order. </details> ::: ### Additional practice (optional) {.unnumbered} 1. Repeat the synthetic analysis over at least 200 independent datasets; report bias, empirical SD, mean SE, coverage, failures, and MCSE. 2. Change only the outcome model and then only the observation model; separate consistency experiments from efficient-inference claims. 3. Derive the variance formula using conditional variance and compare it with the hypothetical complete-data variance. 4. For the optional median example with $F(y)-1/2\approx cy^3$, balance the population CDF drift against noise of order $n^{-1/2}$. Explain the resulting rate and compare the failed condition with the regular case in [Quantiles and local expansions](../statistical-models/quantiles-and-local-expansions.qmd). ## Reading - Kennedy, [review of efficient functional estimation](https://arxiv.org/html/2203.06469v2): influence-function constructions and remainder analysis. - Chernozhukov et al., [Double/debiased machine learning](https://arxiv.org/abs/1608.00060): orthogonal equations, cross-fitting, and inferential conditions. ## Continue through the guide {#continue-through-the-collection} Apply the construction in [An end-to-end expression analysis](../analysis-problems/adjusted-expression-case-study.qmd), which combines group standardization with missing assays and multiple testing. Continue to [Evaluating a statistical method](../method-development/evaluating-methods.qmd) and the [Completion checkpoints](../estimation-and-inference/checkpoints.qmd) page.