Dependent observations and cluster-aware inference

Targets, influence contributions, and the independent unit

Before reading: Units of analysis, replication, and study design, means, and variances. Start with the person-summary examples; Influence functions and Estimating equations and asymptotic linearity support the contribution and sandwich derivations when you return to those sections.

1 Count the independent draws before counting rows

Repeated specimens, cells nested within people, technical replicates, and samples grouped by collection site can be correlated. A sandwich formula using one row at a time only addresses dependence if those rows are the independent units required by its argument.

This page uses independently sampled clusters, typically people. Within a cluster, observations may be dependent. Let G be the number of clusters and M_i the number of records in cluster i. Large-cluster-count asymptotics send G to infinity, with moment and non-dominance conditions; merely collecting many records from a fixed small number of people does not invoke that theorem. Dependence across people, such as family or site effects, may require a larger cluster unit or another framework.

Think of one statistical observation as the whole record C_i=(M_i,Y_{i1},\ldots,Y_{iM_i}). In the basic setup, these cluster records are i.i.d. The total row count N=\sum_iM_i may be much larger than G, but the LLN and CLT are applied to averages of functions of the G independent cluster records. Within-person correlation is preserved inside each function.

Read Units of analysis, replication, and study design first for target weighting. This page supplies the uncertainty calculation and its connection to influence functions. Cameron and Miller discuss practical cluster-inference limitations, including few clusters and multiple clustering dimensions.

2 Two means, two cluster contributions

Let T_i=\sum_{j=1}^{M_i}Y_{ij} and \bar Y_i=T_i/M_i, with M_i\geq1. A randomly sampled person’s mean record outcome has target

\begin{aligned} \mu_P&=E[\bar Y_i],\\ \widehat\mu_P&=G^{-1}\sum_i\bar Y_i. \end{aligned}

A record-weighted target is instead a ratio over the same cluster population:

\begin{aligned} \mu_R&=\frac{E[T_i]}{E[M_i]},\\ \widehat\mu_R&=\frac{\sum_i T_i}{\sum_i M_i}. \end{aligned}

Their cluster-level influence functions are

\begin{aligned} D_{P,i}&=\bar Y_i-\mu_P,\\ D_{R,i}&=\frac{T_i-\mu_R M_i}{E[M_i]}. \end{aligned}

The weighting difference can be seen at the row level. In \widehat\mu_P, each row from person i has weight 1/(GM_i): the person’s total weight is 1/G. In \widehat\mu_R, every row has weight 1/N, so people with more records contribute more weight. The two targets can therefore differ when record count and outcomes are associated.

For a concrete population, suppose half the people have one record with outcome 0 and half have five records, each with outcome 3. Then E[\bar Y_i]=(0+3)/2=1.5, while E[T_i]=7.5 and E[M_i]=3, giving \mu_R=2.5. Both targets concern the same population of people; they answer different averaging questions.

To derive the contributions, first apply the cluster LLN to \bar Y_i, T_i, and M_i, under the required integrability conditions. The person estimator is an ordinary mean of person summaries. For the record estimator, write \bar M=G^{-1}\sum_iM_i and use the exact identity

\widehat\mu_R-\mu_R =\frac{G^{-1}\sum_i(T_i-\mu_RM_i)}{\bar M}.

The numerator is an average of mean-zero cluster quantities. If E[M_i]<\infty, the LLN gives \bar M\xrightarrow{p}E[M_i]>0. With finite variance for T_i-\mu_RM_i, the CLT and Slutsky allow replacement of \bar M by E[M_i] at first order, giving D_{R,i} above. This is the ratio rule from Checkpoint SP-05, now applied to a person’s totals and counts.

Each contribution is one random cluster, even for the record-weighted target. Correcting an SE changes uncertainty about the selected target; it does not change the target weights.

Neither is intrinsically correct for every study. Irregular visits can be informative, and an observed-record mean is not automatically an outcome at a fixed scheduled time. Dropout and measurement timing may require identification assumptions before either target answers the intended longitudinal question.

3 Estimate variance across cluster contributions

For an estimator with expansion

\widehat\theta-\theta_0 =G^{-1}\sum_{i=1}^G D_i+o_p(G^{-1/2}),

a cluster CLT gives \sqrt G(\widehat\theta-\theta_0)\xrightarrow{d}N(0,V_C) when E[D_i]=0 and 0<V_C=\operatorname{Var}(D_i)<\infty. The estimator’s first-order variance is V_C/G.

Estimate V_C by the variance of the estimated cluster contributions:

\begin{aligned} s_D^2&=\frac1{G-1}\sum_{i=1}^G(\widehat D_i-\overline{\widehat D})^2,\\ \widehat{\operatorname{SE}}(\widehat\theta)&=\frac{s_D}{\sqrt G}. \end{aligned}

This is the same logic as the sample-mean SE, with the random object changed from a row to a cluster. Consistency of the estimated contribution variance and a negligible remainder justify studentization. When cluster sizes grow with the study, extra moment and non-dominance conditions are needed so one large cluster cannot supply most of the fluctuation. A normal interval remains an asymptotic statement about increasing independent cluster counts.

Code

ClusterMean <- function(y, id, target = c("person", "record")) {
  target <- match.arg(target)
  stopifnot(is.numeric(y), length(y) == length(id),
            all(is.finite(y)), !anyNA(id))
  totals <- rowsum(cbind(total = y, size = rep(1, length(y))), id, reorder = FALSE)
  G <- nrow(totals); stopifnot(G >= 2)
  T <- totals[, "total"]; M <- totals[, "size"]
  estimate <- if (target == "person") mean(T/M) else sum(T)/sum(M)
  D <- if (target == "person") T/M - estimate else (T - estimate*M)/mean(M)
  c(estimate = estimate, SE = sd(D)/sqrt(G), clusters = G, records = length(y))
}
y <- c(0, rep(3, 5)); id <- c("short", rep("long", 5))
rbind(person = ClusterMean(y, id, "person"), record = ClusterMean(y, id, "record"))

       estimate        SE clusters records
person      1.5 1.5000000        2       6
record      2.5 0.8333333        2       6

The two-cluster output demonstrates the formulas, not trustworthy normal inference from two people. For the person mean, the SE is simply the SE of the person summaries. For the record target, the varying denominator enters the influence function. Using the row SD divided by the square root of the row count ignores within-person covariance.

4 Why extra records do not supply extra independent people

Take a balanced random-intercept model Y_{ij}=\mu+B_i+\epsilon_{ij}, with independent zero-mean components, \operatorname{Var}(B_i)=\tau^2, and \operatorname{Var}(\epsilon_{ij})=\sigma^2. There are m records per person. Write the grand mean’s error as

\widehat\mu-\mu =\frac1G\sum_iB_i +\frac1{Gm}\sum_i\sum_j\epsilon_{ij}.

The shared person component B_i appears once in every record from that person, so averaging their records does not remove it. Its variance contribution is \tau^2/G. The independent record noise contributes \sigma^2/(Gm). Adding them gives

\begin{aligned} \operatorname{Var}(\widehat\mu) &=\frac{\tau^2+\sigma^2/m}{G},\\ \text{design effect} &=1+(m-1)\rho,\\ \rho&=\frac{\tau^2}{\tau^2+\sigma^2}. \end{aligned}

Two records from the same person have covariance \tau^2, so \rho is their intraclass correlation. Treating all Gm rows as independent would use variance (\tau^2+\sigma^2)/(Gm). Dividing the correct variance by that calculation gives the displayed design effect. Its square root is the corresponding SE inflation.

For G=100, m=4, \tau^2=0.8, and \sigma^2=0.2, the correct variance is 0.0085 and SE about 0.0922. The independent-row calculation gives variance 0.0025 and SE 0.05. The variance ratio is 3.4; the SE ratio is \sqrt{3.4}\approx1.84.

With G fixed, increasing m leaves a variance floor \tau^2/G when \tau^2>0. Additional records reduce measurement noise, but estimating the population mean consistently in this model requires more independent people. This design-effect formula belongs to the stated balanced model; other weighting schemes or cluster structures need their own variance calculation.

Code

m_grid <- 1:20; G <- 100
correct <- sqrt((0.8 + 0.2/m_grid)/G)
naive <- sqrt(1/(G*m_grid))
plot(m_grid, correct, type = "l", lwd = 2, col = "#2F80C8", ylim = c(0, 0.11),
     xlab = "Records per person", ylab = "SE of the grand mean")
lines(m_grid, naive, lwd = 2, col = "#D55E00", lty = 2)
abline(h = sqrt(0.8/G), lty = 3)
legend("right", c("Cluster model", "Independent-row calculation", "Person-variance floor"),
       col = c("#2F80C8", "#D55E00", "black"), lty = c(1, 2, 3), bty = "n")

The correct SE decreases toward a positive floor as records per person increase, while the independent-row calculation continues toward zero. — Figure 1: Exact SEs in a balanced random-intercept model with 100 people, person variance 0.8, and record-noise variance 0.2. Additional records reduce noise but do not remove the person component.

Now check coverage under this explicitly Gaussian, balanced generator. There are 500 independent simulated datasets, each with 100 people and four records. Both estimators have the same point estimate; the SE calculation differs. Monte Carlo SE describes uncertainty in the estimated coverage, not uncertainty in one study’s mean.

Code

set.seed(2701)
coverage <- replicate(500, {
  id <- rep(seq_len(100), each = 4)
  y <- rep(rnorm(100, sd = sqrt(0.8)), each = 4) + rnorm(400, sd = sqrt(0.2))
  fit <- ClusterMean(y, id)
  se <- c(row = sd(y)/sqrt(length(y)), cluster = unname(fit['SE']))
  abs(mean(y)) <= qnorm(0.975) * se
})
coverage_rate <- rowMeans(coverage)
data.frame(method = names(coverage_rate), coverage = coverage_rate,
           MCSE = sqrt(coverage_rate * (1-coverage_rate)/ncol(coverage)))

         method coverage       MCSE
row         row    0.750 0.01936492
cluster cluster    0.948 0.00992935

5 The cluster sandwich is the same linearization at a different unit

Suppose \widehat\theta solves G^{-1}\sum_i S_i(\theta)=0, where S_i is the whole-cluster score. If A=E[\partial S_i(\theta_0)/\partial\theta^T] is nonsingular and consistency, a CLT, differentiability, and remainder control hold, then

\begin{aligned} D_i&=-A^{-1}S_i(\theta_0),\\ \widehat{\operatorname{Var}}(\widehat\theta) &=G^{-1}\widehat A^{-1} \widehat{\operatorname{Cov}}(S_i) \widehat A^{-T}. \end{aligned}

Here A^{-T}=(A^{-1})^\top. Evaluate the cluster scores and their derivatives at \widehat\theta to estimate the covariance and A, with consistency justified by the corresponding LLNs. For OLS, S_i=\sum_j X_{ij}(Y_{ij}-X_{ij}^\top\beta). Writing each row contribution as \psi_{ij} shows what changes in the variance calculation:

S_iS_i^\top=\sum_j\sum_\ell\psi_{ij}\psi_{i\ell}^\top.

The terms with j\ne\ell retain covariance between records from the same person. A rowwise sandwich uses only the same-record terms and therefore can miss the shared variation. The derivative and variance factors must both use the same whole-cluster normalization.

The point estimate still targets the selected regression equation. If each row enters equally, people with more rows contribute more to that equation. A cluster variance correction preserves those weights and the target they define. Omitted confounding and a rank-deficient design require separate arguments.

6 Cross-fitting and resampling must preserve clusters

Place every record from a person in the same fold when people are independent. Train preprocessing, tuning, feature selection, and nuisance fits within training clusters. Otherwise the held-out record can still share a person’s outcome information with its training records. With fixed K folds, condition on training clusters and apply the same held-out empirical-term argument as in SP-12, now to whole-cluster contributions. Appropriate cluster-level L2/product-remainder and variance conditions remain necessary.

In the random-intercept example, a learner trained on one record can gain information about that person’s B_i. Another record from the same person is consequently not a fresh independent validation draw conditional on the training data. Holding out the whole person restores the intended separation under independent-person sampling. It does not by itself make the fitted nuisance functions accurate enough for inference.

For a cluster bootstrap, resample people with replacement and carry all their records together. Give repeated copies distinct bootstrap cluster IDs if refitting. Whether nuisance fits should be recomputed depends on the inferential procedure; a shortcut must be justified rather than assumed. A valid permutation scheme instead follows the assignment or exchangeability design, which may differ from a bootstrap. See the Multivariate comparisons: permutation and distance-based tests page.

With few clusters, highly unequal leverage, one dominant person/site, cross-cluster dependence, or few clusters per treatment group, the normal approximation may fail. Small-sample corrections and specialized bootstrap/randomization methods require their own assumptions; no single cluster-count threshold validates them all. Report the cluster definition, count and size distribution, target weights, fold/resampling scheme, and variance method.

7 Exercises

Try each checkpoint before opening its hint or solution.

Checkpoint · CL-01 — Derive the cluster contribution for the chosen target

Task type: Derivation and interpretation.

Let T be a person’s outcome sum and M≥1 their record count. Derive influence functions for E[T/M] and E[T]/E[M] under independent person sampling. For equally common types (M,Y)=(1,0) and (5,3), calculate both targets and explain why replacing only the SE does not change the point target.

Ready to continue when: you can derive both cluster-level contributions and connect target weighting to the independent-unit variance.

Transfer: If visit frequency depends on disease severity, which target would a scheduled-time scientific question require?

Hint

Use the mean derivative for the first target and the ratio rule for the second.

Solution outline — open after trying

The contributions are T/M−μ_P and (T−μ_R M)/E[M]. The targets are 1.5 and 2.5. Their variance is calculated across independent people, but they average different quantities. A covariance correction changes uncertainty, not the equation defining the point estimate.

Checkpoint · CL-02 — Construct a valid fold and variance argument

Task type: Derivation and diagnosis.

In the balanced random-intercept model, derive Var(grand mean) and the design effect. Use G=100, m=4, τ²=0.8, σ²=0.2. Then explain why random row folds and a row bootstrap are inappropriate for an estimator trained on repeated person outcomes.

Ready to continue when: you can account for the shared component and preserve the independent unit in training, uncertainty, and resampling.

Transfer: If people are nested in a small number of sites with shared errors, is person-level independence still defensible?

Hint

Average the person intercept once per person and the record noise over all records.

Solution outline — open after trying

The variance is (0.8+0.2/4)/100=0.0085 and SE≈0.0922. The independent-row variance would be 1/400=0.0025; the design effect is 3.4. Row folds can put one person in both training and validation; row resampling breaks their joint record distribution. Use person folds and person contributions/resampling under independent-person sampling, with the requisite asymptotic and nuisance conditions.

Additional practice (optional)

Vary the cluster size distribution in the two-type example and compare person and record targets.
Repeat the coverage simulation with larger G and with larger m; separate Monte Carlo variation from the predicted SE behavior.
Write whole-cluster scores for a prespecified covariate-adjusted regression and identify its weighting and rank conditions.

8 Reading

Cameron and Miller, A practitioner’s guide to cluster-robust inference: cluster variance and practical limitations.

9 Continue through the guide

Apply the independent-unit argument to Multivariate comparisons: permutation and distance-based tests and An end-to-end residual-network analysis.

Continue to Bootstrap and resampling for statistical inference for a worked whole-person bootstrap and the conditions behind resampling approximations.

--- title: "Dependent observations and cluster-aware inference" subtitle: "Targets, influence contributions, and the independent unit" guide-status: "Available" --- **Before reading:** [Units of analysis, replication, and study design](../questions-and-design/units-and-study-design.qmd), means, and variances. Start with the person-summary examples; [Influence functions](../statistical-models/influence-functions.qmd) and [Estimating equations and asymptotic linearity](estimating-equations.qmd) support the contribution and sandwich derivations when you return to those sections. ## Count the independent draws before counting rows Repeated specimens, cells nested within people, technical replicates, and samples grouped by collection site can be correlated. A sandwich formula using one row at a time only addresses dependence if those rows are the independent units required by its argument. This page uses **independently sampled clusters**, typically people. Within a cluster, observations may be dependent. Let $G$ be the number of clusters and $M_i$ the number of records in cluster $i$. Large-cluster-count asymptotics send $G$ to infinity, with moment and non-dominance conditions; merely collecting many records from a fixed small number of people does not invoke that theorem. Dependence across people, such as family or site effects, may require a larger cluster unit or another framework. Think of one statistical observation as the whole record $C_i=(M_i,Y_{i1},\ldots,Y_{iM_i})$. In the basic setup, these cluster records are i.i.d. The total row count $N=\sum_iM_i$ may be much larger than $G$, but the LLN and CLT are applied to averages of functions of the $G$ independent cluster records. Within-person correlation is preserved inside each function. Read [Units of analysis, replication, and study design](../questions-and-design/units-and-study-design.qmd) first for target weighting. This page supplies the uncertainty calculation and its connection to influence functions. [Cameron and Miller](https://cameron.econ.ucdavis.edu/research/Cameron_Miller_JHR_2015.pdf) discuss practical cluster-inference limitations, including few clusters and multiple clustering dimensions. ## Two means, two cluster contributions Let $T_i=\sum_{j=1}^{M_i}Y_{ij}$ and $\bar Y_i=T_i/M_i$, with $M_i\geq1$. A randomly sampled person's mean record outcome has target $$ \begin{aligned} \mu_P&=E[\bar Y_i],\\ \widehat\mu_P&=G^{-1}\sum_i\bar Y_i. \end{aligned} $$ A record-weighted target is instead a ratio over the same cluster population: $$ \begin{aligned} \mu_R&=\frac{E[T_i]}{E[M_i]},\\ \widehat\mu_R&=\frac{\sum_i T_i}{\sum_i M_i}. \end{aligned} $$ Their cluster-level influence functions are $$ \begin{aligned} D_{P,i}&=\bar Y_i-\mu_P,\\ D_{R,i}&=\frac{T_i-\mu_R M_i}{E[M_i]}. \end{aligned} $$ The weighting difference can be seen at the row level. In $\widehat\mu_P$, each row from person $i$ has weight $1/(GM_i)$: the person's total weight is $1/G$. In $\widehat\mu_R$, every row has weight $1/N$, so people with more records contribute more weight. The two targets can therefore differ when record count and outcomes are associated. <span id="fig-cluster-targets"></span> For a concrete population, suppose half the people have one record with outcome 0 and half have five records, each with outcome 3. Then $E[\bar Y_i]=(0+3)/2=1.5$, while $E[T_i]=7.5$ and $E[M_i]=3$, giving $\mu_R=2.5$. Both targets concern the same population of people; they answer different averaging questions. To derive the contributions, first apply the cluster LLN to $\bar Y_i$, $T_i$, and $M_i$, under the required integrability conditions. The person estimator is an ordinary mean of person summaries. For the record estimator, write $\bar M=G^{-1}\sum_iM_i$ and use the exact identity $$ \widehat\mu_R-\mu_R =\frac{G^{-1}\sum_i(T_i-\mu_RM_i)}{\bar M}. $$ The numerator is an average of mean-zero cluster quantities. If $E[M_i]<\infty$, the LLN gives $\bar M\xrightarrow{p}E[M_i]>0$. With finite variance for $T_i-\mu_RM_i$, the CLT and Slutsky allow replacement of $\bar M$ by $E[M_i]$ at first order, giving $D_{R,i}$ above. This is the ratio rule from [Checkpoint SP-05](../statistical-models/influence-functions.qmd#checkpoint-sp-05), now applied to a person's totals and counts. Each contribution is one random **cluster**, even for the record-weighted target. Correcting an SE changes uncertainty about the selected target; it does not change the target weights. Neither is intrinsically correct for every study. Irregular visits can be informative, and an observed-record mean is not automatically an outcome at a fixed scheduled time. Dropout and measurement timing may require identification assumptions before either target answers the intended longitudinal question. ## Estimate variance across cluster contributions For an estimator with expansion $$ \widehat\theta-\theta_0 =G^{-1}\sum_{i=1}^G D_i+o_p(G^{-1/2}), $$ a cluster CLT gives $\sqrt G(\widehat\theta-\theta_0)\xrightarrow{d}N(0,V_C)$ when $E[D_i]=0$ and $0<V_C=\operatorname{Var}(D_i)<\infty$. The estimator's first-order variance is $V_C/G$. Estimate $V_C$ by the variance of the estimated **cluster contributions**: $$ \begin{aligned} s_D^2&=\frac1{G-1}\sum_{i=1}^G(\widehat D_i-\overline{\widehat D})^2,\\ \widehat{\operatorname{SE}}(\widehat\theta)&=\frac{s_D}{\sqrt G}. \end{aligned} $$ This is the same logic as the sample-mean SE, with the random object changed from a row to a cluster. Consistency of the estimated contribution variance and a negligible remainder justify studentization. When cluster sizes grow with the study, extra moment and non-dominance conditions are needed so one large cluster cannot supply most of the fluctuation. A normal interval remains an asymptotic statement about increasing independent cluster counts. ```{r} ClusterMean <- function(y, id, target = c("person", "record")) { target <- match.arg(target) stopifnot(is.numeric(y), length(y) == length(id), all(is.finite(y)), !anyNA(id)) totals <- rowsum(cbind(total = y, size = rep(1, length(y))), id, reorder = FALSE) G <- nrow(totals); stopifnot(G >= 2) T <- totals[, "total"]; M <- totals[, "size"] estimate <- if (target == "person") mean(T/M) else sum(T)/sum(M) D <- if (target == "person") T/M - estimate else (T - estimate*M)/mean(M) c(estimate = estimate, SE = sd(D)/sqrt(G), clusters = G, records = length(y)) } y <- c(0, rep(3, 5)); id <- c("short", rep("long", 5)) rbind(person = ClusterMean(y, id, "person"), record = ClusterMean(y, id, "record")) ``` The two-cluster output demonstrates the formulas, not trustworthy normal inference from two people. For the person mean, the SE is simply the SE of the person summaries. For the record target, the varying denominator enters the influence function. Using the row SD divided by the square root of the row count ignores within-person covariance. ## Why extra records do not supply extra independent people Take a balanced random-intercept model $Y_{ij}=\mu+B_i+\epsilon_{ij}$, with independent zero-mean components, $\operatorname{Var}(B_i)=\tau^2$, and $\operatorname{Var}(\epsilon_{ij})=\sigma^2$. There are $m$ records per person. Write the grand mean's error as $$ \widehat\mu-\mu =\frac1G\sum_iB_i +\frac1{Gm}\sum_i\sum_j\epsilon_{ij}. $$ The shared person component $B_i$ appears once in every record from that person, so averaging their records does not remove it. Its variance contribution is $\tau^2/G$. The independent record noise contributes $\sigma^2/(Gm)$. Adding them gives $$ \begin{aligned} \operatorname{Var}(\widehat\mu) &=\frac{\tau^2+\sigma^2/m}{G},\\ \text{design effect} &=1+(m-1)\rho,\\ \rho&=\frac{\tau^2}{\tau^2+\sigma^2}. \end{aligned} $$ Two records from the same person have covariance $\tau^2$, so $\rho$ is their intraclass correlation. Treating all $Gm$ rows as independent would use variance $(\tau^2+\sigma^2)/(Gm)$. Dividing the correct variance by that calculation gives the displayed **design effect**. Its square root is the corresponding SE inflation. For $G=100$, $m=4$, $\tau^2=0.8$, and $\sigma^2=0.2$, the correct variance is 0.0085 and SE about 0.0922. The independent-row calculation gives variance 0.0025 and SE 0.05. The variance ratio is 3.4; the SE ratio is $\sqrt{3.4}\approx1.84$. With $G$ fixed, increasing $m$ leaves a variance floor $\tau^2/G$ when $\tau^2>0$. Additional records reduce measurement noise, but estimating the population mean consistently in this model requires more independent people. This design-effect formula belongs to the stated balanced model; other weighting schemes or cluster structures need their own variance calculation. ```{r} #| label: fig-cluster-information #| fig-height: 4.5 #| fig-cap: "Exact SEs in a balanced random-intercept model with 100 people, person variance 0.8, and record-noise variance 0.2. Additional records reduce noise but do not remove the person component." #| fig-alt: "The correct SE decreases toward a positive floor as records per person increase, while the independent-row calculation continues toward zero." m_grid <- 1:20; G <- 100 correct <- sqrt((0.8 + 0.2/m_grid)/G) naive <- sqrt(1/(G*m_grid)) plot(m_grid, correct, type = "l", lwd = 2, col = "#2F80C8", ylim = c(0, 0.11), xlab = "Records per person", ylab = "SE of the grand mean") lines(m_grid, naive, lwd = 2, col = "#D55E00", lty = 2) abline(h = sqrt(0.8/G), lty = 3) legend("right", c("Cluster model", "Independent-row calculation", "Person-variance floor"), col = c("#2F80C8", "#D55E00", "black"), lty = c(1, 2, 3), bty = "n") ``` Now check coverage under this explicitly Gaussian, balanced generator. There are 500 independent simulated datasets, each with 100 people and four records. Both estimators have the same point estimate; the SE calculation differs. Monte Carlo SE describes uncertainty in the estimated coverage, not uncertainty in one study's mean. ```{r} set.seed(2701) coverage <- replicate(500, { id <- rep(seq_len(100), each = 4) y <- rep(rnorm(100, sd = sqrt(0.8)), each = 4) + rnorm(400, sd = sqrt(0.2)) fit <- ClusterMean(y, id) se <- c(row = sd(y)/sqrt(length(y)), cluster = unname(fit['SE'])) abs(mean(y)) <= qnorm(0.975) * se }) coverage_rate <- rowMeans(coverage) data.frame(method = names(coverage_rate), coverage = coverage_rate, MCSE = sqrt(coverage_rate * (1-coverage_rate)/ncol(coverage))) ``` ## The cluster sandwich is the same linearization at a different unit Suppose $\widehat\theta$ solves $G^{-1}\sum_i S_i(\theta)=0$, where $S_i$ is the whole-cluster score. If $A=E[\partial S_i(\theta_0)/\partial\theta^T]$ is nonsingular and consistency, a CLT, differentiability, and remainder control hold, then $$ \begin{aligned} D_i&=-A^{-1}S_i(\theta_0),\\ \widehat{\operatorname{Var}}(\widehat\theta) &=G^{-1}\widehat A^{-1} \widehat{\operatorname{Cov}}(S_i) \widehat A^{-T}. \end{aligned} $$ Here $A^{-T}=(A^{-1})^\top$. Evaluate the cluster scores and their derivatives at $\widehat\theta$ to estimate the covariance and $A$, with consistency justified by the corresponding LLNs. For OLS, $S_i=\sum_j X_{ij}(Y_{ij}-X_{ij}^\top\beta)$. Writing each row contribution as $\psi_{ij}$ shows what changes in the variance calculation: $$ S_iS_i^\top=\sum_j\sum_\ell\psi_{ij}\psi_{i\ell}^\top. $$ The terms with $j\ne\ell$ retain covariance between records from the same person. A rowwise sandwich uses only the same-record terms and therefore can miss the shared variation. The derivative and variance factors must both use the same whole-cluster normalization. The point estimate still targets the selected regression equation. If each row enters equally, people with more rows contribute more to that equation. A cluster variance correction preserves those weights and the target they define. Omitted confounding and a rank-deficient design require separate arguments. ## Cross-fitting and resampling must preserve clusters Place every record from a person in the same fold when people are independent. Train preprocessing, tuning, feature selection, and nuisance fits within training clusters. Otherwise the held-out record can still share a person's outcome information with its training records. With fixed $K$ folds, condition on training clusters and apply the same held-out empirical-term argument as in [SP-12](../semiparametric-methods/influence-function-to-estimator.qmd#checkpoint-sp-12), now to whole-cluster contributions. Appropriate cluster-level L2/product-remainder and variance conditions remain necessary. In the random-intercept example, a learner trained on one record can gain information about that person's $B_i$. Another record from the same person is consequently not a fresh independent validation draw conditional on the training data. Holding out the whole person restores the intended separation under independent-person sampling. It does not by itself make the fitted nuisance functions accurate enough for inference. For a cluster bootstrap, resample people with replacement and carry all their records together. Give repeated copies distinct bootstrap cluster IDs if refitting. Whether nuisance fits should be recomputed depends on the inferential procedure; a shortcut must be justified rather than assumed. A valid permutation scheme instead follows the assignment or exchangeability design, which may differ from a bootstrap. See the [Multivariate comparisons: permutation and distance-based tests](../analysis-problems/multivariate-comparisons.qmd) page. With few clusters, highly unequal leverage, one dominant person/site, cross-cluster dependence, or few clusters per treatment group, the normal approximation may fail. Small-sample corrections and specialized bootstrap/randomization methods require their own assumptions; no single cluster-count threshold validates them all. Report the cluster definition, count and size distribution, target weights, fold/resampling scheme, and variance method. ## Exercises Try each checkpoint before opening its hint or solution. ::: {#checkpoint-cl-01 .checkpoint} ### Checkpoint · CL-01 — Derive the cluster contribution for the chosen target {.unnumbered} **Task type:** Derivation and interpretation. Let T be a person's outcome sum and M≥1 their record count. Derive influence functions for E[T/M] and E[T]/E[M] under independent person sampling. For equally common types (M,Y)=(1,0) and (5,3), calculate both targets and explain why replacing only the SE does not change the point target. **Ready to continue when:** you can derive both cluster-level contributions and connect target weighting to the independent-unit variance. **Transfer:** If visit frequency depends on disease severity, which target would a scheduled-time scientific question require? <details class="checkpoint-help"> <summary>Hint</summary> Use the mean derivative for the first target and the ratio rule for the second. </details> <details class="checkpoint-help"> <summary>Solution outline — open after trying</summary> The contributions are T/M−μ_P and (T−μ_R M)/E[M]. The targets are 1.5 and 2.5. Their variance is calculated across independent people, but they average different quantities. A covariance correction changes uncertainty, not the equation defining the point estimate. </details> ::: ::: {#checkpoint-cl-02 .checkpoint} ### Checkpoint · CL-02 — Construct a valid fold and variance argument {.unnumbered} **Task type:** Derivation and diagnosis. In the balanced random-intercept model, derive Var(grand mean) and the design effect. Use G=100, m=4, τ²=0.8, σ²=0.2. Then explain why random row folds and a row bootstrap are inappropriate for an estimator trained on repeated person outcomes. **Ready to continue when:** you can account for the shared component and preserve the independent unit in training, uncertainty, and resampling. **Transfer:** If people are nested in a small number of sites with shared errors, is person-level independence still defensible? <details class="checkpoint-help"> <summary>Hint</summary> Average the person intercept once per person and the record noise over all records. </details> <details class="checkpoint-help"> <summary>Solution outline — open after trying</summary> The variance is (0.8+0.2/4)/100=0.0085 and SE≈0.0922. The independent-row variance would be 1/400=0.0025; the design effect is 3.4. Row folds can put one person in both training and validation; row resampling breaks their joint record distribution. Use person folds and person contributions/resampling under independent-person sampling, with the requisite asymptotic and nuisance conditions. </details> ::: ### Additional practice (optional) {.unnumbered} 1. Vary the cluster size distribution in the two-type example and compare person and record targets. 2. Repeat the coverage simulation with larger G and with larger m; separate Monte Carlo variation from the predicted SE behavior. 3. Write whole-cluster scores for a prespecified covariate-adjusted regression and identify its weighting and rank conditions. ## Reading - Cameron and Miller, [A practitioner’s guide to cluster-robust inference](https://cameron.econ.ucdavis.edu/research/Cameron_Miller_JHR_2015.pdf): cluster variance and practical limitations. ## Continue through the guide {#continue-through-the-collection} Apply the independent-unit argument to [Multivariate comparisons: permutation and distance-based tests](../analysis-problems/multivariate-comparisons.qmd) and [An end-to-end residual-network analysis](../analysis-problems/residual-network-case-study.qmd). Continue to [Bootstrap and resampling for statistical inference](bootstrap-and-resampling.qmd) for a worked whole-person bootstrap and the conditions behind resampling approximations.