Microbiome data and compositional targets

Scale, references, zeros, and the population question

Before reading: The Understanding an omics dataset page, ratios and logarithms; Preprocessing and the quantity being estimated.

1 Start with what the library samples

Amplicon and shotgun sequencing can summarize taxonomic or functional features, but they do not measure the same objects. An amplicon assay targets a marker region; an amplicon sequence variant distinguishes a reported marker sequence, not necessarily a species. Shotgun sequencing samples DNA more broadly and can support taxonomic or functional summaries depending on the pipeline. Neither produces a direct table of live cells.

Record the sampled material, extraction and amplification protocol, reference database, feature definition, and specimen amount. A taxon’s DNA contribution depends on more than its cell count: extraction efficiency and marker copy number can differ. Contamination can also add DNA unrelated to the intended community. These belong in the measurement interpretation before a statistical model is chosen.

For this primer, use an idealized observation model. A specimen has positive absolute feature quantities x_1,\ldots,x_D. The sequencing probabilities are

\begin{gathered} p_j=\frac{x_j}{\sum_k x_k},\\ \mathbf C=(C_1,\ldots,C_D),\\ \mathbf C\mid N,p \sim\operatorname{Multinomial}(N,p). \end{gathered}

Here D is the number of features, N the total library count, and C_j the reads assigned to feature j. Conditioning on N treats the read budget as fixed. Then E[C_j\mid N,p]=Np_j and

\operatorname{Var}(C_j/N\mid N,p)=\frac{p_j(1-p_j)}{N}.

More reads reduce this conditional sampling variation. They do not supply the overall biological scale of x. The model intentionally omits feature-specific bias; a richer model can use probabilities proportional to b_jx_j. The compositional-data paper by Gloor et al. explains the role of ratios.

2 Two absolute worlds can give the same library

Multiplying every feature by a positive factor a leaves p unchanged, because ax_j/\sum_k ax_k=x_j/\sum_kx_k. Thus x and 10x give the same multinomial count law at the same N. This is nonidentification: different absolute communities produce the same observation distribution. More reads can reveal their common composition more precisely without distinguishing their absolute totals.

absolute <- cbind(World1 = c(10, 20, 30), World2 = c(100, 200, 300))
rownames(absolute) <- c("Taxon A", "Taxon B", "Taxon C")
p <- sweep(absolute, 2, colSums(absolute), "/")
cols <- c("#2F80C8", "#D55E00", "#009E73")
par(mfrow = c(2, 1), mar = c(3, 4, 2, 1))
barplot(absolute, col = cols, ylab = "Absolute quantity", ylim = c(0, 750))
legend("topleft", rownames(absolute), fill = cols, horiz = TRUE, bty = "n")
barplot(p, col = cols, ylab = "Expected read fraction")
stopifnot(max(abs(p[, 1] - p[, 2])) < 1e-12)

Figure 1: Two invented absolute communities with identical compositions. The expected library is the same at any fixed read budget although total biological quantity differs tenfold.

Both worlds have p=(1/6,1/3,1/2), but totals 60 and 600 in the toy’s absolute units. If a compatible independent measurement supplied a total T=\sum_jx_j, the ideal model would give x_j=Tp_j. Sequencing supplies p_j; the extra measurement supplies T.

Real load measurements must match the specimens and intended units. Converting a read fraction to cells per gram needs an argument about extraction, marker copy number, and calibration. Vandeputte et al. combine sequencing with flow-cytometric enumeration; this adds information absent from composition alone.

3 The reference defines a log-ratio question

For positive quantities, an additive log ratio compares feature j with a specified reference r:

\begin{aligned} L_{jr}&=\log_2(p_j/p_r)\\ &=\log_2(x_j/x_r). \end{aligned}

The target could be a difference in E[L_{jr}\mid A=a] between groups. It concerns feature j relative to reference r. A stable ratio can mean both quantities stay fixed or both increase together. It does not mean feature j is unchanged absolutely.

Under multiplicative feature biases, the measured log ratio gains the term \log_2(b_j/b_r). If that term is constant across people and groups, it cancels from a group mean difference. If recovery changes by group, it need not cancel. McLaren, Willis, and Callahan develop this measurement-bias model. A relative representation alone does not guarantee removal of every assay bias.

A centered log ratio (CLR) uses the geometric mean of a specified feature set as reference. For D positive components,

\operatorname{clr}_j(p) =\log_2p_j-\frac1D\sum_{k=1}^D\log_2p_k.

Changing the feature set changes this reference and potentially the contrast. Summing the displayed coordinates gives zero: the sum of the log components is subtracted once in total. Their covariance matrix is therefore singular; all coordinates cannot be treated as unrestricted variables in an ordinary inverse-covariance calculation.

The following exact contrast makes A and B both double in absolute quantity while C stays fixed. A/B is unchanged, but A/C rises. The two answers describe different questions.

x0 <- c(A = 10, B = 20, C = 30); x1 <- c(A = 20, B = 40, C = 30)
clr <- function(x) log2(x) - mean(log2(x))
contrast <- c(`A / B` = log2(x1['A']/x1['B']) - log2(x0['A']/x0['B']),
              `A / C` = log2(x1['A']/x1['C']) - log2(x0['A']/x0['C']),
              `CLR of A` = unname(clr(x1)['A'] - clr(x0)['A']))
names(contrast) <- c("A / B", "A / C", "CLR of A")
barplot(contrast, col = cols, ylab = "Log2 group contrast", ylim = c(-0.1, 1.2))
abline(h = 0)
contrast

    A / B     A / C  CLR of A 
0.0000000 1.0000000 0.3333333 

Figure 2: Exact log2 contrasts for an invented community change from (10,20,30) to (20,40,30). Reference choice changes the question: A/B is unchanged while A/C doubles. CLR uses all three features here.

The absolute log2 changes are (1,1,0). The A/B change is 1-1=0, A/C is 1-0=1, and CLR(A) is 1-(1+1+0)/3=1/3. If C is filtered out and CLR is recomputed over A and B, the reference change is 1 and CLR(A)’s change becomes zero. Filtering then changes the variable, not just the number of hypotheses.

These are population quantities. A log ratio calculated from finite counts is a noisy statistic; its expectation need not equal the latent log ratio. CLR creates dependence among coordinates within a specimen, and a reference estimated across people can introduce additional between-person dependence. The complete pipeline needs an inferential argument. ALDEx2 provides one framework for count/compositional uncertainty, with its own assumptions.

4 A zero does not diagnose its own cause

Under the multinomial model, a feature with p_j>0 can have zero reads. Its probability is (1-p_j)^N. Biological absence is a different possibility. Extraction failure, contamination filtering, and annotation rules create additional mechanisms. A zero count alone does not identify which occurred.

read_depth <- c(100, 1000, 10000)
data.frame(read_depth, probability_zero_if_present = (1 - 0.001)^read_depth)

  read_depth probability_zero_if_present
1        100                9.047921e-01
2       1000                3.676954e-01
3      10000                4.517335e-05

counts <- c(0, 5, 50)
sapply(c(0.1, 1, 5), function(offset) log2((counts[1] + offset) / (counts[2] + offset)))

[1] -5.672425 -2.584963 -1.000000

For p_j=0.001, the zero-count probabilities are approximately 0.905 at depth 100, 0.368 at depth 1,000, and 0.000045 at depth 10,000. The same present feature can often be missed in a shallow library. Observed detection prevalence can therefore differ between groups just because their depths differ.

An offset makes a count log ratio finite but defines an offset- and depth-dependent statistic. With offset 1, counts (0,5) give \log_2(1/6)\approx-2.58; multiplying counts by ten gives \log_2(1/51)\approx-5.67. The empirical composition is unchanged. The offset changes the statistic, not the evidence about an unobserved absolute scale. A biological-presence target needs a model connecting presence to the assay’s detection process.

5 Inference follows the target and sampling unit

For a defensibly defined, observed log-ratio outcome, the shared framework can estimate a covariate-adjusted contrast with flexible nuisance functions. Choose the reference, feature family, zero treatment, and target population before analysis; state any additional model needed to connect finite counts to the desired latent ratio. An end-to-end expression analysis demonstrates an estimation framework for an observed continuous outcome, not a turnkey estimator for latent microbial composition.

Repeated stools from one person remain dependent. Relative co-variation is not automatically an absolute ecological interaction. Distance-based comparisons depend on the representation and distance, and do not identify a changed taxon by themselves. See Dependent observations and cluster-aware inference, Network inference from omics data, and Multivariate comparisons: permutation and distance-based tests.

6 A practical audit

Keep person/specimen IDs, collection timing and specimen amount, extraction and sequencing batches, negative controls, reference/annotation versions, raw counts and read depth, and any load measurements. Inspect contamination and missing specimens, choose a reference/scale, and distinguish outcome-blind filtering from selection on group differences. Report whether the conclusion concerns read composition, a specified log ratio, measured load, or an identified absolute quantity.

7 Exercises

The checkpoint is this assay specialization’s completion target. Additional practice is optional. Try the checkpoint before opening its hint or solution.

Checkpoint · MIC-01 — Demonstrate scale nonidentification and name the reference

Task type: Derivation and interpretation.

Compare absolute communities (10,20,30) and (100,200,300). Show why the idealized fixed-depth library has the same distribution. Now compare (10,20,30) with (20,40,30): derive the log2 contrasts for A/B, A/C, and CLR(A) using all three taxa. Explain what a zero read count does not establish.

Ready to continue when: you can exhibit observationally equivalent absolute communities and distinguish reference-specific relative changes from absolute changes.

Transfer: If all taxa double and extraction efficiency halves, which external measurements would you need to support cells-per-gram claims?

Hint

Normalize by the total; for CLR subtract the average of the three log changes.

Solution outline — open after trying

Both scaled communities have probabilities (1/6,1/3,1/2), so their multinomial count laws agree at the same depth. In the second comparison the absolute log2 changes are (1,1,0); hence A/B changes by 0, A/C by 1, and CLR(A) by 1−2/3=1/3. A zero can occur by sampling even at positive abundance. More reads inform composition but not the unobserved overall biological scale.

Additional practice (optional)

1. Repeat the CLR calculation after filtering out C and explain the changed estimand.
2. For a fixed count offset of 1, compare log ratios for (0,5) and (0,50). Explain why depth now matters.
3. Write a longitudinal log-ratio question and specify person-level resampling and folds.

8 Reading

• Gloor et al., Microbiome datasets are compositional: the role of relative information.
• Vandeputte et al., quantitative microbiome profiling: sequencing plus microbial-load measurements.
• McLaren, Willis, and Callahan, consistent measurement bias: feature-specific efficiencies and ratio interpretation.
• ALDEx2 vignette: a worked compositional modeling framework.