Visualizing repeated measures (not longitudinal)

Question

I have repeated measures for a large number of variables and about a hundred individuals.

These measures are repeated to assure reproducibility and are not longitudinal time points.

I want to provide summaries and/or plots for these variables, but any calculation across the whole column (even weighted on the number of measures per individual) would lose the important information of the intra-individual variance.

On the other hand, presenting grouped data for this many individuals is not realistic.

Here is a simulation on 9 individuals of the unsatisfying plots I have so far. Both are not scalable with a lot of individuals.

library(tidyverse)
N1=9 #individuals
N2=25 #measures
#for each N1 individuals, take N2 values based on a specific mean and variance (both from a normal distribution)
df = expand.grid(individual=factor(1:N1), measure=LETTERS[1:N2]) %>% 
  arrange(individual) %>% 
  group_by(individual) %>% 
  mutate(
    base_mean = rnorm(1, 0, 50),
    base_var = abs(rnorm(1, 0, 10)),
    value = rnorm(n(), base_mean, base_var),
  ) %>% 
  identity()

#draw 1 boxplot with individuals as colors
ggplot(df, aes(x="x", y=value)) + 
  geom_boxplot() + 
  geom_jitter(aes(color=individual), width=0.1, alpha=0.9)

#draw 1 boxplot per individual
ggplot(df, aes(x=individual, y=value)) + 
  geom_boxplot()

^{Created on 2021-09-18 by the reprex package (v2.0.0)}

Is there a way to visualize or summarise the data on both intra- and inter-individual levels?

Answer 1

In my opinion the 2nd plot is pretty good. I might just add colour = so that each individual has their own colour, but the two main things that jump out about that plot are:

• there is considerably variation between individuals

• there is, by comparison, much less variation within individuals

• there is considerable heterogeneity. In particular, three individuals appear to have extremely low variation

Answer 2

Ed Tufte's spare redesign of the boxplot permits a large "small multiple" graphic to be displayed. Another point Tufte makes is that by ordering small multiples according to another factor, one often gets "free" information out of the graphic. Ordering the plots by median or box height is usually insightful, because relationships among the statistics (especially between level and spread) suggest useful ways of re-expressing the data.

Here are examples based on the code in the question (to generate sample data) and code offered by former CV moderator chl to make the plots.

Nine boxplots

100 boxplots

500 boxplots (log scale)

50 boxplots ordered by spread

R code

#
# Courtesy chl.  Code has been simplified and customized.
#
tufte.boxplot <- function(x, g, thickness=1, col.med="White", ...) {
  k <- nlevels(g)
  plot(c(1,k), range(x), type="n",
       xlab=deparse(substitute(g)), ylab=deparse(substitute(x)), ...)
  for (i in 1:k)
    with(boxplot.stats(x[as.numeric(g)==i]), {
      segments(i, stats[2], i, stats[4], col=gray(.10), lwd=thickness) # "Box"
      segments(i, stats[1], i, stats[2], col=gray(.7))   # Bottom whisker
      segments(i, stats[4], i, stats[5], col=gray(.7))   # Top whisker
      points(rep(i, length(out)), out, cex=.8)           # Outliers
      points(i, stats[3], cex=1.0, col=col.med, pch=19)  # Median
    })
}
#
# Create data.
#
N <- 9       # Number of individuals
# N <- 100
# N <- 50
set.seed(17) # For reproducibility

# Vary the counts, medians, and spreads
l <- lapply(3 + rpois(N, 5), function(n) 
  exp(rnorm(n, log(rgamma(1, 20, scale=1/20)), sqrt(rgamma(1, 15, 60))))
)
df <- do.call(rbind, lapply(seq_along(l), 
                     function(i) data.frame(Individual=factor(i), Value=l[[i]])))
#
# Visualize.
#
# Order by decreasing median
df$Individual <- with(df, reorder(Individual, Value, function(x) -median(x)))
# Alternatively, order by decreasing IQR
df$Individual <- with(df, reorder(Individual, Value, 
                                  function(x) diff(quantile(x, c(3/4, 1/4)))))

with(df, tufte.boxplot(Value, Individual, bty="n", xaxt="n", log="", 
                       col.med="#8080f080", thickness=2,
                       main="Ordered Boxplots Ordered by Spread (IQR)"))