Correlation plots with missing values

Question

I have 9 vectors of numeric data. I want to make correlation plots for these 9 vectors. However some vectors are of a different length to others - some have up to 240 observations while others have only 159 observations. R can only make correlation plots for vectors of the same length. To make the vectors the same length, I simply need to use:

length(x) = length(y)

The problem is that this chops of up to 81 observations from some vectors. Is there any way round this, other than simply chopping off observations? If I do need to chop off observations, how do I determine which observations to chop off?

Answer 1

Say you have two random variables, $X$ and $Y$.

You are interested in the (linear) relation between $X$ and $Y$.

To investigate this, you took a sample of $n$ independent individuals and you measured both $X$ and $Y$ on each individual.

This results in your sample of observations at hand, $(x_1, y_1), \ldots, (x_n, y_n)$, from which you can compute the empirical correlation which provides an estimate of the true correlation of interest.

But, of course, both $x_i$ and $y_i$ should be taken on individual $i$. You should therefore have equally sized vectors of observations. Otherwise, you made something wrong in your experiment, and you cannot investigate correlation with your data.

Now, missing values may occur. For example, it may be that you managed to measure $X$ on individual $i$, but not $Y$. In that case, it might be reasonable to ignore individual $i$.

Answer 2

It is possible to display complete cases and missing values altogether: just assign a pair of artificial values to observations having one missing value, such that they appear, e.g., in the lower left of your display without overlapping with pairwise complete observations (it is also possible to add small perturbations to their coordinates, like random jittering, to avoid overlapping, and use a different color or symbol shape). An alternative solution is to plot missing values in the margin, as shown below (Source: InfoVis 03 conference):

In any case, you should work with pairwise complete observations, for visualization purpose, not complete cases on the entire data set.

Klimt also handles missing values in a nice way as they are displayed along the horizontal (when y-value is missing) and vertical (when x-value is missing) axis, as shown in the following picture.

Here is a little R function that should handle this:

miss.value.plot <- function(x, y, ...) {
  plot(x, y, ...)
  rug(jitter(x[is.na(y)], amount=.1), side=1, lwd=1.2)
  rug(jitter(y[is.na(x)], amount=.1), side=2, lwd=1.2)
}

It can be used as a panel for pairs as well: Just replace plot with points. Of course, you can replace rug with points if your prefer to draw points or other symbols. In this case, you will have to provide constant values for the x or y complementary coordinates.

It is also possible to rely on software that allows dynamic brushing and linked plots, like GGobi. Basically, the idea is to have your original data matrix, and a copy of it, only composed of 0's and 1's and coding for missingness. Linking the two tables allows to study relationships between any two variables while looking at pattern of missingness in the other variables. Some of these techniques are discussed in the chapter on Missing Data from Interactive and Dynamic Graphics for Data Analysis: With Examples Using R and GGobi, by Cook and Swayne. There is an R interface that allows to work directly from R. This really is for situations where missing patterns are of interest. See also MissingDataGUI: A Graphical User Interface for Exploring Missing Values in Data.

Another software for interactive scatterplot displays is Mondrian. (There was also Manet but I cannot get it to work on my Intel Mac anymore.)

Answer 3

Use dummy variables if you wish to have n = 240