9

The second peak of the density plot is large in this example. Why does the rug representation of the data--which seems to show few high values--not appear to match the much higher density estimated there?

How can one make rug plots less misleading?

enter image description here

Here's its R code:

plot(density(Boston$tax))
rug(Boston$tax, col=2, lwd=3.5)
whuber
  • 281,159
  • 54
  • 637
  • 1,101
Mona Jalal
  • 514
  • 6
  • 17
  • 2
    You are using subjective terms like "so high" and "inadequate"...why is the second peak "too high" and why is the rug function not acceptable? What is it doing that you don't like. –  Feb 12 '14 at 04:40
  • This question appears to be relying on unjustified premises (at least in general). Are these questions someone else has set for you to do? – Glen_b Feb 12 '14 at 06:49
  • 2
    How many data points are above the level of 600? – Michael M Feb 12 '14 at 07:25
  • 5
    The "problem" with the rug plot is just that one tick and several ticks in the same place look the same. You could use a spike for each distinct value whose length was proportional to number of values, i.e. a histogram with bins the distinct values. – Nick Cox Feb 12 '14 at 11:05

2 Answers2

13

The way to make a rug plot less misleading is often to use something different instead. Rug plots necessarily are relatively good at showing distinct values and very poor at indicating their relative frequency.

Here is a spike representation of the frequency distribution of the data used in the original post. The principle is that often used to show discrete distributions, a spike proportional in height to the frequency of each distinct value. The Boston data as available and documented here were read into Stata and the spikeplot command used. Something similar should be trivial in all good statistical software. If you like, this is a hybrid of a histogram and a rug plot, although historically graphs of this kind may long predate rug plots.

enter image description here

Stata documentation for spikeplot, including, more to the point of this question, further examples of such graphs, is accessible here

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
11

From the R package MASS, of the $506$ total observations in Boston, $369$ have a value for tax below 470 and $137$ have a value for tax above 665. In fact 666 is by far the most common value in the data set, appearing $132$ times.

So if the area of the density plot to the left is about twice the area to the right, then that could reasonably be taken as representing the distribution. Visual inspection suggests this might be what is happening.

A more accurate representation would have the right peak much higher and narrower, and this could be achieved by adjusting the parameters.

Added for comments:

For example with a much narrower bandwidth for the density function and some manual jitter:

library(MASS)
plot(density(Boston$tax, bw=5)) 
rug(Boston$tax + rnorm(length(Boston$tax), sd=5), col=2, lwd=3.5)

you would get something like this

Bostontaxdensity

Henry
  • 30,848
  • 1
  • 63
  • 107
  • 1
    You have hit upon the reason for the discrepancy. But upon adjusting the parameters of the density estimate, the *area* of the right peak should not change and so there will still be a strong visual discrepancy between what the rug plot displays and the density plot displays. Effective solutions will--as the O.P. suggests and as @Nick Cox hints in a comment--modify the rug plot rather than the density plot. – whuber Feb 12 '14 at 16:50
  • 3
    Yeah, I used jitter for modifying the rug. – Mona Jalal Feb 12 '14 at 17:09
  • @MonaJalal could you post what the chart looks like after you made your changes? I'd be interested in seeing it. – Rick Henderson Apr 04 '16 at 17:34
  • 2
    @RickHenderson: I added a chart with a narrower bandwidth and manual jitter – Henry Apr 04 '16 at 19:18