3

If I have a large number of relationships I want to check that may or may not be linear (as in the bottom row of image below), is there a method that will reject the hypothesis the scatter plot is showing white noise? I want to check for the presence of any relationship without knowing what the form of that may be. What method/algorithm of measuring deviations from white noise captures the most different types of deviation?

enter image description here

image from: https://en.wikipedia.org/wiki/Correlations

Flask
  • 1,711
  • 1
  • 14
  • 24
  • 3
    You might want to check Brownian correlation http://en.wikipedia.org/wiki/Distance_correlation. It taps many but not all nonlinear "associations/shapes". – ttnphns Nov 19 '13 at 15:25
  • 2
    It is interesting that all but one of the scatterplots on the bottom can be identified with a (simple) test of homoscedasticity. The leftmost is arguably homoscedastic, but it is easily detected with a local smoother, unlike all but one of the others (the "smile"). The relevance for the present question is that you need to specify what forms of relationships are meaningful in your application. Otherwise, I can tell you with extremely high confidence *a priori* that *every* finite scatterplot you draw will have *some* non-white-noise "relationship" in it. – whuber Nov 19 '13 at 15:39
  • @ttnphns Thanks, that looks very relevant and I am investigating further. It still fails for the bottom right case though. I think there should be some way to do this in general since humans are obviously capable of detecting the structure by looking at the plots. – Flask Nov 19 '13 at 15:39
  • @whuber I agree there will always be some pattern that it is possible to see, yet I am capable of looking at a scatter plot and saying "that is for sure not just noise", which seems like I am performing a significance test. I do not think I am matching the patterns to previous ones I have seen, or sequentially matching combinations "primitive shapes" to the image then choosing the best fit. – Flask Nov 19 '13 at 15:44
  • Given that human eye is [apophenic](http://en.wikipedia.org/wiki/Apophenia) and that some real structures cannot be cought by eye, your wish for "general" universal detection is doomed – ttnphns Nov 19 '13 at 15:48
  • There are some interesting questions of human perception here, to be sure, but as a practical matter *what your eyes think they see* and *what matters for your application* are rarely the same. I'm just seeking clarification concerning whether your question is about psychology or statistics. If the latter, then please describe the kinds of patterns you need to detect. – whuber Nov 19 '13 at 15:48
  • @whuber I am pretty sure what I want is a test that rejects the null hypothesis that a 2-d data set, or possibly image, is white noise. The purpose is to categorize many different relationships into interesting and not interesting. If interesting I would inspect further. Distance correlations appears to be very close to what I am looking for. – Flask Nov 19 '13 at 15:51
  • 1
    The problem is that your description is inadequate: you need to stipulate more precisely *in what ways* a scatterplot might deviate from "white noise." This is perfectly analogous to simpler hypothesis tests, such as comparing two distributions. When we test whether data might come from a Normal distribution, we are testing for particular forms of non-Normality: shifts in location, scale, or even in a broader sense as measured by the max. difference in CDFs. Regardless, *some* measure of deviation from the null is required. How do you want to measure deviation from white noise? – whuber Nov 19 '13 at 15:59
  • @whuber "_How do you want to measure deviation from white noise?_" I guess this is my question. Rephrased: What method of measuring deviations from white noise captures the most different types of deviation? Distance correlation appears to perform very well, perhaps it is the best out there, but it seems like there should be a better algorithm since my brain can obviously do it. – Flask Nov 19 '13 at 16:07
  • Can you give a source for the image, or is it yours? – Nick Cox Nov 19 '13 at 16:09
  • @NickCox It was wikipedia, I added the link. – Flask Nov 19 '13 at 16:12
  • Thanks. Being pedantic, but the patterns for correlations of +1 or -1 are not quite right in so far as small deviations from linearity are implied. Otherwise I like it. – Nick Cox Nov 19 '13 at 16:16

1 Answers1

6

Maximal Information coefficient is one method that has been used for this. "In statistics, the maximal information coefficient (MIC) is a measure of the strength of the linear or non-linear association between two variables X and Y."

Detecting Novel Associations in Large Data Sets. D. Reshef et. al

http://en.wikipedia.org/wiki/Maximal_information_coefficient

pat
  • 3,706
  • 1
  • 22
  • 25
  • 1
    Important to look at the comments to that article too. – Nick Cox Nov 19 '13 at 16:27
  • @pat Thanks, [this paper](http://arxiv.org/pdf/1301.7745v1.pdf) linked in the comments also seems relevant. It is unclear to me from scanning the two papers whether the MIC may superior for small datasets (mine are actually ~n=10) – Flask Nov 19 '13 at 16:56
  • Not certain how well it would do on a sample that small, but there was a comment on relaxing false discovery rates to find significant associations for a small GENE dataset (n=24) in the Science article. – pat Nov 19 '13 at 17:21