4

Gary King made the following statement on Twitter:

scale invariance sounds cool but is usually statisticians shirking responsibility & losing power by neglecting subject matter info

What is an example of this phenomena, where scale invariance causes a loss of power?

Edit:

Gary responded:

not [statistical] power, but scale invariance loses the power that can be extracted from knowledge of the substance

Now this makes more sense. How can scale invariance cause a loss of explanatory power from the resulting analysis?

Shane
  • 11,961
  • 17
  • 71
  • 89
  • sorry if it is my english (unfortunatly I'm french) but is it proper english ? what do you mean by "losing power" is it about testing? if yes, testing what ? – robin girard Aug 19 '10 at 14:22
  • 1
    There's nothing unfortunate about being French. :) Regretfully, I don't entirely know what Gary was talking about (hence the question). I might assume that he meant "loss of information", but would be happy to get guidance in that respect. I would say that this is correct english, but questionable statistics. – Shane Aug 19 '10 at 14:28
  • Perhaps, you can email him and ask? You could use the opportunity to point to this thread seeking his permission to post his answer here. (indirect promotion of the site) –  Aug 19 '10 at 14:41
  • @Srikant: Done; we'll see what he says. – Shane Aug 19 '10 at 14:54
  • I saw this as well, and assumed it was a warning against blindly studentizing data. – shabbychef Aug 19 '10 at 17:47
  • He says: "not stat'l power, but scale invariance loses the power that can be extracted from knowledge of the substance" http://twitter.com/kinggary/status/21624260888 – onestop Aug 20 '10 at 14:40
  • Thanks @onestop: now I understand what he meant. Even though I know the answer, I'll update the question for future searches. – Shane Aug 20 '10 at 15:21

3 Answers3

4

I still feel negatively about what seems to be a gratuitous insult on King's part but I can see where he might be coming from. "Scale-invariance" is a restriction on a statistical procedure. Thus, limiting our choice of procedures to scale-invariant ones (or to linear ones or to unbiased ones or minimax ones, etc.) potentially excludes procedures that might perform better. Whether this is actually the case or not depends. In many situations, data are reported in units that are essentially independent of what is being studied. It shouldn't matter whether you measure distances in angstroms or parsecs, for example. In this context, any procedure that is not scale invariant is therefore an arbitrary one--and arbitrariness is not a positive attribute in this field. In other situations, though, there is a natural scale. The most obvious of these concern counted data. A procedure that treats counted data as if they were measurements on a continuous scale (e.g., using OLS for a counted response) is potentially inferior to other available procedures and may be (likely is, I suspect) inadmissible in the decision-theoretic sense. This can be a tricky and subtle point because it's not always obvious when we have counted data. One example I'm familiar with concerns many chemical or radioactivity measurements, which ultimately originate as counts on some machine. Said counts get converted by the laboratory into a concentration or activity that forever after is treated as a real number. (However, attempts to exploit this fact in the chemometrics literature have not yielded superior statistical procedures.)

Just to stave off one possible misunderstanding: I wouldn't view a selection of an informative prior for a scale parameter (in a Bayesian analysis) as a scale-dependent procedure. Such a prior obviously favors some ranges of values over others, but does not affect the scale invariance of the procedure itself.

whuber
  • 281,159
  • 54
  • 637
  • 1,101
  • Could you tone down the criticism in this a little? Your point is fair, but I would love to see Gary get involved in the site (if at all possible)... – Shane Aug 27 '10 at 14:13
  • Normally I am reluctant to criticize (for example, I have yet to vote down any contribution in these forums), but the criticism seems warranted in light of the tone of that tweet. Nevertheless, I share your interests and will delete the unnecessary parts of the response. – whuber Aug 27 '10 at 15:19
  • Thanks. I'll delete my comments here too now so that this all just looks like a coherent train of thought. :) – Shane Aug 27 '10 at 15:22
3

We have a tendency to crunch data according to pre-established algorithms and methods, and forget that "data" is actually information about the real world. I recall as a child in school solving a second-degree equation where the teacher had stated that the answer represented the length of a pencil. Some students actually reported that the answer was "one inch plus or minus two inches".

Before you plug your data into any software, you should first get to really know and understand it, which you can only accomplish if you keep the subject matter in mind. That's the only way you can spot any quirky data points (such as a pencil measuring -1 inch) or determine which scales make sense in the real world.

Carlos Accioly
  • 4,715
  • 4
  • 25
  • 28
  • But what does this have to do with statistical "power" in the King quotation? (I would ask the same question of stormygeorge, too.) – whuber Aug 26 '10 at 01:15
  • 1
    @whuber Please see in the question where Gary King explicitly says that he is not talking about statistical power, but about "the power that can be extracted from knowledge of the substance". – Carlos Accioly Aug 26 '10 at 14:23
  • Ah, thanks: I was focusing on the first quotation only. But that leaves us wondering specifically what King means by "power." Evidently it's something that is gained by not "shirking responsibility" ;-). – whuber Aug 26 '10 at 15:52
1

Let me take a stab at this...

If I understand the original sentiment and subsequent response, I think what Gary King is getting at is that noticing a scale invariance effect on your data is a very gross understanding of the phenomena of whats going on. While this 'birds eye view' of whatever phenomena you are observing could be insightful, one might gloss over useful information at the microscopic level.

This example might not be the best, but consider Conway's 'Game of Life'. This is completely determined, as in it is a deterministic system. Consider looking at some statistic of this system, cluster longevity, say, for some appropriate definition of cluster. For arguments sake lets say this follows a power law (I don't know if it does or not, but just for this example, lets say it does). This gives a gross high level description of the system but you've washed all the details of how gliders race across the board, how they collide to give glider guns and other useful information that you might be able to use to determine some specifics about your system.

I'm not sure this is the best example or even if I've gotten the gist of what Gary King was trying to say, but thats my 2 cents.