What is Exact definition of Invariance principle

Question

Books wrote a lot about invariance definition and method to obtain invariance estimators, tests, and etc. However I couldn't find exactly the definition of principle of invariance. Is there any good source of clear definition of this principle?

Answer 1

The basic intuition behind invariance is that statistical conclusions should not depend on choice of measurement scale. Some examples:

1. Measurement of distance in meters or parsecs.

2. Angle measurement in degrees or radians

3. Choice of origin of measurement---Geography doesn't change if you change from the Greenwich meridian to the Paris meridian!

4. ...

This kind of change of scale and origin can be expressed as transformations of the form $ y \to a y+c$. The invariance requirement is then that the two following paths should lead to the same conclusions: $$ 1\colon \text{data $y$} \to \text{transform to $ay+c$} \to \text{analyze on transformed scale} \to \text{transform conclusions back to original scale} \\ 2\colon \text{Analyse on original scale}\to\text{conclusions on original scale.} $$ Mathematicians call this a commutative diagram. This gives some intuitive understanding for the invariance principle. Seems like quite obvious? but in application is maybe not that obvious, and have been developed into a rich theory with connections to for instance noninformative priors in Bayes analysis, and also to many other ways of transforming data (and models, parameters) then the one example above.

One book treatment giving definitions is Xi'ans own The Bayesian Choice (chapter 9 in second edition). A paper you could look at with interesting applications is this one.