A document I read many years ago (to do with actuarial education; sorry I can't link to a copy) used chi-square tests to assess lack of fit of various models to mortality rates which were treated either as (scaled) binomial or Poisson, or as normal.
It advocated a blanket rule of thumb for dealing with models identified by 'looking at the data', including placement of knots for splines: it recommended a deduction of 25% of the degrees of freedom (this is in addition to any loss for the number of parameters fitted).
Now it's clear that the resulting statistic won't be chi-square, and the 25% makes no reference to what was actually done, and the document admits that it's somewhat arbitrary (... to me that seems to be a large understatement).
[The document itself contains many other more or less puzzling proscriptions and suggestions. It contains few references or sources for such things, most of which have been passed down for what turns out to be a surprising number of decades, basically as somewhat justified folklore, and their origins seem to have been forgotten. Nevertheless, over a period of time, I managed to track down sources for almost all the suggestions and statements that weren't already obvious or standard - usually ancient papers. However, this last one eludes me. He tasks me, and I shall have him.]
Even leaving aside the distribution and just working on expectation I can see no reasonable way to justify any particular number, whether 10% or 50% or anything else.
Can you think of a possible source from which such a number might have some justification?
Failing that, can you perhaps think of a way of even very roughly justifying/intuiting a 25% reduction in degrees of freedom?
Any pointers or ideas would be helpful.
