6

I am a little bit confused about what an orthogonal design is and how it relates to the model matrix. It appears that there are many perspectives into the definition of orthogonal inside this post but none exactly helps me understand my problem. I am seeking an explanation with an experimental design context.

In Bailey 2008 pg 179, It introduces two factors G and F as orthogonal iff the subspaces $V_G \cap(V_F \cap V_G)^\bot$ and $V_F \cap(V_F \cap V_G)^\bot$ are both orthogonal to each other (or $V_G \cap V_{G \wedge F}^\bot$ and $V_F \cap V_{G \wedge F}^\bot$ is orthogonal)

A more intuitive theorem in the same book says (in terms of factors and levels) that

F and G on the same set are orthogonal to each other iff

  • every F-class meets every G-class
  • all these interesections have size proportional to the product of the sizes of the relevant F-class and G-class

However, the problem with these definitions is that it does help with 3 factors being orthogonal to each other or or when continuous covariates are included in your model. I was guessing that perhaps the model matrix when encoded can reveal something about orthogonality. However, my guess is that this is not true because It is obvious that when dummy coding the columns of the model matrix is not orthogonal to each other.

So my question is,

  1. What is a general definition of an orthogonal design? Is there a more general definition of orthogonal design? Including continuous covariates
  2. What are the advantages of orthogonal designs?
  3. Does the model matrix reveal anything about orthogonality?
Community
  • 1
tintinthong
  • 750
  • 2
  • 6
  • 17
  • The orthogonality of two factors in a design is a concept subtler than people would think of. A mathematical rigorous and intuitive definition (which has similar flavor to the definition you listed in your question) of orthogonality of design is given in Section 2.3 of *The Coordinate-Free Approach to Linear Models*. Though it might be quite abstract at the first reading, this is the best exposition of orthogonal design I have ever encountered. If I got time, I would like to summarize the author's idea therein. – Zhanxiong Jan 03 '18 at 05:25
  • I am not surprised that there is confusion about what is an Orthogonal Design. Orthogonal Designs were the subject of the 1979 book A.V.Geramita and Jennifer Seberry, Orthogonal Designs: Quadratic forms and Hadamard matrices, Marcel Dekker, New York - Basel, (1979), viii, 460 pages. This has now been republished as Jennifer Seberry, Orthogonal Designs, Springer Nature, 2017 – Jennifer Seberry Jan 03 '18 at 05:08

1 Answers1

6

  1. Perhaps you haven't fully grasped the definition yet. The requirements for orthogonal designs are that the blocking is orthogonal and the treatment is orthogonal. This simply means that crossproducts total to zero, whether the blocking or treatment is continuous, pseudo-continuous, polytomous, or binary. As @whuber correctly points out, statisticians often call dot products cross products, and furthermore often assume blocking and treatment factors have mean 0. So any blocking factor or treatment factor "crossed" with any other will come out to 0.

  2. Efficiency.

  3. Absolutely. We would expect that cross products between any two columns of the design matrix will total out to zero.

AdamO
  • 52,330
  • 5
  • 104
  • 209
  • 2
    +1 In (2) it might perhaps be at least as important to list *interpretability*. – whuber Aug 08 '16 at 15:28
  • @AdamO thanks for the reply. 1. I definitely haven't grasped the definition. Cross products of what exactly? 2. I am guessing that you mean dot product and not cross product (or do you really mean cross product). 3. Surely, one can make a design matrix (using dummy coding) such that dot product of two cols will not be zero. – tintinthong Aug 08 '16 at 16:33
  • @tintinthong I neglected to mention that one should consider all factors to be mean-centered. That is an (unfortunately) common omission. You're right about dot product. – AdamO Aug 08 '16 at 16:53
  • 2
    tintin, in the regression context the term "cross product" often is used to refer to dot products: that is, a sum of squares or products. And yes, of course you can design a non-orthogonal experiment (almost surely a random design matrix will not be orthogonal): but that possibility seems to have no relevance to your questions. – whuber Aug 08 '16 at 16:53
  • 2
    In addition to efficiency, if the design matrix is not orthogonal, there is no unique partition of the sums of squares. I think this is what @whuber means by interpretability: there would be different possible tests (w/ possibly different outcomes) & the data would not necessitate that 1 of them was the 'right' test. – gung - Reinstate Monica Aug 08 '16 at 16:58
  • @whuber I can create an experimental design eg 2 factorial. I run the model in R. I usually get that the factors are dummy coded into the design matrix. ie the cols of the design matrix are non-orthogonal. Does that mean that I did not create an orthogonal design? (Still unsure with definition) – tintinthong Aug 08 '16 at 21:08
  • As AdamO pointed out in a comment, orthogonality depends on how the variables are coded. Just as it's possible to find linear combinations of regressors that are orthogonal--but have no clear interpretation--it is, conversely, possible to create linear combinations of orthogonal regressors that are no longer orthogonal. Perhaps it would help to study the options offered by `R` for coding categorical variables. – whuber Aug 08 '16 at 21:50
  • @whuber What does it mean "any blocking factor or treatment factor "crossed" with any other will come out to 0."? E.g. let's have a latin square with rows 123,231,312. How can I see that here? – John V Jun 24 '17 at 14:11