11

There are more phenomena to which experimental design may be applied than there are alternative valid design strategies. This should be true, though there are many ways to properly design an experiment.

What are the best "problems" that truly demonstrate the value and nuance for the different types of optimal design of experiments? (A, D, E, C, V, phi, ....)

Can you provide books, links, articles, references, or at least good empirically driven opinions?

EngrStudent
  • 8,232
  • 2
  • 29
  • 82
  • 1
    Atkinson & Donev, *Optimum Experimental Designs* is a good reference for the alphabetic optimality criteria. – Scortchi - Reinstate Monica Aug 06 '13 at 17:21
  • 2
    I own that one. It was the textbook for one of the courses in my masters program, so I have read it aggressively. It is all in SAS (I'm a MatLab guy) but more importantly - although it enumerates the procedure to implement each of the styles of optimal DOE, it does not give a characteristic application. For instance, there exists a variation on c or L optimality that accounts for the cost of executing the particular experiment but there is no "canonical" example showing its implementation, nor a discussion of why it is the canonical example. – EngrStudent Aug 06 '13 at 19:57
  • I have no answer for this bounty, yet. – EngrStudent Aug 13 '13 at 13:42
  • Did you learn about any datasets where one can test the different design criterions? Preferably some without time component as in the NIST link. – scleronomic Sep 14 '20 at 13:13
  • I know some, but don't have a good and traversive answer. – EngrStudent Sep 14 '20 at 13:14

1 Answers1

4

This is a work in progress, and it is meant to answer my own question. (Not complete yet)

Common types of Optimal

NIST provides (link) the following definitions for the types of Optimal Design of experiments.

A-Optimality
[A] criterion is A-optimality, which seeks to minimize the trace of the inverse of the information matrix. This criterion results in minimizing the average variance of the parameter estimates based on a pre-specified model. The fundamental assumption then is that average variance of the prior model describes overall variance of the actual system.

D-Optimality
[Another] criterion is D-optimality, which seeks to maximize |X'X|, the determinant of the information matrix X'X of the design. This criterion results in minimizing the generalized variance of the parameter estimates based on a pre-specified model. The fundamental assumption then is that the generalized variance of the prior model describes overall variance of the actual system.

G-Optimality
A third criterion is G-optimality, which seeks to minimize the maximum prediction variance, i.e., minimize max. [$d=x'(X'X)^{-1}x$], over a specified set of design points. Like $H_{\infty}$ control this minimizes the maximum error given the prior model.

V-Optimality
A fourth criterion is V-optimality, which seeks to minimize the average prediction variance over a specified set of design points.

Requirements and ...

NIST says that the requirements include:

  • An a-priori appropriate analytic model
  • A discrete set of samples points as candidates elements of the DOE

Working

Here are "textbook" statistical analyses. DOE should apply to them, and if there is a healthy connection between "textbook statistics" and "statistical design of experiment" then they should be relevant for the answer of this question.

http://www.itl.nist.gov/div898/handbook/eda/section3/4plot.htm

The NIST case studies include:

  • Normal random numbers
  • Uniform random numbers
  • Random walk (running sum of shifted uniform random)
  • Josephson junction cryothermometry (discretized uniform random)
  • Beam deflections (periodic with noise)
  • Fitler transmittance (autocorrelation polluted measurements)
  • Standard resistor (linear with additive noise, violates stationarity and autocorrelation)
  • Heat flow (well behaved process, stationary, in control)
Community
  • 1
EngrStudent
  • 8,232
  • 2
  • 29
  • 82