0

It appears that reliability and construct validity have a similar backing in terms of statistics. Is it true ?

  • Your title asks one question, your text asks a different question. – Peter Flom Jan 13 '14 at 12:28
  • what is the minimum sample required for a construct validity through PCA (using a PLS)? is 40 enough for example? –  Jun 17 '14 at 11:31

2 Answers2

4

No, reliability and construct validity do not have a similar purpose.

Reliability is about whether a test measures something accurately and reliably. Validity is about whether it measures what it purports to measure.

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
  • I think validity here reflects a different thinking and still converges into reliability. It could be equivalent groups from different schools that generate approximately equal results say in terms of I.Q. or weight. If you want to check for validity (here meaning cross-validation, you need to have dissimilar groups from the same school or from other school with dissimilar characteristics. –  Jan 13 '14 at 12:48
  • 1
    No, it does not. Reliability can be assessed based on a single measure on one sample on one test (e.g. split half reliability) or two measures on the same sample on the same test (test retest reliability). Validity cannot. – Peter Flom Jan 13 '14 at 13:26
4

To expand a bit on @PeterFlom : Think of a target at which an individual can shoot.

Reliability means that the shots are all close together, but it does not necessarily mean that they have to be close to the bullseye.

Validity means that the shots are on average close to the bullseye, but it does not necessarily mean that the shots are close together.

So a valid measure can be reliable or not and a reliable measure can be valid or not; all combinations are perfectly feasible. So these are really two separate characteristics of a measurement.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
Maarten Buis
  • 19,189
  • 29
  • 59
  • I think your is somewhat different understanding of the idea and psychometry. The psychometry has different ideas to reach at truth and statistics presumes and applies different ideas to concord with the philosophy of psychometry ie reliability theory. A measure could be reliable but not valid. –  Jan 13 '14 at 13:22
  • 2
    My PhD is in psychometrics and @Maarten is correct. – Peter Flom Jan 13 '14 at 13:24
  • Thanks. I do not have any intention to cross-examine the user. I simply have understood from the books and other literature in statistics and its applications in psychology what I stated. –  Jan 13 '14 at 13:36
  • @subhash: Can you give any reference for a _different_ understanding of the concepts of reliability & validity? – Scortchi - Reinstate Monica Jan 13 '14 at 13:45
  • 1
    @Maarten, Albeit I concur with your response, on the overall, your metaphor about valididty is questionnable. Do you want to say that sending shots to 1 m left and 1 m right from the bullseye is as "valid" as sending them to 1 cm left and 1 cm right from it? – ttnphns Jan 13 '14 at 14:05
  • 1
    Yes, that is exactly what I mean. Your example shows that it is a good idea to look at more than one criterium. In this example it would help to also look at reliability. Based on that criterium you would obviously prefer the ones that are cms apart. It becomes an interesting tradeoff when the cms appart are somewhat less valid than the ones ms apart. – Maarten Buis Jan 13 '14 at 19:55
  • 1
    @Maarten, your sense of validity as, actually, unbiasedness, is uncommon and counterintuitive. It appears that very blunt instrument is as valid as a sharp one, if they are both unbiased. In fact, the correct definition of valid series of shots is the series which is as close as possible to the bullseye and as far as possible from some "false bullseye" which you want not to hit. – ttnphns Jan 14 '14 at 08:59
  • 1
    P.S. The problem with your explanation in the example herein is that you treat deviations from the bullseye (the misses) as random errors. But validity is defined vis-a-vis systematic errors only (such as "false" bullseyes around the true one, which possibly are the projections of the shooter's idiosyncratic technical flaws on the target). It is reliability, not validity way of thinking, which deals with random errors. – ttnphns Jan 14 '14 at 09:23