Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [repeatability]

25 questions
29
votes
1 answer

Computing repeatability of effects from an lmer model

I just came across this paper, which describes how to compute the repeatability (a.k.a. reliability, a.k.a. intraclass correlation) of a measurement via mixed effects modelling. The R code would be: #fit the model fit =…
Mike Lawrence
  • 12,691
  • 8
  • 40
  • 65
10
votes
1 answer

How to compare repeatability (ICC) of different groups?

I have calculated the ICC values for two groups and would now like to compare the ICC values to determine if the groups differ in their repeatability. In the literature people have simply used t-tests to compare repeatability but it is unclear to me…
crazjo
  • 752
  • 1
  • 10
  • 19
9
votes
5 answers

How to assess repeatability of multivariate and method-specific outcomes?

Method "A" describes biological samples using multivariate "fingerprints" that consist of about 30 different variables. Different variables show different typical distribution and many of them closely correlate one with another. From prior…
David D
  • 717
  • 5
  • 13
6
votes
1 answer

Statistical significance, repeatability, and sample size (50 shades of grey)

Reading about the problems with significance I wondered how one would approach an experiment in the right way, considering that in practice the sample size cannot be arbitrarily large. While some meta-studies deal with confirmation bias and similar…
Stefan
  • 61
  • 1
4
votes
0 answers

Given $p=0.05$ in one study, what is the likelihood of $p \ge 0.05$ in a repeated study?

This question comes from a discussion on reddit. One user asked a question: Given $p=0.05$ in one study, what is the likelihood of $p \ge 0.05$ if the same study was repeated under the exact same condition in the same population? With a set of…
Karolis Koncevičius
  • 4,282
  • 7
  • 30
  • 47
4
votes
2 answers

Completely different results after each cross validation

I'm running some classification algorithms in MATLAB and validating them with a 10-fold cross validation. The problem is that every time I execute the cross validation, it gives a very different result. I see that it is normal the result to be a bit…
Alb_a
  • 115
  • 1
  • 8
4
votes
2 answers

How to assess whether experimental measurements obtained from different technicians are biased?

Suppose I have a list of measurements from an experiment; for example, 34 31 55 18 19 22 44 48 23 . . . But I then learn that these experiments were conducted by two different technicians, so I embolden those conducted by tech #1, and I leave…
Fixee
  • 555
  • 1
  • 4
  • 6
4
votes
1 answer

Confidence intervals for repeatability

I have calculated the repeatability of individuals' responses to a stimulus using the methodology of Lessells & Boag (1987) Auk 104:116, where repeatability r = among-groups variance component / (among-groups variance component + within-groups…
Freya Harrison
  • 3,212
  • 4
  • 25
  • 31
2
votes
1 answer

Determine individual consistency from mixed model?

To investigate the difference between three tasks I tested 30 individuals on each of the three tasks and ran the linear model as follows: model <- lme(behaviour ~ task, random = ~ 1|subject, data=dat, method="ML") With contrasts using glht to…
crazjo
  • 752
  • 1
  • 10
  • 19
2
votes
0 answers

How to analyze intra-rater reliability with 31 raters who all rated 4 subjects twice

I'm trying to analyze the intra-rater reliability of an occupational therapy assessment. The assessment consists of 57 items, where some items are ratio data and some items are ordinal data. In my study 31 raters rated 4 subjects (on video) twice…
Bianca van Keulen
  • 21
  • 2
2
votes
1 answer

Repeatability estimation using a mixed model with repeated measures by four raters

I am a medical doctor and definitely not an expert in statistics, although I think I do okay and I am familiar with R. I have discussed the following issue with a statistician but I am still not sure whether we came up with the right solution. I…
Dennis
  • 21
  • 3
1
vote
0 answers

What measurements or tests should I take for testing stability of patterns?

I have a dataset containing behaviours of users. Let's say we have 1000 users and 40 similar tasks. The tasks are similar but the difficulty level vary.To do a task, people can follow 10 patterns. Since the 40 tasks are similar, the 10 patterns…
nan
  • 905
  • 2
  • 10
  • 19
1
vote
0 answers

Correlation vs Standardized repeatability

Update: I reworded my question as I felt I didn't clearly state the root of the issue and inflated my question with too much irrelevant information. Repeatability and correlation are linked. Especially consistency repeatability, seems to be closely…
R. Iersel
  • 33
  • 7
1
vote
0 answers

How is it possible to control for observer bias in measurements with a random effect?

I have a dataset with repeated (but not always) measures of animals. I need to control for observer bias in measuring the morphology of my animal. I'm wondering what is the way to first estimate repeatability, and then correct the measurements of…
M. Beausoleil
  • 941
  • 3
  • 10
  • 23
1
vote
1 answer

What statistic to use to assess repeatability of measurement

I want to assess the repeatability of a measurement device. Let's say I have made three measurement with the same device under the same conditions. What statistic is useful for assessing the repeatability for such a case. An example of the…
H321
  • 11
  • 1
1
2