Update: I reworded my question as I felt I didn't clearly state the root of the issue and inflated my question with too much irrelevant information.
Repeatability and correlation are linked. Especially consistency repeatability, seems to be closely linked to correlation. So closely in fact, that it makes me wonder what the major discernible difference between them is. For definition see:
( "consistency definition, concerns the degree to which one rater’s score (y) can be equated to another rater’s score (x) plus a systematic error (c) (ie, y = x + c), whereas absolute agreement concerns about the extent to which y equals x", from A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research)
I've been using these consistency repeatabilities, as I wanted to correct for learning between trials. But I am now starting to question whether this is the correct approach. Which leads me to my questions:
Q1: What is the added benefit of calculating consistency repeatabilities if one already has correlations? As they will (as far as I can tell) lead you to interpret the exact same thing.
Q2: When comparing repeatability across the same test, if one is interested in the repeatability of an individual, should one consider only agreement repeatabilities (even if e.g. learning is present and individuals appear to have differential learning slopes)?
