Base-Rate of Low, High, and Extreme Scores

Due to sampling errors, it is relatively common for an 'average' person to show one or more abnormally low or high (or both) scores in a test battery (similar to a 'type 1' error in statistics). The chances of this happening increases, the more tests you administer, and is also influenced by how closely correlated the tests are. This tool allows you to estimate how often a cognitively typical person would be expected to obtain one or more abnormally low, high, or extreme scores in a battery, accounting for the number of tests administered and the inter-test correlations.

Please note that this calculator uses statistical modelling only- actual population-based rates can be found in the wider literature (e.g. Binder, Iverson & Brooks, To Err is Human: “Abnormal” Neuropsychological Scores and Variability are Common in Healthy Adults, Archives of Clinical Neuropsychology 24(1): 31–46, https://doi.org/10.1093/arclin/acn001

Inputs

Number of tests (k) ▸ What does this mean?

If you include a subtest score (e.g. Digit Span) here, do not also include the index it contributes to (e.g. WMI). This is automatically set to 36, which reflects a typical test battery comprising: all WAIS-IV core subtests (10), all CVLT subtests including forced choice (12), all D-KEFS conditions (12), and WMS Logical Memory I & II (2).

Cut-off percentile per tail (e.g. 0.02 = 2nd percentile) ▸ What percentile should I enter?

Enter the lower-tail percentile corresponding to your definition of an 'abnormally low' score. 0.02 corresponds to the 2nd percentile (>−2 SD below the mean), a commonly used cut-off when identifying unusually low scores. A value of 0.07 would correspond to >-1.5 SD below the mean and 0.16 to >-1 SD below the mean.

Computation mode ▸ Which mode should I choose?

Use Approximation mode when you do not know the exact correlations between all tests. It uses an effective number of tests based on the mean inter-test correlation. Use Monte Carlo when you have a full correlation matrix and want a more precise estimate. Monte Carlo is slower but more accurate.

Mean inter-test correlation (r̄) ▸ Why does this matter?

The more correlated the tests are with each other, the fewer extreme scores you will see by chance. Typical cognitive batteries show inter-test correlations between 0.3 and 0.6. If unsure, 0.4 is a reasonable estimate.