Visual assessment of movement quality: a study on intra- and interrater reliability of a multi-segmental single leg squat test

Table 3 Interrater reliability for experienced raters with > 20 years of clinical experience and novice rater with ≤4 years of clinical experience

Raters	PA^a	Kappa^b (CI 95%)	Kappa_max^c	PI^d	BI^e	PABAK^f (CI 95%)
Experienced
Rater 1 vs. Rater 2
Foot	1.0	1.00 (1.00–1.00)	1.0	0.91	0	1.00 (1.00–1.00)
Knee	0.71	0.42 (0.21–0.64)	0.73	0.09	−0.14	0.42 (0.19–0.64)
Pelvis	0.77	0.44 (0.22–0.66)	0.52	0.46	− 0.20	0.54 (0.33–0.75)
Trunk	0.86	0.63 (0.40–0.85)	0.71	0.52	−0.11	0.72 (0.55–0.90)
All segments^g	0.84	0.57 (0.46–0.68)	0.71	0.50	−0.11	0.67 (0.58–0.76)
Novice
Rater 3 vs. Rater 4
Foot	0.99	0.66 (0.02–1.00)	0.66	0.95	0.02	0.97 (0.91–1.00)
Knee	0.88	0.41 (0.10–0.72)	0.88	0.69	−0.03	0.69 (0.51–0.87)
Pelvis	0.88	0.44 (0.12–0.76)	0.58	0.75	0.09	0.75 (0.60–0.92)
Trunk	0.89	0.68 (0.46–0.90)	0.68	0.58	0.11	0.79 (0.63–0.94)
All segments^g	0.90	0.55 (0.40–0.70)	0.79	0.75	0.05	0.80 (0.73–0.87)
All raters	PA^a	Generalised kappa^h (CI 95%)				PABAK^f (CI 95%)
Rater 1–4
All segments^g	0.85	0.52 (0.43–0.61)				0.70 (0.65–0.76)

^aPA Percent agreement
^bKappa: Cohen’s kappa, calculated by; \( \upkappa =\frac{P_o-{P}_c}{1-{P}_c} \)
Where; P_o (observed agreement) \( =\frac{a+d}{n} \) and P_c (chance agreement) \( =\frac{\left(\frac{f_1x{g}_1}{n}\right)+\left(\frac{f_2x{g}_2}{n}\right)}{n} \)
^cKappa_max: Is calculated so that the proportions of positive and negative judgements by each rater (i.e. the marginal totals) are taken as fixed, and the distribution of paired ratings (i.e. the cell frequency a,b,c and d) is adjusted so as to represent the greatest possible agreement. That will say, the maximum possible agreement for either presence or absence of the disease is the smaller of the marginal totals in each case [39]
^dPI: Prevalence index, calculated by; \( PI=\frac{a-d}{n} \)
^eBI Bias index, calculated by; \( BI=\frac{b-c}{n} \)
^fPABAK: Prevalence-adjusted bias-adjusted kappa, calculated by; PABAK = 2P₀ − 1
^gAll segments: Denotes a merged kappa coefficient for the interrater reliability of each of the segments together (foot, knee, pelvis and trunk)
^hGeneralised kappa: A generalisation of Scott’s pi presented by Fleiss in order to calculate the interrater reliability of multiple raters [40, 42]

ISSN: 2052-1847