Citlivosť na zmenu je dôležitou témou v hodnotení kvality meracích nástrojov na identifikáciu psychoterapeutickej zmeny. Štúdia sa snaží zamerať na problém, ako môže túto citlivosť ovplyvniť psychometrický prístup použitý pri hodnotení. Predmetom štúdie je porovnanie klasickej teórie testov (CTT) a teórie odpovede na položku (IRT) v ich citlivosti na zmenu. 166 osobám podstupujúcim psychoterapeutickú a psychiatrickú intervenciu bol administrovaný dotazník CORE-OM (Evans et al., 2000) pred a po intervencii. Na hodnotenie možnej zmeny boli vypočítané tak klasické skóre, ako aj IRT skóre. V oboch prípadoch bol použitý 95% konfidenčný interval štandardnej chyby merania na identifikáciu signifikantnej zmeny. Výsledky ukázali, že IRT prístup identifikoval viac osôb so signifikantnou zmenou než CTT. Ako sa ukázalo, dôvodom bola variabilita štandardnej chyby merania na špecifických úrovniach škály. and Measurement of psychotherapy change: Comparison of classical test score and IRT based score
Objectives. The present study focuses on the problem of sensitivity to change as an important issue in evaluating the quality of measures focusing on psychotherapy change. It deals with the question on the problem how a psychometric approach used in evaluation can influence this sensitivity. The goal of the study is to compare the two psychometric approaches: Classical test theory (CTT) and Item response theory (IRT), in their sensitivity to change.
Subjects and setting. The sample was 166 persons undergoing psychotherapeutic and psychiatric treatments as outpatients and inpatients. Their symptom level was assessed by COREOM, 34-items long questionnaire based on selfrating. It was made specifically for measurement of psychotherapeutic change. Statistical analysis. Both CTT and IRT scores were computed as sources for assessing possible change. For IRT score, Samajima graded response model was used. In both cases, 95% confidence interval of standard error of measurement (SEM) was used to determine the statistically significance of change.
Results. The results showed that IRT approach identified more persons with statistically significant change in comparison to CTT. As it was found, higher sensitivity of IRT was due to variability of SEM in IRT approach, which allows more reliable measurement on specific levels of trait scale. Also, difference in CTT and IRT was found in the estimation of the trait levels, especially at low level.
Study limitation. The generalization of the results depends on item parameters of the considered scales. Concept of clinical changes should be taken into consideration, to analyze whether statistically significant change identified by IRT corresponds with actual changes in client’s functioning.