The purpose of this study was to verify whether the computer-based test is appropriate for measuring Korean speaking ability by comparing the computer-based test with the face-to-face interview test in the Korean speaking test as a foreign language. F... The purpose of this study was to verify whether the computer-based test is appropriate for measuring Korean speaking ability by comparing the computer-based test with the face-to-face interview test in the Korean speaking test as a foreign language. For this purpose, two test modes were compared by analyzing psychometric properties at the test and item level, the examinee performance and the rater performance based on the results of the computer-based test and the face-to-face interview test. The results of this study were as follows. First, regarding the comparison of the psychometric properties at the test level, the correlation coefficient and disattenuated correlation coefficient between the test scores calculated from the computer-based test and the face-to-face interview test were highly correlated. In addition, convergent validity was confirmed through the multitrait-multimethod matrix using the correlation coefficient of items between the two test modes. The multi-group analysis in structural equation modeling confirmed the configural invariance, metric invariance, scalar invariance, and factor variance invariance of factor structure between the computer-based test and the face-to-face interview test. Comparing the test reliability of the computer-based test and the face-to-face interview test with the Cronbach α and the generalizability coefficient, the reliability of the two test modes was found to be similar, which was above 0.8. Second, in terms of the comparison of the psychometric properties at the item level, patterns of the item difficulty and item discrimination index in the computer-based test and the face-to-face interview test were very similar. The item difficulty average of the two test modes was identical. Also, the mean difference of item discrimination was 0.01, which was very slight. Moreover, there was no statistically significant difference in the average of item difficulty based on the many-facet Rash measurement between the two test modes. However, it was revealed that there was a statistically significant difference in item difficulty of two items. In order to find out whether this difference was due to the test mode, the effect of the interaction between test mode and item on the total test scores was examined. The results showed that the interaction effect between test mode and item was 0% in the total test score variance, indicating that there was no interaction effect between test mode and item. Thus, the difference of item difficulty of two items was not the result of the test mode effect. Third, in relation to the comparison of the examinee performance, the mean of the total scores calculated by the computer-based test and the face-to-face interview test did not show a statistically significant difference. Moreover, the analysis of the many-facet Rasch measurement showed that there was no statistically significant difference in the examinee ability parameters between the computer-based test and the face-to-face interview test. Furthermore, the generalizability theory was used to investigate the effect of test mode on the total test scores. The results showed that the variance rate of the test mode effect was 0%, which means that the effect of test mode on test scores was little. Fourth, with regards to the comparison of the rater performance, the analysis of many-facet Rash measurement revealed that there was no statistically significant difference in terms of the mean of the rater severity parameters between the computer-based test and the face-to-face interview test. The inter-rater correlation and inter-rater agreement in the computer-based test were higher than the ones in the face-to-face interview test. According to the total test score variance, the variance of interaction between test mode and rater performance was 0.18%, indicating that the interaction effect of between test mode and rater performance was quite small. In conclusion, in the Korean speaking test, the differences of the psychometric properties at the test and item level, the examinee performance and the rater performance between the computer-based test and the face-to-face interview test were statistically insignificant or so small as to be negligible. The results of this study suggest that the computer-based test functions very similarly to the face-to-face interview test in the Korean speaking test. Therefore, the computer-based test can be regarded as an appropriate test mode in measuring Korean speaking ability.
참고문헌 (Reference)
활용도 분석
View
Usage
Share
|
