The purpose of this study is to investigate how the scores of Korean language raters are influenced by the rater’s characteristics when evaluating the writing of learners who are learning Korean as a second foreign language. Rater characteristics fo... The purpose of this study is to investigate how the scores of Korean language raters are influenced by the rater’s characteristics when evaluating the writing of learners who are learning Korean as a second foreign language. Rater characteristics for this purpose are rater's scoring strictness, native speakers(Korean) and non - native speakers(non-Korean), and Korean language teaching experience. The 24 raters have majored Korean language education, also, they were 12 Korean language teachers who were native speakers of Korean language teaching and 12 Korean language teachers who were non-native speakers of Korean language teaching. The raters consisted of 12 instructors with less than 5 years and 12 instructors with more than 5 years depending on their teaching experience in Korean. Based on the test of five written s written by five intermediate learners as holistic scoring and analytic scoring, The Cross-Classified Multilevel Model was applied to analyze whether there is a difference in the scoring scoring between native Korean raters and non-native foreigner raters. And I found out whether there is an mean difference in scoring based on assessment methods. Also, An analysis of the nationality of the Korean raters and foreign raters' nationality and Korean teaching experience were analyzed on the basis of the Korean citizenship. The results of this study are summarized as follows. First, the Cross-Classified Multilevel Model was applied to examine the characteristics of the rater. Intraclass Correlation Coefficient(ICC) was obtained from statistical values derived from the unconditional model, which is the basic model of the Cross-Classified Multilevel Model. It was understood that the rater's grading standards were very large with 34.1 %. Second, in the holistic scoring and analytical scoring, averages of the holistic scoring are 71.716 and analytical scoring are 60.924. As the overall mean difference between the two assessment methods is 10.792, holistic scoring score was 10.792 higher than the analytical scoring score. Third, in the study which explains the influence of the rater 's nationality on the testing, the raters of the Korean who are the native speakers in the holistic scoring are 71.085, and the raters of the foreigners who are the non-native speakers are 72.348. The Korean raters, who are native to the analytic scoring score, are 60.925 and the foreigner raters, who are non-native are 62.188. The mean difference between the holistic scoring and analytical scoring methods in two assessment groups is 1.263. This shows that there is little difference, and that the nationality of the rater does not have a significant impact on scoring. Fourth, if you look at the results of the survey, which reveals the impact of the raters' teaching experience and evaluations, the holistic scoring of raters with less than 5 years of Korean teaching experience is 71.684, and the holistic score of raters with more than 5 years of Korean teaching experience is 71.749. The analytical scoring of raters with less than 5 years of Korean teaching experience is 60.932, and the analytical score of raters with more than 5 years of Korean teaching experience is 60.997. Difference between the two scoring methods for two groups is almost imperceptible. In other words, the rater's career experience in Korean language does not affect the evaluation. Overall, collectively, the rater's grading strictness was 34.1% in the ICC, so the rater' s influence was very large. The holistic scoring was evaluated to be about /10'higher than analytical scoring. In the study on the nationality of the rater and the history of teaching Korean, the difference is about ± 1 score, which has little effect on the evaluation. The difference between the holistic and analytical scoring of native Korean raters is 11.944. Also, the difference between the two grades of non-native foreigner raters is 9.639. This means that the native Korean raters rated the analytical scoring more strictly than the non-native foreigner raters. The significance of this study is as follows. First, I tried to solve the problems of traditional statistical methods by attempting Korean writing test as Cross-Classified Multilevel Model. Second, the Cross-Classification Multilevel Model was applied to reveal the effect of differences in the scoring rigor of the raters who were not covered in the existing Korean writing test study. Third, I was able to verify the effect on non-native speaker raters which were not studied in Korean writing test. Fourth, the degree of influence of rater's teaching history on grading was revealed.
본 연구는 제2외국어로서 한국어를 배우는 학습자의 글을 평가할 때, 한국어교육 평가자의 쓰기 채점 점수가 평가자의 특성에 따라 어떠한 영향을 받는지를 교차분류 다층모형으로 밝히는 ... |
