2020

Klein, Pascal; Lichtenberger, Andreas; Küchemann, Stefan; Becker, Sebastian; Kekule, Martina; Viiri, Jouni; Baadte, Christiane; Vaterlaus, Andreas; Kuhn, Jochen

This study used eye-tracking to record the students' visual attention while taking the test of understanding graphs in kinematics (TUG-K). A total of $N=115$ upper-secondary-level students from Germany and Switzerland took the 26-item multiple-choice instrument after learning about kinematics graphs in the regular classroom. Besides choosing the correct alternative among research-based distractors, the students were required to judge their response confidence for each question. The items were presented sequentially on a computer screen equipped with a remote eye tracker, resulting in a set of approx. 3000 paired responses (accuracy and confidence) and about 40 hours of eye movement-data (approx. 500.000 fixations). The analysis of students' visual attention related to the item stems (questions) and the item options reveal that high response confidence is correlated with shorter visit duration on both elements of the items. While the students' response accuracy and their response confidence are highly correlated on the score level, $r(115)=0.63, p<0.001$, the eye-tracking measures do not sufficiently discriminate between correct and incorrect responses. However, a more fine-grained analysis of visual attention based on different answer options reveals a significant discrimination between correct and incorrect answers in terms of an interaction effect: Incorrect responses are associated with longer visit durations on strong distractors and less time spent on correct options while correct responses show the opposite trend. Outcomes of this study provide new insights into the validation of concept inventories based on students' behavioural level.