In this paper, we argue for the importance of conducting replication studies over various schools and countries when addressing topics about learning and instruction and propose educational technology to be a tool for this endeavor. We present an example of a cross-cultural replication study that makes use of educational technology in the form of a digital game-based automated feedback system. The study addresses feedback related behavior in 11-15-year-old students in US and Swedish classrooms, investigating students' choices to seek confirmatory (i.e., positive) or critical (i.e., negative) feedback, as well as their subsequent choices to revise their work based on this feedback. Comparisons of the data collected at several schools in the US and Sweden showed similar patterns of relationships among students' feedback-seeking behavior, their tendency to revise their work, and their learning outcomes in and outside the assessment environment. Overall, the findings revealed that this assessment approach seems to be generalizable from a North American to a European population. However, the findings showed both a significant difference between Sweden and the US regarding the preference for critical feedback and between different schools within each country. Thus, it is possible that the difference between countries reflects school differences rather than cultural differences.

Syftet med detta kapitel är att belysa det mångvetenskapliga forskningsfältet learning science. Utifrån en konkret studie försöker vi illustrera den centrala roll som forskningsfrågorna har samt hur fältet förhåller sig till metoder, teorier och kunskapsanspråk.

The paper presents a systematic examination of data from two early math interventions, involving 188 children aged 3,5-6,5 and their teachers. The aim is to cast light on how a digital early math game can be spontaneously extended into the physical environment by children and by teachers. Questions were: i) how the math content was extended to the physical room, ii) which elements of the game and the interventions inspired and provided affordances for the extensions iii) implications for children’s learning of early math. The analyses revealed a great variety of ways in which children and teachers – acting on their own or together – brought the game out to the physical room. Among the underlying factors for this were everyone’s experiences with the game, familiar narratives, and simple game design in terms of rules and visual features. Finally, positive influences on early math learning from the extensions were identified.

Background

Previous research shows that critical constructive feedback, that scaffolds students to improve on tasks, often remains untapped. The paper’s aim is to illuminate at what stages students provided with such feedback drop out of feedback processing.

Methods

In our model, students can drop out at any of five stages of feedback processing: (1) noticing, (2) decoding, (3) making sense, (4) acting upon, and (5) using feedback to make progress. Eye-tracking was used to measure noticing and decoding of feedback. Behavioral data-logging tracked students’ use of feedback and potential progress. Three feedback signaling conditions were experimentally compared: a pedagogical agent, an animated arrow, and no signaling (control condition).

Findings

Students dropped out at each stage and few made it past the final stage. The agent condition led to significantly less feedback neglect at the two first stages, suggesting that students who are not initially inclined to notice and read feedback text can be influenced into doing so.

Contribution

The study provides a model and method to build more fine-grained knowledge of students’ (non)processing of feedback. More knowledge on at what stages students drop out and why can inform methods to counteract drop out and scaffold more productive and fruitful responses.

Digitally animated characters are promising tools in research studying how we integrate information from speech and visual sources such as gestures because they allow specific gesture features to be manipulated in isolation. We present an approach combining motion capture and 3D-animated characters that allows us to manipulate natural individual gesture strokes for experimental purposes, for example to temporally shift and present gestures in ecologically valid sequences. We exemplify how such stimuli can be used in an experiment investigating implicit detection of speech–gesture (a) synchrony, and discuss the general applicability of the workflow for research in this domain.

This study investigated how preschool children processed and understood critical information in Magical Garden, a teachable agent-based play-&-learn game targeting early math. We analyzed 36 children’s (ages 4–6 years) real-time behavior during game-use to explore whether children: (i) processed the information meant to support number sense development; (ii) showed an understanding of the teachable agent as an entity with agency. An important methodological goal was to go beyond observable behavior and shed some light on how cognitive processing and understanding in children of such young age can be studied. First, the children played Magical Garden for three weeks to get acquainted with the game. Second, in an experimental part of the study, the children’s gaze behaviors were measured during 5 rounds of interaction with an experimental version of one of the sub-games. The analyses suggest that two of the gaze behaviors were positively correlated with the game performance measure, as hypothesized. Another result was that children looked at the teachable agent significantly more often when the teachable agent had been in charge of gameplay than when it had not. This can be interpreted as an indication that the children had an understanding of their teachable agent as an entity that, like themselves and unlike other dynamic visual elements in the game, made decisions based on own ‘knowledge’. In a broader context, the findings are important in showing the potential gains of combining log data with eye-tracking data for developing and refining AI algorithms for adaptive individual feedback and scaffolding.

This study examines the effects of teachable agents’ expressed self-efficacy on students. A total of 166 students, 10- to 11-years-old, used a teachable agent-based math game focusing on the base-ten number system. By means of data logging and questionnaires, the study compared the effects of high vs. low agent self-efficacy on the students’ in-game performance, their own math self-efficacy, and their attitude towards their agent. The study further explored the effects of matching vs. mismatching between student and agent with respect to self-efficacy. Overall, students who interacted with an agent with low self-efficacy performed better than students interacting with an agent with high self-efficacy. This was especially apparent for students who had reported low self-efficacy themselves, who performed on par with students with high self-efficacy when interacting with a digital tutee with low self-efficacy. Furthermore, students with low self-efficacy significantly increased their self-efficacy in the matched condition, i.e. when instructing a teachable agent with low self-efficacy. They also increased their self-efficacy when instructing a teachable agent with high self-efficacy, but to a smaller extent and not significantly. For students with high self-efficacy, a potential corresponding effect on a self-efficacy change due to matching may be hidden behind a ceiling effect. As a preliminary conclusion, on the basis of the results of this study, we propose that teachable agents should preferably be designed to have low self-efficacy.

For learning to take place in digital learning environments, learners need to use educational software – more or less – as intended. However, previous studies show that some school children, instead of trying to learn and master a skill, choose to systematically exploit or outsmart the system to gain progress. But what about preschoolers? The present study explores the presence of this kind of behavioral patterns among preschoolers who use a teachable agent-based play-&-learn game in early math. We analyzed behavioral data logs together with interviews and observations. We also analyzed action patterns deviating from the pedagogical design intentions in terms of non-harmful gaming, harmful gaming, and wheel-spinning. Our results reveal that even if pedagogically not intended use of the game did occur, harmful gaming was rare. Interestingly, the results also indicate an unexpected awareness in children of what it means to learn and to teach. Finally, we present a series of possible adjustments of the used software in order to decrease gaming-like behavior or strategies that signalize insufficient skills or poor learning.

In a study with 108 10- to 12-year-olds who used a digital educational game targeting history, we addressed the phenomenon of perseverance, that is, the tendency to stick with a task even when it is challenging. The educational game was designed to make all students encounter tasks they did not succeed to solve, at which point they were offered a set of choices corresponding to perseverance and non-perseverance. Methods used were behavioral log data, post-questionnaires, and an in-game questionnaire conducted by a game character, who asked the students about the reason for their choice. Overall, we found no differences between high and low-perseverance students as to their experiences of effort, difficulty, and learning, and neither in their self-reported motives for persevering – when doing so. With respect to performance, however, high-persevering students solved significantly more tasks at higher difficulty levels. Comparing high-perseverance students who tended to take a break directly after a failed test – before they continued with the same task – with those who did not take a break, we found no significant differences, indicating that taking a break is not detrimental to learning and perseverance.