We present a protocol to examine the use of morphological cues during real-time sentence comprehension by children with autism.
Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.
Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information1,2,3,4,5,6,7,8,9,10,11. Prior research has found that young typically developing (TD) children incrementally compute the meaning of a sentence using both linguistic and non-linguistic cues12,13,14,15,16,17,18,19. However, there is currently a paucity of research exploring how preschool children with autism understand a sentence using different types of cues. The mechanisms underlying their sentence comprehension remains largely unclear.
It is generally acknowledged that there is enormous variability in the language abilities of children with autism, especially in their expressive language; for example, some children with autism have relatively good structural language, some exhibit deficits in both lexical and grammatical domains, some demonstrate impaired grammar, and some never acquire functional spoken language20,21,22,23,24,25. In addition, prior research seems to suggest that their receptive language is relatively more impaired than their expressive language26,27,28,29. Most research that has assessed sentence comprehension abilities of children with autism have used offline tasks (e.g., standardized tests, caregiver reports), and the findings suggest that their sentence comprehension abilities might be particularly impaired30,31,32,33,34,35,36,37. However, it has been pointed out that poor comprehension abilities are more likely related to these children's overall lack of social responsiveness than to language processing deficits38,39. Note that these offline tasks used in previous research often require high response demands or interactions with the experimenters, which might pose particular difficulties for children with autism, because they often exhibit various challenging behaviors or symptoms. As a result, this may interact with the high task and communication demands and mask their comprehension abilities [for an overview of methods for assessing receptive language in children with autism, see Kasari et al. (2013)27 and Plesa-Skwerer et al. (2016)29]. Thus, experimental paradigms that can better control these confounding factors are required to further understand the nature of sentence-processing mechanisms in autism.
In the current study, we present an eye-tracking paradigm that can directly and effectively assess sentence comprehension abilities of children with autism. Compared to offline tasks, eye-tracking is a more sensitive testing paradigm to demonstrate children's comprehension abilities. It is sensitive to the time course of the comprehension process and requires no explicit motor or language responses from the participant, making it a promising method to study younger children and minimally verbal children with autism. In addition, we record eye movements as automatic responses to linguistic input instead of measuring eye movements that accompany conscious responses to linguistic input.
This study has been approved by the Ethics Committee of the School of Medicine at Tsinghua University. Informed consent has been obtained from all individual participants included in the study.
1. Participant Screening and Study Preparation
2. Warm-up Session
3. Conditions and Experimental Design
4. Experimental Procedure
5. Data Treatment and Analysis
The present study uses minimal pairs as in Examples 1a and 1b to investigate if and how fast children with autism can use event information encoded in two morphological markers during real-time sentence comprehension. It was predicted that if they are able to rapidly and effectively use event information in the two markers during real-time sentence comprehension, then they should look more at the BA-target event when hearing BA than when hearing BEI. Also, they should fixate more on the BEI-target event after listening to BEI than after listening to BA.
The comparison between 5-year-olds with autism and their age-matched TD peers is presented in the representative results. Figure 2 shows the average fixation proportions of the TD 5-year-olds on the BA-target event (Panel A) and BEI-target event (Panel B) in the two conditions. Figure 3 summarizes the average fixation proportions of the 5-year-olds with autism.
The figures show that the autism group displayed eye movement patterns similar to the age-matched TD group. Both groups exhibited more fixations on the BA-target event when hearing BA than when hearing BEI, occurring after onset of the object NP and before onset of the adverb. To be specific, the effect occurred in the TD group during the window between 1400 and 1600 ms (Figure 2), whereas the effect occurred in the autism group during the window between 1800 and 2000 ms (Figure 3). By contrast, an opposite eye movement pattern was found in the BEI-target event for both groups: more fixations on the BEI-target event were observed when hearing BEI than when hearing BA, again occurring after onset of the object NP and prior to onset of the adverb.
Fixation proportions were then transformed using the empirical logit formula49: probability = ln[(y+0.5)/(n-y+0.5)], where y is the number of fixations on the areas of interest during a particular temporal bin and n is the total number of fixations in that temporal bin. Linear mixed-effects models were then fitted to the transformed data. Statistical models were computed for the two groups separately based on their fixations in the two interest areas in the critical time windows, where time and marker type (BA versus BEI) were treated as fixed effects. Random intercepts and slopes were included for both participants and items50. The fitting process was conducted via functions lmer from package lme4 (v1.1-12)51 of the R (v3.2.5) software environment52. A Wald test was then used to compute p–values for each fixed effect.
The model results for the TD 5-year-olds in the two interest areas: in the BA-target event area, hearing BA caused the TD children to look significantly at this event more than when hearing BEI (β= 0.54, p < .001). In addition, there was a significant interaction between marker type and time (β= 0.33, p < .001), indicating that the probability of fixating on the BA-target event increased over time after the onset of BA. However, the TD children exhibited an opposite eye movement pattern in the BEI-target event area. Hearing BEI triggered more fixations on the BEI-target event than hearing BA (β= -0.60, p < .001). Again, there was a significant interaction between marker type and time (β= -0.21, p < .001), suggesting that the TD group's tendency to look at the BEI-target event declined over time after the onset of BA.
The model results for the 5-year-olds with autism in the two interest areas: the autism group showed similar eye movement patterns. Hearing BA triggered more fixations on the BA-target event than hearing BEI (β= 0.50, p < .001). Hearing BEI triggered more looks at the BEI-target event than hearing BA (β= -0.54, p < .001). Like the TD group, the autism group exhibited significant interactions in both interest areas. In both the BA-target and the BEI-target event areas, the children with autism displayed a significant interaction between marker type and time (β= 0.15, p < .01 in the BA-target event area; β= -0.16, p < .01 in the BEI-target event area).
Overall, the eye patterns exhibited by the 5-year-olds with autism provide evidence that they were able to use the event information encoded in two morphological markers rapidly and effectively during real-time sentence comprehension. The results show that the recorded eye movements as automatic responses to the linguistic input are sensitive measures of sentence comprehension abilities in both TD children and children with autism.
Figure 1: Example visual image. (A) Indicates a BA-target event. (B) Represents a BEI-target event. This figure has been reprinted with permission from Zhou and Ma (2018)19. Please click here to view a larger version of this figure.
Figure 2: Average fixation proportions from marker onset in both conditions in TD 5-year-olds. (A) Shows the fixation proportions on the BA-target event. (B) Illustrates the fixation proportions on the BEI-target event. Please click here to view a larger version of this figure.
Figure 3. Average fixation proportions from marker onset in both conditions in 5-year-olds with autism. (A) Shows the fixation proportions on the BA-target event. (B) Illustrates the fixation proportions on the BEI-target event. Please click here to view a larger version of this figure.
In the current study, we present an eye-tracking paradigm that can directly and effectively assess the sentence comprehension abilities of children with autism. We found that 5-year-old children with autism, like their age-matched TD peers, exhibited eye gaze patterns that reflect effective and rapid use of linguistic cues during real-time sentence comprehension.
The findings provide evidence that eye-tracking (in particular, the visual world paradigm) is a sensitive measure of real-time sentence comprehension in children with autism. Compared to offline methods, the paradigm has several advantages. First, it is sensitive to the time course of sentence comprehension. Second, it minimizes the task and communication demands involved, thus is a better-suited method that can be used with children exhibiting challenging behavioral features. Third, it simply records eye movements as automatic responses to linguistic input without asking participants to provide conscious judgments about the input, significantly reducing the computational burden of the participants.
The visual world paradigm is based on a linking assumption that eye movements in the visual world are synchronized to the real-time processing of concurrent linguistic stimuli. Thus, an effective language comprehension study using the visual world paradigm requires a close mapping between eye gaze patterns in the visual world and referential processing of the spoken language. To ensure a close mapping between the two, it is important to first design the visual stimuli in a way so that eye movements in the visual images reflect only the processes underlying comprehension of the spoken language, and that other factors that may affect participants' eye movements are well-controlled. Second, it is important to time-lock participants' eye movements to the onset of a critical linguistic marker in the spoken language, and to make sure that each element and boundaries of the spoken language elements can be clearly identified for later analyses.
The visual world paradigm has been used successfully to test TD children's language abilities. The present study explored the potential of conducting visual world studies on language comprehension in preschool children with autism. As discussed, these findings provide evidence for the validity and sensitivity of the paradigm in testing linguistic knowledge in children with autism. The findings also invite us to rethink questions surrounding the language comprehension abilities of children with autism. As discussed, previous research seems to suggest that the sentence comprehension abilities of children with autism might be severely impaired; however, as noted by Kasari et al.27 and Plesa-Skwerer et al.29, it is often difficult to evaluate the comprehension abilities of children with autism using traditional methods like standardized tests or other off-line tasks, because these tasks require high response demands or interactions with the experimenters; as a result, this might pose particular difficulties for children with autism. Using the visual world paradigm, the present study shows for the first time that when minimal task and communication demands are involved, young children with autism are able to use linguistic cues effectively and rapidly during real-time sentence comprehension. Their sentence comprehension abilities are far better than has been suggested by previous research. The findings also provide evidence that poor comprehension performance of children with autism in past research is perhaps due to a lack of social responsiveness and the high task and communication demands involved in these traditional tasks.
The visual world paradigm can be systematically applied to establish eye gaze patterns associated with language processing in autism, which will help us better understand the nature of sentence processing mechanisms in autism as well as help to identify early clinical markers for autism.
The authors have nothing to disclose.
This work was funded by the National Social Science Foundation of China [16BYY076] to Peng Zhou and the Science Foundation of Beijing Language and Cultural University under the Fundamental Research Funds for the Central Universities [15YJ050003]. The authors are grateful to the children, parents, and teachers at the Enqi Autism Platform and Taolifangyuan Kindergarten in Beijing, China, for their support in running the study.
EyeLink 1000 plus eye tracker | SR Research Ltd. | The EyeLink 1000 plus allows remote eye tracking, without a head support. The eye tracker provides information about the participant’s point of gaze at a sampling rate of 500 Hz, and it has accuracy of 0.5 degrees of visual angle. |