We present a protocol to explore the relative activation sequence of phonology and semantics in visual word recognition. The results show that consistent with interactive accounts, semantic and phonological representations may be processed interactively, and higher-level linguistic representations may affect early processing.
Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on orthographic information after which, other representations, such as phonology and semantics are activated, or whether these are fully interactive and high-level semantic information affects early processing. An interference paradigm was implemented in the presented protocol of phonological and semantic judgment tasks that utilized the same precede-target pairs to explore the relative order of phonological and semantic activation. The high- and low-frequency target words were preceded with three conditions: semantically related, phonological-related (homophones), or unrelated. The results showed that the induced P200 component of low-frequency word pairs was significantly greater than high-frequency words in both the semantic and phonological tasks. In addition, both the homophones in the semantic task and the semantically related pairs in the phonological task caused reduction in N400 when compared to the the control condition, word frequency-independently. It is worth noting that for the low-frequency pairs in the phonological judgment task, the P200 released by the semantically related word pairs was significantly larger than that in the control condition. Overall, semantic processing in phonological tasks and phonological processing in semantic tasks were found in both high- and low-frequency words, suggesting that the interaction between semantics and phonology may operate in a task-independent manner. However, the specific time this interaction occurred may have been affected by the task and frequency.
The critical issue in any word recognition model is understanding the role of phonology in the process of semantic access1. For alphabetic languages, many studies consistently view phonology as playing an important role in semantic access, including English2,3,4, Hebrew5, French6, and Spanish7. In other words, written word recognition involves not only orthographic but also phonological and semantic processing. This observation in the interactive connectionist model is explained by extensions activated throughout the network, where orthography is associated with phonological and semantic representations through weighted connections8. This proliferation of activation provides the core mechanism for the visual word recognition model, which assumes that phonological and semantic representations are automatically activated in response to orthographic input9.
However, current empirical evidence supporting the hypothesis of interactive automation remains controversial. Some studies claim that the activation of phonological and semantic representations can be adjusted or prevented by task demands or attention, which implies a certain top-down influence on the high-level processes involved in word perception10,11. However, the aforementioned description has been questioned by many findings that report phonological and semantic effects in visual word recognition even though these representations are completely irrelevant to the task or cannot be directly accessed12, thereby supporting the view that semantics and phonology may be accessed automatically and forcibly during the reading process13. Therefore, there is uncertainty on whether the phonological and semantic activation in visual word recognition depends on the specific task or whether it occurs forcibly and automatically in a task-independent manner.
The answer to aforementioned question is difficult for Chinese readers. Compared to English, Chinese is a logographic script whose characters represent morphemes instead of phonemes14. At present, the role of phonology for semantic access to Chinese words remains controversial. Some studies have claimed that phonology plays an important role in semantic access to Chinese words15,16,17. Others, however, have held the opposite view18,19. After evaluating the aforementioned research for Chinese phonological processing, we found that the experimental paradigm and specific research methods differ. On the whole, it was mainly divided into two paradigms: word priming15,18,19 and violation paradigm in the sentence17,20,21. The target word is usually embedded at the end of the sentence in the violation paradigm22. In terms of language mechanism, a short two-word phrase is a more manageable unit than a complete sentence that is difficult to process23. In addition, variables that are difficult to control in a sentence, such as syntax, context, or other factors, may lead to different conclusions24. The word priming paradigm is a method commonly used to explore word recognition models, whether in alphabetic languages or Chinese. The task of this paradigm is to judge whether the target word preceded by the primes is a real word or pseudoword; that is, this paradigm usually contains only one lexical task. However, a single lexical decision task may not be the best choice to solve the problem of whether the activation of phonology and semantics depends on the task. Therefore, two different tasks may be more suitable for exploring this question.
Therefore, this research aimed to explore the role of phonology in Chinese word recognition and simultaneously attempt to determine whether the activation of phonology and semantics is in task independent. Our research includes two tasks using the interference paradigm: semantic judgment and phonological judgment. To the best of our knowledge, this is the first event-related potential (ERP) study of Chinese two-character compound recognition using this interference paradigm, and this method rarely appears in studies of alphabetic languages. Specifically, in the semantic judgment task, participants must judge whether the target word and its precedent are semantically related, while in the phonological task, they must judge whether the paired words have the same pronunciation.
The former is a semantic matching task that does not require a priori phonological processing, and the latter is a phonological judgment task that does not require a priori semantic processing. Therefore, we compared homophone pairs and unrelated control groups in the semantic judgment task to reveal whether and how phonology affects semantic processing. Similarly, we compared semantically related word pairs with unrelated control conditions in the phonological judgment task to reveal whether and how semantics affect phonological processing. In addition, the aforementioned problem was verified in high- and low-frequency words. Thus, this complementary semantic and phonological judgment task can not only reveal the importance of phonological processing in Chinese word recognition but also reveal whether and how phonology and semantics interact.
If the phonology and semantic processes are early, automatic, and interactive, the effect of phonological and semantic activation should be observed in the response time of the two tasks. For ERP, phonological and semantic processes trigger two different electrophysiological markers2,7. In addition, their time courses and their spatial distributions should be different. An early positive component (P200) should reflect phonological processing, and the typical semantic processing marker N400 should also be identified20,21. We assumed that both the phonologically related pairs in the semantic task and the semantic-related pairs in the phonological task would cause a significant decrease in N400, which would have indicated that phonological processing may lead to some degree of activation at lexical-semantic levels. In addition, we monitored whether the P200, which characterizes phonological processing, appeared in the semantic judgment task or the phonological judgment task. In the phonological judgment task, semantic-related conditions trigger the P200, which can be seen as evidence of the early influence of semantics on phonological processing.
The protocol used for this study was approved by the Institutional Review Board of Tsinghua University.
1. Stimuli construction and presentation
2. Experiment preparation and electrophysiological recording
3. EEG preprocessing
This protocol was used in a recent study to investigate the role of phonology in Chinese two-character compound recognition and to infer the word recognition model26. All stimuli used in this study were fully disclosed26. Three time windows were selected on the basis of global field power (GFP): at 100-150 ms, 160-280 ms, and 300- 500 ms for N1, P200, and N400 components, respectively26. The average amplitudes of the above two time windows were analyzed by repeated measures analysis of variance (ANOVA), and frequency (low and high), relationship type (phonologically or semantically related, or unrelated), and lateral areas (left and right hemispheres × front, middle, and posterior regions = six areas in total) or midline electrodes (Fz, Cz, Pz), which were the three within-participant factors involved. More detailed results and graphs can be found in Wang et al. (2021)26.
ERP results for the semantic judgment task
The 100-150 ms Period (N1)
For the midline electrodes, ANOVA yielded a main effect of Frequency [F(1, 23) = 9.451, P = .005, ƞ2p = .291], indicating that high-frequency pairs elicited a significantly more negative waveform than low-frequency condition. ). A similar significant main effect of Frequency was also observed at the lateral sites. In addition, for high-frequency pairs, a significant main effect of Relation Type was also observed [F(1, 23) = 8.826, P = .007, ƞ2p = .277], showing that unrelated pairs elicited a significantly more negative waveform than homophone condition. A similar significant main effect of Relation Type for high-frequency pairs was also observed at the left hemisphere.
The 160–280 ms Period (P200)
For the midline electrodes, ANOVA yielded a main effect of Frequency [F(1, 23) = 5.546, P = .027, ƞ2p = .194], indicating that low-frequency pairs elicited a significantly more positive waveform than high-frequency condition. No other significant effects or interactions were observed in the midline electrodes. Moreover, the main effect of Frequency was also found at lateral sites.
The 300–500 ms Period (N400)
In the 300-500 ms time window, ANOVA yielded a significant main effect of Relation Type [F(1, 23) = 27.783, P < .001, ƞ2p = .547] in the midline electrodes, showing that a target primed by homophones elicited significantly less negative amplitude than did the unrelated condition (see Figure 1). A similar significant main effect of Relation Type was observed at lateral sites.
Figure 1: Grand mean event-related potentials in response to target words, from representative electrodes (Fz, Cz, Pz), for homophonic and control pairs in the semantic task. This figure, taken from Wang et al. (2021)26, shows that in the semantic judgment task, homophone pairs released a smaller N400 components than irrelevant conditions regardless of high and low frequencies. Please click here to view a larger version of this figure.
ERP Results for the Homophone Judgment Task
The 100-150 ms Period (N1)
No significant effect or interaction was found in the midline electrodes or lateral sites.
The 160–280 ms Period (P200)
No significant main effect for Relation Type or Frequency (ps > .1) was observed at the anterior frontal electrodes. However, a significant interactive effect between Frequency and Relation Type was found [F(1, 23) = 7.951, P = .010, ƞ2p = .257]. Further analysis found that the influence of Relationship type was only significant under low-frequency conditions at the anterior frontal electrodes (FPz: P =.055; FP1: P =.027; FP2: P =.004; AF3: P =.060; AF4: P =.021; AF8: P =.009), indicating that in the P200 time window, the ERP signal was significantly more positive under semantically related conditions than under unrelated conditions (see Figure 2).
In addition, the analyses showed that the effect of Frequency was significant in two regions (left central: F(1, 23) = 4.506, P = .045, ƞ2p = .164 and left posterior: F(1, 23) = 10.470, P = .004, ƞ2p = .313).
Figure 2: Grand mean event-related potentials in response to target words from six anterior frontal electrodes (Fpz, Fp1, Fp2, AF3, AF4, AF8) for the semantically related and control pairs of low-frequency in the homophone task. This figure, taken from Wang et al. (2021)26, shows that in the phonological judgment task, low-frequency semantic related words released a more positive P200 component than unrelated words. Please click here to view a larger version of this figure.
The 300–500 ms Period (N400)
In the N400 time window, a significant main effect of Relation Type was found [F(1, 23) = 9.082, P = .006, ƞ2p = .283] in the midline electrodes, indicating that the target primed by semantically-related words released a significantly less negative amplitude than unrelated primes (see Figure 3). In addition, a significant main effect of Relation Type was observed at the lateral sites.
Figure 3: Grand mean event-related potentials in response to target words, from representative electrodes (Fz, Cz, Pz), for the semantically related and control pairs in the homophone task. This figure, taken from Wang et al. (2021)26, shows that in the phonological judgment task, semantically related pairs released a smaller N400 components than irrelevant conditions regardless of high and low frequencies. Please click here to view a larger version of this figure.
Experimental results and significance:
The purpose of this protocol was to infer the following: 1) whether the word recognition model is a feedforward model or an interactive model and 2) the interaction between the phonological and semantic patterns in Chinese two-character compound recognition of high and low frequency under different tasks. An interference paradigm of phonological and semantic matching task using the ERP technique was adopted. The ERP responses preceded by homophones and unrelated words to targets were compared in semantic judgment tasks that did not require a priori phonological processing to reveal when and whether phonology affects semantic processing. Similarly, for phonological judgment tasks that did not require a priori semantic processing, the ERP responses of target words triggered by semantic and unrelated words were compared to reveal whether and semantics interferes with phonological processing. Next, according to the latency of the relevant ERP components, the relative time course of semantic and phonological processing was compared, and the influence of word frequency on such processing patterns was checked.
Our results showed that the target primed by semantically related precedents in the phonological task and the targets primed by homophones in the semantic judgment task both triggered a significantly less negative N400 component than did the unrelated precedes, regardless of word frequency. Hence, for high-frequency words, the data suggested that semantic activation occurs earlier or at least no later than phonological processing during the recognition of Chinese two-character compounds in different tasks. In addition, the induced P200 component of low-frequency word pairs were significantly more positive than that of high-frequency word pairs in both semantic and phonological tasks. Other studies have also concluded that early ERP components may be sensitive to word frequency27,28. The earliest N1 and the P200 effects can also be, at least partly, due to the semantic processing of the previous word. However, for low-frequency words in the phonological judgment task, it was found that the semantically related pairs released a significantly larger P200 than the control condition. In contrast, it was found that P200 triggered by low-frequency semantically related word pairs was significantly more positive than low-frequency control conditions in phonological judgment tasks. This result does not seem to be difficult to explain for low-frequency words, as phonological activation is expected in phonological judgment tasks, but the obvious P200 component was triggered by semantic precedents, which again strengthened the hypothesis that semantic processing may not occur later than phonological processing.
The above interaction between semantics and phonology confirmed the interaction model of word recognition that proposed that the system may be fully interactive, with low-level information flowing from bottom to top to the entire lexical information and high-level information flowing from top to bottom to form early visual word processing1. In addition, the P200 caused by frequency effects in the two tasks also confirmed the speculation that higher-level linguistic information may already exert its influence during early processing. Note that the interactions found in the two tasks, regardless of the high- and low-frequency conditions, supported the automatic and possibly mandatory access to phonology and meaning during reading. However, the specific time at which this interaction occurred may have been affected by the task and frequency. For example, for low-frequency words, it was found that the interaction occurred in the P200 time window in the phonological task while in the N400 time window in the semantic judgment task. Nevertheless, for high-frequency words, the interaction was observed in the N400 time window for both semantic or phonological tasks. In conclusion, the current findings suggest the automatic interaction of semantics and phonology in a task-independent manner while considering that the interactive time and mode may be affected by tasks, frequency, etc.
Effectiveness of the method
In general, this interference paradigm can more comprehensively explore the interaction modes of phonological and semantic processing. Our experiments included phonological matching tasks that do not require priori semantic processing and semantic matching tasks that do not require priori phonological processing. In this way, the influence of semantics on phonological processing or the influence of phonology on semantic processing can be observed more clearly. Furthermore, as the phonology or semantics of the precedents and the target word need to be compared in both tasks, the phonology or semantics are forcibly activated in the two tasks. Therefore, if any interference effect occurs, it will be more obvious. The common method of exploring word recognition is a lexical decision task that includes priming conditions. Specifically, it only needs to judge whether the target word is a real word or a pseudoword. First, the semantic activation of lexical decision tasks may not be strong enough, and second, a single judgment task cannot explore the interaction mode under different tasks. Therefore, the interference paradigm of the two tasks may be more suitable for exploring the word recognition model. For the two distinct tasks of the interference paradigm, one needs to strongly activate semantic processing, and the other needs to strongly activate phonological processing, which is more conducive to exploring whether the interaction between phonology and semantics is task-independent and how to interact under different tasks.
Future applications of the technique
The present protocol was the first to use the interference paradigm to explore the semantic access to Chinese two character compound of high- and low-frequency. Currently, the two-task interference paradigm rarely appears in the studies of word recognition in alphabetic languages. Therefore, this method may provide a new opportunity for different languages characterizing by different relations between orthography, phonology, and semantics to explore word recognition models.
The authors have nothing to disclose.
This work was supported by Major Program of the National Natural Science Foundation of China (62036001).
BrainAmp DC amplifier system (Brain Products GmbH) | Brain Products, Gilching, Germany | BrainAmp S/N AMP13061964DC Input 5.6DC=150mA Operation 7mA Standby | |
Easycap (Brain Products GmbH) | Brain Products, Gilching, Germany | 62 Ag/AgCl electrodes with a configuration of the international 10–20 system of electrode |