A cognitive training intervention in elderly population together with the assessment of the pre training cognitive abilities is presented. We show two versions of training – experimental and active control – and demonstrate their effects on the array of cognitive tests.
The efficacy of cognitive training interventions is recently highly debated. There is no consensus on what kind of training regimen is the most effective. Also, individual characteristics as predictors of training outcome are still being investigated. In this article, we show the attempt to address this issue by examining not only the impact of working memory (WM) training on cognitive effectiveness in older adults but also the influence of the initial WM capacity (WMC) on the training's outcome. We describe in detail how to perform 5 weeks of an adaptive dual n-back training with an active control group (memory quiz). We are focusing here on technical aspects of the training as well as on the initial assessment of participants' WMC. The evaluation of pre and post training performance of other cognitive dimensions was based on the results of tests of memory updating, inhibition, attention shifting, short-term memory (STM) and reasoning. We have found that the initial level of WMC predicts the efficiency of the n-back training intervention. We have also noticed the post training improvement in almost all aspects of cognitive functioning we measured, but those effects were mostly intervention independent.
In many cognitive trainings studies, the dual n-back task is used as a method of working memory (WM) training. WM is a common target of cognitive interventions because of its importance for other, higher level intellectual functions1. However, the effectiveness of such training and its potential for creating a more general improvement in cognition, has been highly debated (for meta-analysis, see2,3,4,5,6,7,14 and for reviews, see4,8,9,10,11,12,13). While some researchers claim that ''… there was no convincing evidence of the generalization of working memory training to other skills''4, others present meta-analytical data, which show highly significant effects of WM training2,3,5,6,11. The separate problem is the effectiveness of WM in the elderly population. Several WM training studies reported greater benefits in younger adults compared to older adults15,16,17,18,19,20, whereas others show that similar effects can be observed in both age groups21,22,23,24,25.
Various elements are believed to forecast the benefits of memory training26. Some of those factors appears to be potential moderators of WM training effectiveness21. Mental capacity, being described as the baseline cognitive capacity or general cognitive resource, seems to be one of the strongest choices for this position. In order to assess the role of the initial intellectual level, we put a special emphasis (the method described here) on the measurement of the cognitive capacity before applying a training regimen. It was dictated by the data showing that participants, who are characterized by higher cognitive capacity at the beginning of the training, achieved substantially better training outcomes compared to those with lower levels of initial cognitive functionning27. A similar phenomenon is observed in educational research, where it is referred to as the Matthew effect28, an observation that people with initially better skill improve even more when compared to those with preliminary lower level of ability in question.
It is thought-provoking, though, that not so many reports have been published on this topic21,29. Moreover, even substantial individual differences, especially when it comes to the elderly population, are often left unattended during data analysis and interpretation30. In the present study, we examine the impact of the initial level of working memory capacity on WM training success in the group of healthy older adults. In order to maintain every element of the training regimens as similar as possible between experimental and control groups, we employed an active control group design. Therefore, the training content (WM versus semantic memory) remained the one crucial factor determining the expected difference in the training results. Both groups performed computerized, home-based trainings. Members of the experimental group were assigned to an adaptive dual n-back training program and an active control group trained with a task based on a semantic memory quiz. New in the approach here is the emphasis on the initial evaluation of the participants' cognitive level by assessing their working memory capacity (WMC). Additionally, the method of assessing the initial WMC level we present in this article has proven to be an effective tool in distinguishing between people who will and will not be successful during subsequent working memory training. We have previously described and published results from this study44. Therefore, in this article we are focusing on a detailed description of the protocol we used.
The SWPS University of Social Sciences and Humanities Ethics Committee assessed the protocol described here. A written informed consent in accordance with the Declaration of Helsinki was obtained from every participant.
1. Participants recruitment
2. The evaluation of the Ethics Committee
3. Initial screening
4. Training group assignment
Figure 1. Study design with examples of a training tasks. Participants underwent two measurement sessions, before and after a 5 week training protocol. Please click here to view a larger version of this figure.
Figure 2. The example of suggested coding form for group assignment.
5. Initial assessment of cognitive functioning
6. Training protocols
7. Training supervision
8. Post-training assessment of cognitive functions
Training-related effects
85 subjects participated in the study (29 were male) and they were on average 66.7 years old. Due to technical problems, data from one participant in the n-back training group was not recorded. Finally, the data from 43 participants in n-back training group and 42 in Quiz training group were analyzed. Multivariate analysis of variance (MANOVA) with repeated measures was used to analyze training specific effects for both training groups over time (pre-, post-training). The results of each cognitive test were dependent variables (Table 1), and training group and measurement points (pre- versus post-training) were independent variables. These results are presented in Table 2.
The results of the analysis indicated a statistically significant post-training improvement in the syllogisms task: (F(1,83)=31,22, p<0.001, ηp2=0.27) and attention switching task: (F(1,83)= 5.79, p=0.02, ηp2=0.07). A significant training group effect was observed for memory SPAN task (F(1,83)=7.72, p=0.01, ηp2=0.09) and OSPAN task (F(1,83)=13.01, p=0.01, ηp2=0.14). None of the interaction effects (time x training group) has proven to be statistically significant. However, we found significant within group effects for some analysis. In the OSPAN task, the n-back training group improved their results in second session), while for the quiz group, both performances were similar. This effect needs to be interpreted in reference to the fact that the quiz and the n-back group differed in the first measurement. Thus, results of the n-back group where initial OSPAN performance was higher improved, while the control group did not. The performance in Sternberg's and a go/no-go task was not related to a training group assignment or the time of measurement.
Overall, the results show that participants' cognitive performance was improved in the post-training execution of attention and higher cognitive functions (reasoning) engaging tests, regardless of the group affiliation.
N-back training | Quiz training | ||||||||
session | N | Mean | Std. | Std. | N | Mean | Std. | Std. | |
Err | Dev. | Err | Dev. | ||||||
OSPAN | 1 | 42 | 15.31 | 1.64 | 10.62 | 40 | 9.07 | 1.77 | 11.22 |
task | 2 | 43 | 20.74 | 2.48 | 16.3 | 40 | 10 | 1.81 | 11.45 |
Syllogisms task | 1 | 43 | 0.59 | 0.03 | 0.2 | 42 | 0.58 | 0.03 | 0.21 |
2 | 43 | 0.67 | 0.03 | 0.21 | 42 | 0.69 | 0.03 | 0.19 | |
Memory SPAN task | 1 | 42 | 0.37 | 0.03 | 0.16 | 42 | 0.2 | 0.02 | 0.16 |
2 | 41 | 0.4 | 0.03 | 0.18 | 42 | 0.22 | 0.03 | 0.18 | |
Go/no-go task | 1 | 42 | 0.14 | 0.05 | 0.33 | 42 | 0.16 | 0.03 | 0.01 |
2 | 42 | 0.17 | 0.03 | 0.18 | 42 | 0.04 | 0.05 | 0.12 | |
Sternberg’s task | 1 | 43 | 0.93 | 0.02 | 0.11 | 42 | 0.9 | 0.02 | 0.15 |
2 | 43 | 0.94 | 0.01 | 0.05 | 42 | 0.93 | 0.01 | 0.07 | |
Attention switching task | 1 | 42 | 0.49 | 0.04 | 0.28 | 41 | 0.52 | 0.05 | 0.3 |
2 | 42 | 0.41 | 0.04 | 0.23 | 42 | 0.46 | 0.04 | 0.25 |
Table 1. Descriptive statistics for the cognitive tasks' results.
Pre- to post-training effect | Training group effect | Interaction effect (time x training group) |
||||||||
F (1,83) | ηp2 | p | F (1,83) | ηp2 | p | F (1,83) | ηp2 | p | significant within-group effects: | |
OSPAN Task | 3.67 | 0.04 | 0.06 | 13.01* | 0.14 | 0.01 | 1.49 | 0.19 | 0.22 | Nback (T1 vs.T2): 5,00* |
Syllogisms Task | 31,22* | 0.27 | 0 | 0.01 | 0 | 0.95 | 0.35 | 0.01 | 0.56 | |
Memory SPAN Task | 3.13 | 0.04 | 0.08 | 7,72* | 0.09 | 0.01 | 0.04 | < .001 | 0.85 | T1 (N-back vs. Quiz): 0,09* |
T2 (N-back vs. Quiz): 0,10* | ||||||||||
Sternberg’s Task | 3.56 | 0.04 | 0.06 | 0.78 | 0.01 | 0.38 | 0.62 | 0.01 | 0.43 | |
Attention Switching Task | 5,79* | 0.07 | 0.02 | 0.75 | 0.01 | 0.39 | 0.02 | < .001 | 0.87 | Nback (T1 vs.T2): -0,08 |
Go/no-go Task | 0.01 | < .001 | 0.93 | 0.21 | 0.01 | 0.65 | 2.82 | 0.03 | 0.09 | T1 (N-back vs. Quiz): -0,01 |
T2 (N-back vs. Quiz): -0,02 | ||||||||||
* statistically significant effect (p < .05) | ||||||||||
T1 vs. T2 - difference in means between 1st and 2nd session; | ||||||||||
N-back vs. Quiz – difference in means between training groups; |
Table 2. Outcome measures: main and interaction effects from MANOVA with training type (n-back vs. Quiz) and time (pre vs post training) as factors.
WMC as a predictor of WM training effectiveness
In a subsequent analysis, performed on the n-back training group only, we used a more refined method – multilevel modeling (MLM) – to observe the learning process during the experimental training. The hierarchical structure of the data was accommodate to the model: at level 1 – repeated measurements, nested within participants (level 2)34. The MLM dataset consisted of 1,050 observations gathered from 42 participants from experimental group within each of 25 training sessions. The model provided for both fixed and random effects: the regression intercept and slope for the average person, and between-subject variability around the average. In the Model 1, the change in the number of points scored in the n-back task over time (number of the training sessions) was modeled. The time variable was centered at 1st day of the intervention. Compared to Model 1, Model 2 added on predicting and moderating effects of a baseline OSPAN score (between-subjects predictor – level 2) on within-subject variability (level 1). Those predictors were tested independently to avoid multicollinearity. In all models, a linear and quadratic effects for the slope were tested, however the quadratic one was subsequently removed because its fixed effects and variance components were not significant. The restricted maximum likelihood served as the estimator. -2 Restricted log likelihood ratio (-2LL) and the Akaike Information Criterion (AIC) were used to assess the goodness of fit for all models. Given the common proximal autocorrelation in the daily data35 we decided to base on a first-order autoregressive [AR(1)] covariance structure.
MLM results showed that OSPAN scores from the pre-training measurement were a significant predictor of the first n-back outcome from the 1 session. Baseline OSPAN level turned out to be a moderator of the whole training course (Table 2). When compared, groups of participants with high or low OSPAN points had similar N-level at the first training session: approx. 2.00 units on a 1+∞ scale (low OSPAN = 1.93; high OSPAN = 2.31). A significant difference manifested in the post-training measurement, when the participants with low initial OSPAN results achieved a .01 unit increase in n-back task, whereas those with the high initial OSPAN scores recorded a .04 point improvement. The observed result clearly indicates the existence of a positive association between the initial OSPAN level and training effectiveness. The n-back scores in the 1st session and a learning curve of a training are higher and stepper for the participants with initially higher OSPAN result (p < .001).
Time – linear | .031 | (.005)*** | .016 | (.007)* | |
(centered at 1st day) | |||||
Initial OSPAN score | — | — | .038 | (.183)* | |
~ (high / low) | |||||
Time × Initial OSPAN score | — | — | .026 | (.008)** | |
Random effects ([co-]variances) |
|||||
Level 2 (between-person) | |||||
Intercept | .285 | (.073)*** | .286 | (.075)*** | |
Time – linear | .001 | (.001)*** | .001 | (.001)*** | |
Intercept and time | .006 | (.003)* | .004 | (.002) ° | |
Level 1 (within-person) | |||||
Residual | .151 | (.009)*** | .149 | (.009)*** | |
Autocorrelation | .339 | (.039)*** | .328 | (.040)*** | |
Model fit | |||||
−2 log likelihood (χ2) | 1069.32 | 1046.37 | |||
Akaike’s Information Criterion (AIC) |
1083.32 | 1056.37 | |||
Results from multilevel modeling. Unstandardized regression coefficients with standard errors in parenthesis. | |||||
°p=0.1, *p<.05, **p<.01, ***p<.001; All p-values are two-tailed | |||||
~ the predictor is dichotomous |
Table 3. Multilevel analysis of the training data (n-back task performance as a dependent variable). Models with training sessions (time) only (MODEL 1) and training sessions plus initial working memory capacity level as a predictor and a moderator, respectively (MODEL 2).
In the study presented here, we have investigated whether older adults could benefit from working memory training and if it is connected to the initial level of their basic cognition. We used an n-back task as an experimental intervention and working memory capacity (measured with the OSPAN task) was the method to probe participants' initial level of intellectual functioning. We had two critical steps in the protocol. The first and most important was the assessment of the initial WM level. The second was the careful matching of the two training regimens in every possible way but the "cognitive content" (i.e., working memory versus semantic memory). By introducing the assessment of the participants' cognitive level at the beginning of the study, we were able to show how important it is to have a good estimate of it at the start of the intervention. It was the most important predictor of the cognitive training's effectiveness. We suspect that in most intervention studies, researchers assess, in one way or another, the initial cognitive level of participants. To obtain such information it is possible to use the results from the first measurement of a trained cognitive task as a predictor of the cognitive training effectiveness. As expected, the N level of n-back task fluctuated substantially through training sessions. What was even more interesting, individuals with higher maximal N achieved in the first training session improved faster than the rest of the group in the following sessions. That implies that the variability in performance between participants, noticed at the beginning of the study, only increased with time and training. To explore this effect deeper we conducted further analysis. The results showed the preliminary score in OSPAN task (WMC) to be a strong predictor of the improvement gained during the training course (in the dual n-back task). Participants characterized by higher initial WMC performed better in the training from the very first day and had stepper learning curve in comparison to seniors with WMC below the average of the sample. We are not the first to report such effect. Foster et al. (2017) described similar results29. They proved the existence of the correlation between the initial WM level and the performance of memory span training. This result is consistent not only with the ones here, but also with research on the so-called Matthew effect in WM training interventions, in which participants with initially higher skills profit more from training and score better in both: trained and untrained, tasks21,36,37,38,39. All this strengthens the conclusion that someone's ability to gain from WM training depends heavily on the initial intellectual level.
Regarding the regimens similarity, we applied the Mill's method of one difference40: when someone observes one situation that leads to a given effect, and another that does not result in the same way, and the only difference between these situations is a presence of a specific factor only in the first situation (here, the difference in the cognitive layer), there is the solid foundation to assume that it is the factor in question that caused the observed effect. We tried to match the training regimens in terms of motivation, superficial similarities (same amount of training sessions, similar feedback, etc.). It is worth noting that the first idea was to use the same task (n-back) but in its easiest form, where the N level is fixed to 1. It quickly becomes obvious that it was a wrong path as the participants (in pilot studies) not only reported weariness but also were dropping off the control condition at a much higher rate than from the experimental (with adaptive level of difficulty). This resulted in a different approach. After several pretests we decided for a "different function" approach (WM versus semantic memory) instead of having the same function in both conditions just with different intensity (fixed level of WM versus adaptive level of WM). One potential problem with such approach is that we can create a control condition, which is more attractive than the experimental condition. And, if motivation to engage is a crucial factor in cognitive trainings, we can have null results because of that decision.
It is worth noticing that there is a substantial change in a way we look now at results from cognitive intervention studies. For example, Reddick et al. suggest that positive effects observed in WM training groups when compared to control groups are due to decrease present in control group and not improvement of performance in experimental groups41. When we think about elderly population, even such output – maintaining of the initial cognitive level – could be a desirable outcome. But, surprisingly, in the study we did not observe a post-training reduction of performance in the control group, except for the go/no-go task. It might be, again, interpreted as evidence that even a simple memory quiz, if it is attractive and encourages participants to engage in some cognitive activity, could produce beneficial effects. What is also important, all of the participants (regardless of group assignment) volunteered for the study and some correlational studies have shown that voluntary work might be a protective factor against cognitive aging42,43. One of limitations of the study is that we do not have the representative population of older people. Instead, the elderly taking part in the study were probably more motivated and more proactive than seniors who, for example, do not leave their homes. However, the level of education and economic status (indirectly controlled – as an occupational activity that generate income) were measured in the study and the analysis showed that these were not factors affecting training progress. It can be also argued that improvement observed in both interventions is the result of mere test-retest effects. Due to the fact that there was no passive control group included in the study, this matter cannot be settled down in this study. It is therefore advisable to include another group in the subsequent tests – passive control. The most important message from the study is that the findings suggest that post-training gains are within reach of older adults, especially those characterized by a good overall cognitive functioning. What we wanted to delineate in this article was the way we were introducing and maintaining the participants in a training regimen. The most important thing in this study was to keep all features of the intervention exactly the same between the two groups apart from one thing – the cognitive function involved undergoing practice. As we did not observe substantial differences between the effectiveness of the training protocols, but the improvement was visible in both groups, it seems valid to conclude that any cognitive engagement can be beneficial for elderly people. As the main result refers to the initial level of cognitive functioning, we strongly recommend including initial measures of the trained function and verifying it as a possible predictor (or at least co-factor) of training effectiveness.
The authors have nothing to disclose.
Described results are obtained from the project supported by the National Science Centre in Poland under grant no. 2014/13/B/HS6/03155.
GEx | n/a | authorial online platform: used for N-back training, Quiz |
|
IBM SPSS Statistics 26.0 | IBM Corporation | SPSS software was used to compute statistical analysis. | |
Inquisit version 4.0.8.0 | Millisecond Software | software: tool for designing and administering experiments used for: The Sternberg Task, The Linear Syllogism Task and presenting the instructions for baseline EEG recording |
|
MATLAB R2018b | The MathWorks, Inc | MATLAB software was used to compute statistics and to export databases and visualisation of the results | |
PsychoPy version 2 v.1.83.04 | Jonathan Peirce; supported by University of Nottingham | open-source software used for: Go/no Go Task, The Switching Task, Running Memory Span Taskckage based on Python |
|
Sublime Text (version 2.0.2) | n/a | open-source software: HTML editor used for: online OSPAN Task |