> didn't numerous studies showed that any of the so called brain trainig apps don't show translation of skill to the real world? e.g. you do get better that the games the apps have, but not helping in actual real life scenarios?
Yes, this is almost always the problem. For quite a while, we thought dual n-back tests had generalized results, and so brain-training apps were basically attempting to stumble on a gamified version of that task. Unfortunately, newer studies[1] suggest that dual n-back also doesn't produce fluid intelligence or working memory improvements, so there's no proof of concept here at all.
As for this study, the game and metric were exceedingly similar, and practice effects on the metric are an obvious risk. The researchers did apply a second metric less similar to their task, a standard dot-connecting task called Trail Making Task. Unfortunately, their stated intent was to show that improved single-task attention didn't degrade focus-switching attention, so this was a substantially different task for which they hypothesized no change.
Their abstract reports a statistically significant improvement on TMT, which could be quite interesting. However, the full paper shows that Decoder had p=0.03 improvement on TMT over the active control (people who played Bingo), but the passive control (no play) was not significantly different than Decoder or the active control. This isn't regarded as a failure because they expected no change, but it means the study failed to show clear improvement in any metric which didn't precisely replicate the game's task.
You're absolutely right to be skeptical here. Practice effects are a constant problem with this sort of research, and the study seems to be confusingly short on attempts to test for that. The actual app focuses more on gamification than using any type of proven test, and the lead author's corporate connection to Peak is at best concerning. I don't think there's any sort of fraud here, but it's a field riddled with well-meaning projects that turn out to lack any general benefit, and this result has all the warning signs for another promising failure.
Yes, this is almost always the problem. For quite a while, we thought dual n-back tests had generalized results, and so brain-training apps were basically attempting to stumble on a gamified version of that task. Unfortunately, newer studies[1] suggest that dual n-back also doesn't produce fluid intelligence or working memory improvements, so there's no proof of concept here at all.
As for this study, the game and metric were exceedingly similar, and practice effects on the metric are an obvious risk. The researchers did apply a second metric less similar to their task, a standard dot-connecting task called Trail Making Task. Unfortunately, their stated intent was to show that improved single-task attention didn't degrade focus-switching attention, so this was a substantially different task for which they hypothesized no change.
Their abstract reports a statistically significant improvement on TMT, which could be quite interesting. However, the full paper shows that Decoder had p=0.03 improvement on TMT over the active control (people who played Bingo), but the passive control (no play) was not significantly different than Decoder or the active control. This isn't regarded as a failure because they expected no change, but it means the study failed to show clear improvement in any metric which didn't precisely replicate the game's task.
You're absolutely right to be skeptical here. Practice effects are a constant problem with this sort of research, and the study seems to be confusingly short on attempts to test for that. The actual app focuses more on gamification than using any type of proven test, and the lead author's corporate connection to Peak is at best concerning. I don't think there's any sort of fraud here, but it's a field riddled with well-meaning projects that turn out to lack any general benefit, and this result has all the warning signs for another promising failure.
[1] https://www.sciencedirect.com/science/article/pii/S016028961...