基于脑机接口与音乐游戏的空间认知训练系统设计与实验验证研究

Design and validation of a spatial cognition training system based on brain–computer interface and music games

  • 摘要: 为提升空间认知训练的趣味性、沉浸性与评估客观性,设计并验证了一种基于脑机接口与音乐游戏的空间认知训练系统. 系统以Unity 3D为开发平台,结合OpenBCI脑电采集设备,构建了训练任务、测试任务与行为–脑电同步采集的一体化实验平台. 其中,训练任务采用基于3D空间音频的声源定位与目标匹配机制,测试任务采用基于空间听觉线索的目标搜索与交互机制,并通过渐进式难度调节提升训练挑战性. 实验共招募32名健康在校生,随机分为实验组和对照组,每组16名,开展为期35天的对照研究. 结果表明:两组受试者训练前在年龄、性别比例及空间认知量表得分方面均无显著差异. 经Bonferroni校正后,实验组吉尔福德–齐默尔曼空间定向测试(Guilford–Zimmerman spatial orientation test, GZSOT)得分显著提高,透视空间定向测试(Perspective taking spatial orientation test, PTSOT)得分呈改善趋势,Corsi块敲击任务(Corsi block-tapping task, CBTT)得分虽有提升但未达到显著水平. 实验组五次测试任务的完成时间持续下降,且第1次与后续各次比较在校正后仍具有显著差异. 变化量比较显示,实验组三项量表平均改善幅度均高于对照组,其中PTSOT呈较明显改善趋势. 脑电耦合分析和受试者级交叉验证下的支持向量机(Support vector machine, SVM)分类结果提示,训练前后脑电图(Electroencephalogram, EEG)特征存在一定可分性,可作为辅助神经证据. 该系统在健康青年群体中具有促进空间定向和空间操作表现改善的潜力,并为基于脑电客观评估的认知训练系统设计提供参考.

     

    Abstract: Spatial cognition is a fundamental component of human cognitive ability and plays an important role in navigation, object localization, route planning, spatial orientation, spatial transformation, spatial memory, and everyday problem solving. Conventional spatial cognition training methods are often constrained by low immersion, weak motivation, monotonous interaction, and insufficient objectivity in outcome assessment. To address these limitations, this study designed and validated a spatial cognition training system that integrates brain–computer interface technology with music-based game interaction. The proposed system was developed using Unity 3D and incorporated an OpenBCI-based electroencephalography (EEG) acquisition device, forming an integrated experimental platform that combines spatial cognition training, spatial cognition testing, behavioral recording, and synchronized EEG data acquisition. The system was intended to enhance user engagement during training and provide a multimodal evaluation framework based on both behavioral performance and EEG signals. In the training task, a beach-based virtual scene was created to improve ecological appeal and immersion. Spatialized 3D audio was used to simulate the position and direction of hidden target objects. Participants were required to localize the sound source based on auditory cues, navigate toward the target, identify it upon visual emergence, and complete a color-matching task between the located object and its corresponding destination item. Instead of using traditional score-based feedback, the system adopted a music performance reinforcement strategy, where successful completion of matching tasks unlocked virtual characters and instrument performance segments. This design integrated auditory guidance, visual interaction, action feedback, and motivational reinforcement. In addition, a progressive difficulty adjustment mechanism was implemented by changing the effective display distance between participants and target objects such that task demand gradually increased alongside training progress. To evaluate the effectiveness of the proposed system, a 35-day controlled experiment was conducted. A total of 32 healthy university students were recruited and randomly assigned to an experimental or control group, with 16 participants in each group. The experimental group received spatial cognition training through the proposed music-based game system, whereas the control group performed a non-spatial casual game task to control for general game exposure and training duration. Before and after the intervention, all participants completed three spatial cognition assessments: perspective taking spatial orientation test, Guilford–Zimmerman spatial orientation test, and Corsi block-tapping task. Repeated spatial cognition testing tasks were also conducted under consistent difficulty settings. During training and testing, behavioral data and EEG signals were synchronously collected. Behavioral performance was mainly measured by task completion time, while EEG analysis included coupling-based brain network characterization and support vector machine (SVM) classification based on time-domain statistical features. For the classification analysis, subject-level cross-validation was adopted to reduce the risk of data leakage caused by samples from the same participant appearing in both the training and testing sets. The results showed that the two groups were well balanced at baseline, with no significant differences in age, sex ratio, or initial spatial cognition scale scores. After Bonferroni correction, the experimental group showed a significant improvement in Guilford–Zimmerman spatial orientation test scores, whereas perspective taking spatial orientation test scores showed an improving trend and Corsi block-tapping task scores increased without reaching statistical significance. The completion time of the experimental group decreased continuously across five testing sessions, and the differences between the first and subsequent tests remained statistically significant after multiple-comparison correction. The comparison of pre–post change scores further revealed that the experimental group had greater mean improvements than the control group in all three spatial cognition measures, among which the perspective taking spatial orientation test displayed a more evident improvement trend. EEG coupling analysis indicated that the training process was accompanied by changes in inter-channel coupling patterns across multiple frequency bands. SVM classification under subject-level cross-validation suggested that EEG features showed a certain degree of separability before and after training, which may serve as auxiliary neural evidence in addition to behavioral and scale-based results, rather than direct proof of spatial cognitive improvement. Overall, the proposed system provides an integrated framework for multisensory task design, synchronized behavioral and neural data acquisition, and objective training evaluation. The findings suggest that the system has the potential to improve spatial orientation and spatial task performance in healthy young adults. Future studies should further optimize active control conditions, include larger and more diverse samples, and introduce frequency-domain, time-frequency, and feature-importance analyses to improve the interpretability and generalizability of the system. This study provides a useful reference for the development of EEG-assisted, game-based, and scientifically assessable cognitive training systems.

     

/

返回文章
返回