基于神经生理测量探究具身交互对创意思维的影响

doi:10.3901/JME.2025.19.237

摘要/Abstract

摘要： 基于多模态技术的具身交互使得设计师以“身心一体”的方式进行设计创作，这种方式对设计过程中的创意思维所带来的影响有待深入探索。通过对比传统“键盘+鼠标”交互和具身交互两种方式下设计师的神经生理测量数据，研究具身交互对设计师创意思维的影响机制。34名设计师分别利用传统和具身交互方式在计算机辅助设计系统中完成了立体构成创意设计任务，同时采集了被试在两种交互方式下的脑电、皮电、心率、工作负荷量表和口语报告等认知数据，并且利用同感评估技术对设计作品进行了创意性评估。数据分析结果显示：①具身交互方式下，设计师顶区和枕区的alpha频段功率显著低于传统交互，说明身体和虚拟设计空间之间的联结增强了设计师的视觉加工，促进创意思维的产生；②具身交互下设计师感觉运动区alpha和beta频段的抑制，说明身体动作、姿势和日常经验的一致性加快了设计师对动作隐喻的理解，对创意思维具有激发作用；③具身交互下设计师的皮电活动和心率值显著增高，说明操作过程的新颖性和灵活性为设计师带来情绪上的唤醒，有利于创意思维的产生；④具身交互方式下设计师需要较多训练，工作负荷加重，可能对创意思维产生不利影响，表明未来多模态技术的成熟度有待进一步提高。

关键词: 神经生理测量, 具身交互, 创意思维, 多模态技术, 脑电图

Abstract: Embodied interaction based on multimodal technology allows designers to create in an "all-in-one" manner, and the impact of this interaction on creative thinking in the design process needs to be explored in depth. The neurophysiological measures of designers in traditional “keyboard + mouse” interaction and embodied interaction are compared to investigate the influence of embodied interaction on designers' creative thinking. Thirty-four designers complete a three-dimensional composition design task in a computer-aided design system using the traditional and embodied interaction respectively. Cognitive data such as electroencephalograms, heart rates, workload scales and verbal reports are collected simultaneously under both interaction modes. In addition, the design solutions are assessed for creativity using the Consensus Assessment Technique. The results of the data analysis show that, (i) the alpha band power of the designer’s parietal and occipital regions under embodied interaction is significantly lower than that of traditional interaction, indicating that the connection between the body and the virtual design space enhances the designer's visual processing and facilitates the generation of creative thinking; and (ii) the suppression of alpha and beta bands of the designer's sensorimotor region under embodied interaction suggests that the consistency of the designer’s body movements, postures, and daily experience accelerates designers' understanding of movement metaphors, which has a stimulating effect on creative thinking; (iii) the significant increase in the designer’s electrodermal activity and heart rate values under embodied interaction suggests that the novelty and flexibility of the operation process can bring about emotional arousal for designers, which is conducive to the generation of creative thinking; (iv) designers need more training and have a heavier workload in embodied interaction, which may have a negative impact on creative thinking. This suggests that the maturity of multimodal technology needs to be further improved in the future.

Key words: embodied interaction, creative thinking, neurophysiological measurements, multimodal technology, electroenceph- alogram

中图分类号:

TB472
TP391

王文娟, 卢兆麟, 郝佳, 杨伟铨, 牛红伟, 左孝林. 基于神经生理测量探究具身交互对创意思维的影响[J]. 机械工程学报, 2025, 61(19): 237-248.

WANG Wenjuan, LU Zhaolin, HAO Jia, YANG Weiquan, NIU Hongwei, ZUO Xiaolin. Exploring the Effects of Embodied Interaction on Creative Thinking Based on Neurophysiological Measures[J]. Journal of Mechanical Engineering, 2025, 61(19): 237-248.

参考文献

[1] 程锦，叶虎强，谭建荣，等. 三维CAD技术研究进展及其发展趋势综述[J]. 机械工程学报，2023，59(23)：158-185. CHENG Jin，YE Huqiang，TAN Jianrong，et al. Review of research progress and development trends of 3D CAD technology[J]. Journal of Mechanical Engineering，2023，59(23)：158-185.
[2] 龙辉，郝佳，牛红伟，等. 基于情境的CAD模型手势操控技术[J]. 机械工程学报，2022，58(10)：374-382. LONG Hui，HAO Jia，NIU Hongwei，et al. Gesture control technology of CAD model based on situation model[J]. Journal of Mechanical Engineering，2022，58(10)：374.
[3] BORGIANNI Y，MACCIONI L，FIORINESCHI L，et al. Forms of stimuli and their effects on idea generation in terms of creativity metrics and non-obviousness[J]. International Journal of Design Creativity and Innovation，2020，8(3)：147-164.
[4] LIIKKANEN L，PERTTULA M K. Contextual cueing and verbal stimuli in design idea generation[C]//Design computing and cognition’06. Springer Netherlands，2006: 619-631.
[5] 宋璐. 姿势开放性对创造性思维的隐喻效应及机制[D]. 天津：天津师范大学，2022. SONG Lu. The metaphor effect of postural openness on creative thinking and its mechanism[D]. Tianjin：Tianjin Normal University，2022.
[6] 牛红伟，郝佳，王璐，等. 功能词对设计思维的启发效用研究[J]. 北京理工大学学报，2022，42(2)：168-176. NIU Hongwei，HAO Jia，WANG Lu，et al. Research on the inspiring effects of design thinking with functional terms[J]. Transactions of Beijing Institute of Technology, 2022，42(2)：168-176.
[7] 卢兆麟，宋浩. 基于视觉动力-灰色关联分析的车身造型形态进化机制研究[J]. 北京理工大学学报，2023，43(9)：903-911. LU Zhaolin，SONG Hao. Evolution mechanism of vehicle styling based on visual dynamics-gray correlation analysis[J]. Transactions of Beijing Institute of Technology，2023，43(9)：903-911.
[8] DOURISH P. Where the Action Is：The Foundations of Embodied Interaction[M]. Cambridge：MIT press，2001.
[9] 姚争为，杨琦，潘志庚，等. 具身交互与全身交互的比较[J]. 计算机辅助设计与图形学学报，2018，30(12)：2366-2376. YAO Zhengwei，YANG Qi，PAN Zhigeng，et al. Comparison of embodied interaction and whole body interaction[J]. Journal of Computer-Aided Design & Computer Graphics，2018，30(12)：2366-2376.
[10] KOSMAS P，ZAPHIRIS P. Embodied interaction in language learning: enhancing students’ collaboration and emotional engagement[C]//Proceedings of the Human- Computer Interaction–INTERACT 2019. Paphos，Cyprus，2019：179-196.
[11] 孔翠婷，潘沪生，张烈. 具身认知视角下的博物馆体感交互设计研究[J]. 装饰，2020(3)：90-93. KONG Cuiting，PAN Husheng，ZHANG Lie. Research on the interaction design of embodied interactive exhibits in museums from the perspective of embodied cognition[J]. ZHUANGSHI，2020(3)：90-93.
[12] LEE L，OKERLUND J，MAHER M L，et al. Embodied interaction design to promote creative social engagement for older adults[C]//Proceedings of the 22nd HCI International Conference. Berlin，Heidelberg: Springer-Verlag，2020：164-183.
[13] SZITA K. Embodied interaction and immersive film experiences on smartphones[J]. Mobile Storytelling in an Age of Smartphones，2021：205-219.
[14] RADU I，XU Y，MACINTYRE B. Embodied metaphor elicitation through augmented-reality game design[C]//Proceedings of the 12th International Conference on Interaction Design and Children. New York，USA: Association for Computing Machinery, 2013：412-414.
[15] LEE-CULTURA S，GIANNAKOS M. Embodied interaction and spatial skills：a systematic review of empirical studies[J]. Interacting with Computers，2020，32(4)：331-366.
[16] ANTLE A N. Exploring how children use their hands to think: an embodied interactional analysis[J]. Behaviour & Information Technology, 2013，32(9)：938-954.
[17] ANTLE A N，WANG S. Comparing motor-cognitive strategies for spatial problem solving with tangible and multi-touch interfaces[C]//Proceedings of the 7th International Conference on Tangible，Embedded and Embodied Interaction. Barcelona，Spain：ACM，2013：65-72.
[18] MEHTA R，SHORTZ A，BENDEN M. Standing up for learning：a pilot investigation on the neurocognitive benefits of stand-biased school desks[J]. International Journal of Environmental Research and Public Health，2015，13(1)：59.
[19] SCHAEFER S. Embodiment helps children solve a spatial working memory task：interactions with age and gender[J]. Journal of Cognitive Enhancement, 2019, 3(2): 233-244.
[20] NIEDENTHAL P M. Embodying emotion[J]. Science，2007，316(5827)：1002-1005.
[21] NAIR S，SAGAR M，SOLLERS J，et al. Do slumped and upright postures affect stress responses? a randomized trial[J]. Health Psychology，2015，34(6)：632-641.
[22] 罗仕鉴，潘云鹤，朱上上. 产品设计中基于图解思维的隐性知识表达[J]. 机械工程学报，2007(6)：93-98. LUO Shijian，PAN Yunhe，ZHU Shangshang. Pattens of tacit knowledge based on graphic thinking in product design[J]. Journal of Mechanical Engineering，2007(6)：93-98.
[23] HAO N，XUE H，YUAN H，et al. Enhancing creativity：proper body posture meets proper emotion[J]. Acta Psychologica，2017，173：32-40.
[24] 卢兆麟，王波，石清吟. 面向汽车造型设计模糊前端的思维机制研究[J]. 汽车工程，2017，39(5)：517-523. LU Zhaolin，WANG Bo，SHI Qingyin. A research on thinking mechanism in fuzzy front end of car styling[J]. Automotive Engineering，2017，39(5)：517-523.
[25] 卢兆麟，FRITZ F. 基于产品语义分析的新能源汽车造型设计研究[J]. 机械设计，2017，34(3)：111-116. LU Zhaolin，FRITZ F. Modeling design of new energy vehicles based on semantic analysis[J]. Journal of Machine Design，2017，34(3)：111-116.
[26] BHATTACHARYA J，PETSCHE H. Drawing on mind’s canvas：differences in cortical integration patterns between artists and non‐artists[J]. Human Brain Mapping，2005，26(1)：1-14.
[27] LIU Y，RITCHIE J M，LIM T，et al. A fuzzy psycho-physiological approach to enable the understanding of an engineer’s affect status during CAD activities[J]. Computer-Aided Design，2014，54：19-38.
[28] 白学军，姚海娟. 高低创造性思维水平者的认知抑制能力：行为和生理的证据[J]. 心理学报，2018，50(11)：1197-1211. BAI Xuejun，YAO Haijuan. Differences in cognitive inhibition between persons with high and low creativity：evidences from behavioral and physiological studies[J]. Acta Psychologica Sinica，2018，50(11)：1197-1211.
[29] 宋晓辉，施建农. 创造力测量手段——同感评估技术(CAT)简介[J]. 心理科学进展，2005(6)：37-42. SONG Xiaohui，SHI Jiannong. A consensual technique for creativity assessent[J]. Advances in Psychological Science，2005(6)：37-42.
[30] AMABILE T M. Social psychology of creativity: a consensual assessment technique[J]. Journal of Personality and Social Psychology，1982，43(5)：997-1013.
[31] 牛红伟，郝佳，曹贝宁，等. 面向产品概念设计的多模态智能交互框架及实现[J]. 计算机集成制造系统，28(8)：2508-2521. NIU Hongwei，HAO Jia，CAO Beining，et al. Multimodal intelligent interaction framework and realization for product conceptual design[J]. Computer Integrated Manufacturing Systems，28(8)：2508-2521.
[32] BALTERS S，WEINSTEIN T，MAYSELESS N，等. Design science and neuroscience：a systematic review of the emergent field of design neurocognition[J]. Design Studies，2023，84：101148.
[33] 袁萍，熊艳，李彦，等. 基于EEGα波活动探究信息激励距离对产品设计思维的影响[J]. 机械工程学报，2017，53(15)：40-48. YUAN Ping，XIONG Yan，LI Yan，et al. An exploration into the influence of stimulation distance of stimulus information on design thinking based on EEG alpha activity[J]. Journal of Mechanical Engineering，2017，53(15)：40-48.
[34] 郑玮琪，刘烨，傅小兰. 感觉-运动系统参与隐喻理解的认知神经机制[J]. 生物化学与生物物理进展，2018，45(3) ：325-335. ZHENG Weiqi，LIU Ye，FU Xiaolan. Cognitive and neural mechanisms of sensory-motor system’s role in metaphor comprehension[J]. Progress in Biochemistry and Biophysics，2018，45(3)：325-335.
[35] 邹迪，李红，王福顺. 唤醒定义探析及其认知神经生理基础[J]. 心理科学进展，2022，30(9)：2020-2033. ZOU Di，LI Hong，WANG Fushun. An investigation into the definition of arousal and its cognitive neurophysiological basis[J]. Advance in Psychological Science，2022，30(9)：2020-2033.
[36] 冯毅雄，娄山河，王绪鹏，等. 面向性能的定制产品感性意象评价方法研究[J]. 机械工程学报，2020，56(9)：181-190. FENG Yixiong，LOU Shanhe，WANG Xupeng，et al. Research on performance-oriented perceptual image evaluation method for customized products[J]. Journal of Mechanical Engineering，2020，56(9)：181-190.
[37] MASTRIA S，AGNOLI S，ZANON M，et al. Clustering and switching in divergent thinking: neurophysiological correlates underlying flexibility during idea generation[J]. Neuropsychologia，2021，158：107890.
[38] 谢平，吴晓光，牛小辰，等. 运动观察与运动想象的皮层节律活动与神经生理机制[J]. 中国科学：生命科学，2015，45(7)：665-676. XIE Ping，WU Xiaoguang，NIU Xiaochen, et al. Cortical rhythmic activity and neurophysiology mechanism during movement observation and motor imagery[J]. SCIENTIA SINICA Vitae，2015，45：665–676.
[39] HART S G，STAVELAND L E. Development of NASA-TLX (Task Load Index)：results of empirical and theoretical research[J]. Advances in Psychology，1988，52：139-183.
[40] SARKAR P，CHAKRABARTI A. Assessing design creativity[J]. Design Studies，2011，32(4)：348-383.
[41] CHRISTENSEN B T，BALL L J. Dimensions of creative evaluation：distinct design and reasoning strategies for aesthetic, functional and originality judgments[J]. Design Studies，2016，45：116-136.
[42] GARDONY A L，EDDY M D，BRUNYÉ T T，et al. Cognitive strategies in the mental rotation task revealed by EEG spectral power[J]. Brain and Cognition，2017，118：1-18.
[43] PIDGEON L M，GREALY M，DUFFY A H B，et al. Functional neuroimaging of visual creativity：a systematic review and meta‐analysis[J]. Brain and Behavior，2016，6(10)：e00540.
[44] ROMINGER C，PAPOUSEK I，PERCHTOLD C M，et al. The creative brain in the figural domain：distinct patterns of EEG alpha power during idea generation and idea elaboration[J]. Neuropsychologia，2018，118：13-19.
[45] YU H，BA S，GUO Y，et al. Effects of motor imagery tasks on brain functional networks based on EEG mu/beta rhythm[J]. Brain Sciences，2022，12(2)：194.
[46] 车敏诗，聂春燕，杨承金，等. 基于灰色关联法的生理信号与情绪关联度研究[J]. 计算机工程与应用，2021，57(11)：168-172. CHE Minshi，NIE Chunyan，YANG Chengjin，et al. Research on correlation between physiological signal and emotion based on grey relational method[J]. Computer Engineering and Applications，2021，57(11)：168-172.
[47] HU Y，OUYANG J，WANG H，et al. Design meets neuroscience：an electroencephalogram study of design thinking in concept generation phase[J]. Frontiers in Psychology，2022，13：832194.