Exploring the Effects of Embodied Interaction on Creative Thinking Based on Neurophysiological Measures

doi:10.3901/JME.2025.19.237

Abstract

Abstract: Embodied interaction based on multimodal technology allows designers to create in an "all-in-one" manner, and the impact of this interaction on creative thinking in the design process needs to be explored in depth. The neurophysiological measures of designers in traditional “keyboard + mouse” interaction and embodied interaction are compared to investigate the influence of embodied interaction on designers' creative thinking. Thirty-four designers complete a three-dimensional composition design task in a computer-aided design system using the traditional and embodied interaction respectively. Cognitive data such as electroencephalograms, heart rates, workload scales and verbal reports are collected simultaneously under both interaction modes. In addition, the design solutions are assessed for creativity using the Consensus Assessment Technique. The results of the data analysis show that, (i) the alpha band power of the designer’s parietal and occipital regions under embodied interaction is significantly lower than that of traditional interaction, indicating that the connection between the body and the virtual design space enhances the designer's visual processing and facilitates the generation of creative thinking; and (ii) the suppression of alpha and beta bands of the designer's sensorimotor region under embodied interaction suggests that the consistency of the designer’s body movements, postures, and daily experience accelerates designers' understanding of movement metaphors, which has a stimulating effect on creative thinking; (iii) the significant increase in the designer’s electrodermal activity and heart rate values under embodied interaction suggests that the novelty and flexibility of the operation process can bring about emotional arousal for designers, which is conducive to the generation of creative thinking; (iv) designers need more training and have a heavier workload in embodied interaction, which may have a negative impact on creative thinking. This suggests that the maturity of multimodal technology needs to be further improved in the future.

Key words: embodied interaction, creative thinking, neurophysiological measurements, multimodal technology, electroenceph- alogram

CLC Number:

TB472
TP391

WANG Wenjuan, LU Zhaolin, HAO Jia, YANG Weiquan, NIU Hongwei, ZUO Xiaolin. Exploring the Effects of Embodied Interaction on Creative Thinking Based on Neurophysiological Measures[J]. Journal of Mechanical Engineering, 2025, 61(19): 237-248.

References

[1] 程锦，叶虎强，谭建荣，等. 三维CAD技术研究进展及其发展趋势综述[J]. 机械工程学报，2023，59(23)：158-185. CHENG Jin，YE Huqiang，TAN Jianrong，et al. Review of research progress and development trends of 3D CAD technology[J]. Journal of Mechanical Engineering，2023，59(23)：158-185.
[2] 龙辉，郝佳，牛红伟，等. 基于情境的CAD模型手势操控技术[J]. 机械工程学报，2022，58(10)：374-382. LONG Hui，HAO Jia，NIU Hongwei，et al. Gesture control technology of CAD model based on situation model[J]. Journal of Mechanical Engineering，2022，58(10)：374.
[3] BORGIANNI Y，MACCIONI L，FIORINESCHI L，et al. Forms of stimuli and their effects on idea generation in terms of creativity metrics and non-obviousness[J]. International Journal of Design Creativity and Innovation，2020，8(3)：147-164.
[4] LIIKKANEN L，PERTTULA M K. Contextual cueing and verbal stimuli in design idea generation[C]//Design computing and cognition’06. Springer Netherlands，2006: 619-631.
[5] 宋璐. 姿势开放性对创造性思维的隐喻效应及机制[D]. 天津：天津师范大学，2022. SONG Lu. The metaphor effect of postural openness on creative thinking and its mechanism[D]. Tianjin：Tianjin Normal University，2022.
[6] 牛红伟，郝佳，王璐，等. 功能词对设计思维的启发效用研究[J]. 北京理工大学学报，2022，42(2)：168-176. NIU Hongwei，HAO Jia，WANG Lu，et al. Research on the inspiring effects of design thinking with functional terms[J]. Transactions of Beijing Institute of Technology, 2022，42(2)：168-176.
[7] 卢兆麟，宋浩. 基于视觉动力-灰色关联分析的车身造型形态进化机制研究[J]. 北京理工大学学报，2023，43(9)：903-911. LU Zhaolin，SONG Hao. Evolution mechanism of vehicle styling based on visual dynamics-gray correlation analysis[J]. Transactions of Beijing Institute of Technology，2023，43(9)：903-911.
[8] DOURISH P. Where the Action Is：The Foundations of Embodied Interaction[M]. Cambridge：MIT press，2001.
[9] 姚争为，杨琦，潘志庚，等. 具身交互与全身交互的比较[J]. 计算机辅助设计与图形学学报，2018，30(12)：2366-2376. YAO Zhengwei，YANG Qi，PAN Zhigeng，et al. Comparison of embodied interaction and whole body interaction[J]. Journal of Computer-Aided Design & Computer Graphics，2018，30(12)：2366-2376.
[10] KOSMAS P，ZAPHIRIS P. Embodied interaction in language learning: enhancing students’ collaboration and emotional engagement[C]//Proceedings of the Human- Computer Interaction–INTERACT 2019. Paphos，Cyprus，2019：179-196.
[11] 孔翠婷，潘沪生，张烈. 具身认知视角下的博物馆体感交互设计研究[J]. 装饰，2020(3)：90-93. KONG Cuiting，PAN Husheng，ZHANG Lie. Research on the interaction design of embodied interactive exhibits in museums from the perspective of embodied cognition[J]. ZHUANGSHI，2020(3)：90-93.
[12] LEE L，OKERLUND J，MAHER M L，et al. Embodied interaction design to promote creative social engagement for older adults[C]//Proceedings of the 22nd HCI International Conference. Berlin，Heidelberg: Springer-Verlag，2020：164-183.
[13] SZITA K. Embodied interaction and immersive film experiences on smartphones[J]. Mobile Storytelling in an Age of Smartphones，2021：205-219.
[14] RADU I，XU Y，MACINTYRE B. Embodied metaphor elicitation through augmented-reality game design[C]//Proceedings of the 12th International Conference on Interaction Design and Children. New York，USA: Association for Computing Machinery, 2013：412-414.
[15] LEE-CULTURA S，GIANNAKOS M. Embodied interaction and spatial skills：a systematic review of empirical studies[J]. Interacting with Computers，2020，32(4)：331-366.
[16] ANTLE A N. Exploring how children use their hands to think: an embodied interactional analysis[J]. Behaviour & Information Technology, 2013，32(9)：938-954.
[17] ANTLE A N，WANG S. Comparing motor-cognitive strategies for spatial problem solving with tangible and multi-touch interfaces[C]//Proceedings of the 7th International Conference on Tangible，Embedded and Embodied Interaction. Barcelona，Spain：ACM，2013：65-72.
[18] MEHTA R，SHORTZ A，BENDEN M. Standing up for learning：a pilot investigation on the neurocognitive benefits of stand-biased school desks[J]. International Journal of Environmental Research and Public Health，2015，13(1)：59.
[19] SCHAEFER S. Embodiment helps children solve a spatial working memory task：interactions with age and gender[J]. Journal of Cognitive Enhancement, 2019, 3(2): 233-244.
[20] NIEDENTHAL P M. Embodying emotion[J]. Science，2007，316(5827)：1002-1005.
[21] NAIR S，SAGAR M，SOLLERS J，et al. Do slumped and upright postures affect stress responses? a randomized trial[J]. Health Psychology，2015，34(6)：632-641.
[22] 罗仕鉴，潘云鹤，朱上上. 产品设计中基于图解思维的隐性知识表达[J]. 机械工程学报，2007(6)：93-98. LUO Shijian，PAN Yunhe，ZHU Shangshang. Pattens of tacit knowledge based on graphic thinking in product design[J]. Journal of Mechanical Engineering，2007(6)：93-98.
[23] HAO N，XUE H，YUAN H，et al. Enhancing creativity：proper body posture meets proper emotion[J]. Acta Psychologica，2017，173：32-40.
[24] 卢兆麟，王波，石清吟. 面向汽车造型设计模糊前端的思维机制研究[J]. 汽车工程，2017，39(5)：517-523. LU Zhaolin，WANG Bo，SHI Qingyin. A research on thinking mechanism in fuzzy front end of car styling[J]. Automotive Engineering，2017，39(5)：517-523.
[25] 卢兆麟，FRITZ F. 基于产品语义分析的新能源汽车造型设计研究[J]. 机械设计，2017，34(3)：111-116. LU Zhaolin，FRITZ F. Modeling design of new energy vehicles based on semantic analysis[J]. Journal of Machine Design，2017，34(3)：111-116.
[26] BHATTACHARYA J，PETSCHE H. Drawing on mind’s canvas：differences in cortical integration patterns between artists and non‐artists[J]. Human Brain Mapping，2005，26(1)：1-14.
[27] LIU Y，RITCHIE J M，LIM T，et al. A fuzzy psycho-physiological approach to enable the understanding of an engineer’s affect status during CAD activities[J]. Computer-Aided Design，2014，54：19-38.
[28] 白学军，姚海娟. 高低创造性思维水平者的认知抑制能力：行为和生理的证据[J]. 心理学报，2018，50(11)：1197-1211. BAI Xuejun，YAO Haijuan. Differences in cognitive inhibition between persons with high and low creativity：evidences from behavioral and physiological studies[J]. Acta Psychologica Sinica，2018，50(11)：1197-1211.
[29] 宋晓辉，施建农. 创造力测量手段——同感评估技术(CAT)简介[J]. 心理科学进展，2005(6)：37-42. SONG Xiaohui，SHI Jiannong. A consensual technique for creativity assessent[J]. Advances in Psychological Science，2005(6)：37-42.
[30] AMABILE T M. Social psychology of creativity: a consensual assessment technique[J]. Journal of Personality and Social Psychology，1982，43(5)：997-1013.
[31] 牛红伟，郝佳，曹贝宁，等. 面向产品概念设计的多模态智能交互框架及实现[J]. 计算机集成制造系统，28(8)：2508-2521. NIU Hongwei，HAO Jia，CAO Beining，et al. Multimodal intelligent interaction framework and realization for product conceptual design[J]. Computer Integrated Manufacturing Systems，28(8)：2508-2521.
[32] BALTERS S，WEINSTEIN T，MAYSELESS N，等. Design science and neuroscience：a systematic review of the emergent field of design neurocognition[J]. Design Studies，2023，84：101148.
[33] 袁萍，熊艳，李彦，等. 基于EEGα波活动探究信息激励距离对产品设计思维的影响[J]. 机械工程学报，2017，53(15)：40-48. YUAN Ping，XIONG Yan，LI Yan，et al. An exploration into the influence of stimulation distance of stimulus information on design thinking based on EEG alpha activity[J]. Journal of Mechanical Engineering，2017，53(15)：40-48.
[34] 郑玮琪，刘烨，傅小兰. 感觉-运动系统参与隐喻理解的认知神经机制[J]. 生物化学与生物物理进展，2018，45(3) ：325-335. ZHENG Weiqi，LIU Ye，FU Xiaolan. Cognitive and neural mechanisms of sensory-motor system’s role in metaphor comprehension[J]. Progress in Biochemistry and Biophysics，2018，45(3)：325-335.
[35] 邹迪，李红，王福顺. 唤醒定义探析及其认知神经生理基础[J]. 心理科学进展，2022，30(9)：2020-2033. ZOU Di，LI Hong，WANG Fushun. An investigation into the definition of arousal and its cognitive neurophysiological basis[J]. Advance in Psychological Science，2022，30(9)：2020-2033.
[36] 冯毅雄，娄山河，王绪鹏，等. 面向性能的定制产品感性意象评价方法研究[J]. 机械工程学报，2020，56(9)：181-190. FENG Yixiong，LOU Shanhe，WANG Xupeng，et al. Research on performance-oriented perceptual image evaluation method for customized products[J]. Journal of Mechanical Engineering，2020，56(9)：181-190.
[37] MASTRIA S，AGNOLI S，ZANON M，et al. Clustering and switching in divergent thinking: neurophysiological correlates underlying flexibility during idea generation[J]. Neuropsychologia，2021，158：107890.
[38] 谢平，吴晓光，牛小辰，等. 运动观察与运动想象的皮层节律活动与神经生理机制[J]. 中国科学：生命科学，2015，45(7)：665-676. XIE Ping，WU Xiaoguang，NIU Xiaochen, et al. Cortical rhythmic activity and neurophysiology mechanism during movement observation and motor imagery[J]. SCIENTIA SINICA Vitae，2015，45：665–676.
[39] HART S G，STAVELAND L E. Development of NASA-TLX (Task Load Index)：results of empirical and theoretical research[J]. Advances in Psychology，1988，52：139-183.
[40] SARKAR P，CHAKRABARTI A. Assessing design creativity[J]. Design Studies，2011，32(4)：348-383.
[41] CHRISTENSEN B T，BALL L J. Dimensions of creative evaluation：distinct design and reasoning strategies for aesthetic, functional and originality judgments[J]. Design Studies，2016，45：116-136.
[42] GARDONY A L，EDDY M D，BRUNYÉ T T，et al. Cognitive strategies in the mental rotation task revealed by EEG spectral power[J]. Brain and Cognition，2017，118：1-18.
[43] PIDGEON L M，GREALY M，DUFFY A H B，et al. Functional neuroimaging of visual creativity：a systematic review and meta‐analysis[J]. Brain and Behavior，2016，6(10)：e00540.
[44] ROMINGER C，PAPOUSEK I，PERCHTOLD C M，et al. The creative brain in the figural domain：distinct patterns of EEG alpha power during idea generation and idea elaboration[J]. Neuropsychologia，2018，118：13-19.
[45] YU H，BA S，GUO Y，et al. Effects of motor imagery tasks on brain functional networks based on EEG mu/beta rhythm[J]. Brain Sciences，2022，12(2)：194.
[46] 车敏诗，聂春燕，杨承金，等. 基于灰色关联法的生理信号与情绪关联度研究[J]. 计算机工程与应用，2021，57(11)：168-172. CHE Minshi，NIE Chunyan，YANG Chengjin，et al. Research on correlation between physiological signal and emotion based on grey relational method[J]. Computer Engineering and Applications，2021，57(11)：168-172.
[47] HU Y，OUYANG J，WANG H，et al. Design meets neuroscience：an electroencephalogram study of design thinking in concept generation phase[J]. Frontiers in Psychology，2022，13：832194.