Flight Simulation Cockpit Imagery Biomimetic Visual Focus Prediction Model Based on FCN-enhanced Genetic Algorithm

doi:10.3901/JME.2025.03.154

Abstract

Abstract: The low cognitive aggregation of cockpit field of view (FOV) is identified as a key issue affecting the efficiency of flight simulation cockpit training. An approach based on fully convolutional networks (FCN) optimized genetic algorithm (GA) is proposed for predicting biomimetic visual focus in cockpit imagery. Firstly, cockpit FOV zones are determined based on grid embedding combined with eye tracker areas of interest, and the existing cockpit cognitive aggregation is validated using eye movement swarm plots and questionnaires. Secondly, a biomimetic design dataset is formed through dataset selection and optimization of eye movement grayscale images, and FCN models are trained for biological, product, and line drawing. Combining the FCN model with predicted inherent visual focus heatmaps and digital matrices of biological shapes, visual cognitive scores are calculated, zones are sorted according to subjective cognitive scores, and a biomimetic mapping relationship is established for expressing schematic designs of characterized zones. Finally, leveraging the FCN model, an adaptability function is established, and GA is utilized for optimizing schematic representations of zones. Decoding the optimal cockpit factors with the best match between visual and subjective cognition, refining the design, and validating the design scheme's visual cognition. The results demonstrate that the FCN-optimized GA model can enhance biomimetic design efficiency, resulting in cockpit FOV designs with higher cognitive aggregation, significantly improving the alignment between drivers' subjective cognition of cockpit FOV and visual cognition.

Key words: visual focus prediction, FCN, flight simulation cockpit, GA, image bionic design

CLC Number:

TB47

CHEN Guoqiang, SHEN Zhengyi, YANG Yuchi, LI Teng, XU Li. Flight Simulation Cockpit Imagery Biomimetic Visual Focus Prediction Model Based on FCN-enhanced Genetic Algorithm[J]. Journal of Mechanical Engineering, 2025, 61(3): 154-166.

References

[1] 宋鑫康，赵尚弘，王翔，等. 航空信息网络服务功能链协同构建与映射策略[J]. 系统工程与电子技术，2022，44(11):3556-3563. SONG Xinkang，ZHAO Shanghong，WANG Xiang，et al. Collaborative construction and embedding strategy of aviation information network service function chain[J]. Systems Engineering and Electronics，2022，44(11):3556-3563.
[2] 罗仕鉴，龚何波，林伟. 智能产品交互设计研究现状与进展[J]. 机械工程学报，2023，59(11):1-15. LUO Shijian，GONG Hebo，LIN Wei. Status and progress of intelligent products interaction design[J]. Journal of Mechanical Engineering，2023，59(11):1-15.
[3] 宋鑫康，赵尚弘，王翔. 支持差异化服务的航空信息网络切片策略[J]. 航空学报，2022，43(3):382-391. SONG Xinkang，ZHAO Shanghong，WANG Xiang. Aviation information network slicing strategy supporting differentiated services[J]. Acta Aeronautica et Astronautica Sinica，2022，43(3):382-391.
[4] 徐江，孙刚，徐静妤，等. 智能交互的认知哲学逻辑及设计评价研究[J]. 机械工程学报，2023，59(11):31-42. XU Jiang，SUN Gang，XU Jingyu，et al. Cognition philosophy logic and design evaluation of intelligent interaction[J]. Journal of Mechanical Engineering，2023，59(11):31-42.
[5] ZHANG Xia，SUN Youchao，ZHANG Yanjun. A task modeling method of intelligent human-computer interaction in aircraft cockpits based on information load flow[J]. IEEE Transactions on Aerospace and Electronic Systems，2022，58(6):5619-5634.
[6] LI Wenbo，XUE Jiyong，TAN Rruichen，et al. Global-local-feature-fused driver speech emotion detection for intelligent cockpit in automated driving[J]. IEEE Transactions on Intelligent Vehicles，2023，8(4):2684-2697.
[7] ZHANG Xia，SUN Youchao，ZHANG Yanjun. Ontology modelling of intelligent HCI in aircraft cockpit[J]. Aircraft Engineering and Aerospace Technology，2021，93(5):794-808.
[8] 陈国强，申正义，侯雨雷，等. 基于实景控制的直升机模拟驾驶平台:中国，CN115457836B[P]. 2023-06-13． CHEN Guoqiang，SHEN Zhengyi，HOU Yulei，et al. Helicopter simulation driving platform based on real scene control:China，CN115457836B[P]. 2023-06-13.
[9] 罗仕鉴，张宇飞，边泽，等. 产品外形仿生设计研究现状与进展[J]. 机械工程学报，2018，54(21):138-155. LUO Shijian，ZHANG Yufei，BIAN Ze，et al. Status and progress of product shape bionic design[J]. Journal of Mechanical Engineering，2018，54(21):138-155.
[10] 王年文，张立杰，毕翼飞，等. 生物原型驱动的手功能康复产品仿生设计[J]. 机械工程学报，2023，59(11):300-307. WANG Nianwen，ZHANG Lijie，BI Yifei，et al. Research on bionic design of hand functional rehabilitation products driven by biological prototype[J]. Journal of Mechanical Engineering，2023，59(11):300-307.
[11] DENG Zhenggen，LÜ Jian，LIU Xiang，et al. Bionic design model for co-creative product innovation based on deep generative and BID[J]. International Journal of Computational Intelligence Systems，2023，16(8):235-246.
[12] 李雄，苏建宁，张志鹏. 基于深度学习的产品概念草图生成设计研究[J]. 机械工程学报，2023，59(11):16-30. LI Xiong，SU Jianning，ZHANG Zhipeng. Product conceptual sketch generation design using deep learning[J]. Journal of Mechanical Engineering，2023，59(11):16-30.
[13] 薛君蕊，王昱潭，曲爱丽，等. 基于改进FCN-8s的灵武长枣图像分割方法[J]. 农业工程学报，2021，37(5):191-197. XUE Junrui，WANG Yutan，QU Aili，et al. Image segmentation method for Lingwu long jujubes based on improved FCN-8s[J]. Transactions of the Chinese Society of Agricultural Engineering，2021，37(5):191-197.
[14] 郭颖，李增元，陈尔学，等. 一种改进的高空间分辨率遥感影像森林类型深度学习精细分类方法:双支FCN-8s[J]. 林业科学，2020，56(3):48-60. GUO Ying，LI Zengyuan，CHEN Erxue，et al. A deep learning method for forest fine classification based on high resolution remote sensing images:Two-branch FCN-8s[J]. Scientia Silvae Sinicae，2020，56(3):48-60.
[15] MOI H Y，MANU G，FATIMA M，et al. Breast ultrasound lesions recognition:End-to-end deep learning approaches[J]. Journal of Medical Imaging，2018，6(1):25-43.
[16] 郭鑫，王海涵，王杰，等. 韧性产品交互式概念设计与知识推理方法研究[J]. 机械工程学报，2023，59(11):74-83. GUO Xin，WANG Haihan，WANG Jie，et al. Research on interactive conceptual design and knowledge reasoning method of resilient products[J]. Journal of Mechanical Engineering，2023，59(11):74-83.
[17] 王剑，陈丰，乔印虎，等. 目标意象约束的人机辅助产品造型进化设计研究[J]. 机械设计，2022，39(2):120-128. WANG Jian，CHEN Feng，QIAO Yinhu，et al. Research on evolutionary design of human-machine aided product modeling constrained by target image[J]. Journal of Machine Design，2022，39(2):120-128.
[18] JIANG Nan. Smart home product layout design method based on real-number coding genetic algorithm[J]. Computational Intelligence and Neuroscience，2022，37(4):245-255.
[19] YAN Chenggang，HAO Yiming，LI Liang，et al. Task-adaptive attention for image captioning[J]. IEEE Transactions on Circuits and Systems for Video Technology，2022，32(1):43-51.
[20] CAI Jiannan，YANG Liu，ZHANG Yuxi，et al. Multitask learning method for detecting the visual focus of attention of construction workers[J]. Journal of Construction Engineering and Management，2021，147(7):04021063.
[21] CHAKRABORTY P，AHMED S，YOUSUF M A，et al. A human-robot interaction system calculating visual focus of human’s attention level[J]. IEEE Access，2021，9:93409-93421.
[22] 赵怡，高淑萍，何迪. 基于深度学习的眼动跟踪数据融合算法[J]. 计算机工程与应用，2021，57(10):211-217. ZHAO Yi，GAO Shuping，HE Di. Eye movement and tracking data fusion algorithm based on deep learning[J]. Computer Engineering and Applications，2021，57(10):211-217.
[23] 刘纪元，祁瀚文，刘志诚，等. 一种基于机器视觉的精准注意力追踪系统[J]. 系统仿真学报，2023，35(10):2087-2100. LIU Jiyuan，QI Hanwen，LIU Zhicheng，et al. A precise attention tracking system based on computer vision[J]. Journal of System Simulation，2023，35(10):2087-2100.
[24] VAJS I，PAPIĆ T，KOVIĆ V，et al. Accessible dyslexia detection with real-time reading feedback through robust interpretable eye-tracking features[J]. Brain Sciences，2023，13(3):405.
[25] GUO Xiaoying，LI Liang，ASANO A，et al. Influences of global and local features on eye-movement patterns in visual-similarity perception of synthesized texture images[J]. Applied Sciences，2020，10(16):5552.
[26] LOTZE A，LOVE K，VELISAR A，et al. A low-cost robotic oculomotor simulator for assessing eye tracking accuracy in health and disease[J]. Behavior Research Methods，2024，56(1):80-92.
[27] BYLINSKII Z，NAM W K，O'DONOVAN P，et al. Learning visual importance for graphic designs and data visualizations[J]. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology，2017，17:57-69.
[28] 段慧煜，任晓雨，王琳琳，等. 全方位空间自闭症儿童眼动信号处理与分析[J]. 信号处理，2022，38(9):1797-1808. DUAN Huiyu，REN Xiaoyu，WANG Linlin，et al. Processing and analysis of eye movement signals for autistic children in omnidirectional space[J]. Journal of Signal Processing，2022，38(9):1797-1808.
[29] 罗会兰，袁璞，童康. 基于深度学习的显著性目标检测方法综述[J]. 电子学报，2021，49(7):1417-1427. LUO Huilan，YUAN Pu，TONG Kang. Review of the methods for salient object detection based on deep learning[J]. Acta Electronica Sinica，2021，49(7):1417-1427.
[30] 陈国强，申正义，孙利，等. 基于BP神经网络优化遗传算法的智能座舱感性意象预测[J]. 汽车工程，2023，45(8):1479-1488. CHEN Guoqiang，SHEN Zhengyi，SUN Li，et al. Intelligent cockpit perceptual image prediction based on BP neural network optimization genetic algorithm[J]. Automotive Engineering，2023，45(8):1479-1488.