基于元学习的多可信度深度神经网络代理模型

doi:10.3901/JME.2022.01.190

机械工程学报 ›› 2022, Vol. 58 ›› Issue (1): 190-200.doi: 10.3901/JME.2022.01.190

扫码分享

基于元学习的多可信度深度神经网络代理模型

张立¹, 陈江涛², 熊芬芬¹, 任成坤¹, 李超¹

1. 北京理工大学宇航学院北京 100081;
2. 中国空气动力研究与发展中心绵阳 621000

收稿日期:2021-02-02 修回日期:2021-07-06 出版日期:2022-01-05 发布日期:2022-03-19
通讯作者: 熊芬芬(通信作者),女,1982年出生,博士,副教授。主要研究领域为飞行器多学科优化设计、不确定性量化、飞行器轨迹优化和制导。E-mail:fenfenx@bit.edu.cn
作者简介:张立,男,1997年出生。主要研究方向为不确定性量化。E-mail:18811367828@163.com
基金资助:
国家数值风洞工程项目（NNW2020ZT7-B31）和科学挑战专题（TZ2019001）资助项目。

Meta-Learning Based Multi-Fidelity Deep Neural Networks Metamodel Method

ZHANG Li¹, CHEN Jiangtao², XIONG Fenfen¹, REN Chengkun¹, LI Chao¹

1. School of Astronautics, Beijing Institute of Technology, Beijing 100081;
2. China Aerodynamic Research and Development Center, Mianyang 621000

Received:2021-02-02 Revised:2021-07-06 Online:2022-01-05 Published:2022-03-19

摘要/Abstract

摘要： 为了降低计算量，多可信度代理模型技术通过融合不同精度和计算量的分析模型构建高精度分析模型的代理模型，在基于仿真的工程优化中得到广泛应用。现有的多可信度建模方法在面对高维问题时往往仍需要大量高精度样本点，计算量很大，且大都基于高斯随机过程理论，超参数估计时长随着问题维数和非线性程度的增加明显增长且不够稳健。为此，充分利用深度神经网络在高维信息提取和近似方面的巨大潜能，以及元学习理论在小样本学习领域的优势，提出一种基于元学习的多可信度深度神经网络（Meta-learning based multi-fidelity deep neural networks，MLMF-DNN）代理模型方法。通过若干数学算例和NACA0012翼型稳健优化问题的应用，表明提出的MLMF-DNN方法相比于经典的Co-Kriging方法，在预测精度和训练时长上均有明显改善，对于高维问题优势更明显。

关键词: 深度神经网络, 元学习, 多可信度, 代理模型, 稳健优化

Abstract: To reduce the computational cost, the multi-fidelity metamodel methods that fusion analysis models with different computational cost and accuracy have been widely used in simulation-based engineering optimization. The existing multi-fidelity modeling methods still need a large number of high-fidelity and computationally expensive sample points especially for high-dimensional problems. Meanwhile, most of them are based on the Gaussian random process theory, and thus the time cost by hyper-parameter estimation increases significantly with the increase of dimension and nonlinearity of problems and the robustness is low. To address these issues, makes full use of the great potential of deep neural networks in high-dimensional information extraction and approximation, as well as the advantages of meta-learning theory in the field of small-sample learning, and develops a meta-learning based multi-fidelity deep neural network surrogate model (MLMF-DNN) method. Through several mathematical examples and the application of NACA0012 airfoil robust optimization problem, it is shown that the proposed MLMF-DNN approach is significantly improved in prediction accuracy and training time cost compared with the classical Co-Kriging method, especially for high-dimensional problems.

Key words: deep neural networks, meta-learning, multi-fidelity, surrogate model, robust optimization

中图分类号:

张立, 陈江涛, 熊芬芬, 任成坤, 李超. 基于元学习的多可信度深度神经网络代理模型[J]. 机械工程学报, 2022, 58(1): 190-200.

ZHANG Li, CHEN Jiangtao, XIONG Fenfen, REN Chengkun, LI Chao. Meta-Learning Based Multi-Fidelity Deep Neural Networks Metamodel Method[J]. Journal of Mechanical Engineering, 2022, 58(1): 190-200.

参考文献

[1] 凌三强,徐乐,付饶,等.基于稳健性设计原理的接触件插拔力质量一致性优化方法[J]. 机械工程学报,2017,53(4):190-197. LING Sanqiang,XU Le,FU Rao,et al.Optimization method of quality consistency for insertion force of electrical contact based on robust design principle[J].Journal of Mechanical Engineering,2017, 53(4):190-197.
[2] 刘宇,李翔宇,张小虎. 考虑载荷动态分配机制的多状态系统可靠性建模及优化[J]. 机械工程学报,2016, 52(6):197-205. LIU Yu, LI Xiangyu, ZHANG Xiaohu. Multi-state system reliability modeling and optimization with considering dynamic load distribution mechanism[J]. Journal of Mechanical Engineering,2016,52(6):197-205.
[3] ELLIS A,ISKANDAR R,SCHMID C,et al. Active learning for efficiently training emulators of computationally expensive mathematical models[J]. Stats in Medicine,2020,39(25):3521-3548.
[4] FORRESTER A,KEANE A,et al. Multi-fidelity optimization via surrogate modelling[J]. Proceedings of the Royal Society a Mathematical,2007,463(2088):3251-3269.
[5] PARUSSINI L,VENTURI D,PERDIKARIS P,et al. Multi-fidelity Gaussian process regression for prediction of random fields[J]. Journal of Computational Physics, 2017,336:36-50.
[6] LIU Y,CHEN S,WANG F,et al. Sequential optimization using multi-level cokriging and extended expected improvement criterion[J]. Structural and Multidisciplinary Optimization,2018,58(3):1155-1173.
[7] SHI R,LONG T,BAOYIN H. Multi-fidelity and multi-objective optimization of low-thrust transfers with control strategy for all-electric geostationary satellites[J]. Acta Astronautica,2020,177:577-587.
[8] HAN Z,GOERTZ S. Hierarchical kriging model for variable-fidelity surrogate modeling[J]. Aiaa Journal, 2012,50(9):1885-1896.
[9] HAN Z,XU C,ZHANG L,et al. Efficient aerodynamic shape optimization using variable-fidelity surrogate models and multilevel computational grids[J]. Chinese Journal of Aeronautics, 2020, 33(1):31-47.
[10] NICOLAS,COURRIER,PIERRE-ALAIN,et al. Variable-fidelity modeling of structural analysis of assemblies[J]. Journal of Global Optimization,2016,64(3):577-613.
[11] ZHOU Q,WANG Y,CHOI S,et al. A sequential multi-fidelity metamodeling approach for data regression[J]. Knowledge-Based Systems,2017,134(15):199-212.
[12] WANG F,XIONG F,CHEN S,et al. Multi-fidelity uncertainty propagation using polynomial chaos and Gaussian process modeling[J]. Structural and Multidisciplinary Optimization, 2019, 60(4):1583-1604.
[13] 韩忠华,许晨舟,乔建领,等.基于代理模型的高效全局气动优化设计方法研究进展[J]. 航空学报,2020,41(5):30-70. HAN Zhonghua,XU Chenzhou,QIAO Jianling,et al. Recent progress of efficient global aerodynamic shape optimization using surrogate-based approach[J]. Acta Aeronautica et Astronautica Sinica,2020,41(5):30-70.
[14] 张浩,吴秀娟. 深度学习的内涵及认知理论基础探析[J]. 中国电化教育,2012(10):7-11,21. ZHANG Hao,WU Xiujuan. An analysis on the connotation and cognitive theoretical basis of deep learning[J]. China Educational Technology,2012(10):7-11,21.
[15] 陈海,钱炜祺,何磊. 基于深度学习的翼型气动系数预测[J]. 空气动力学学报,2018,36(2):294-299. CHEN Hai,QIAN Weiqi,HE Lei. Aerodynamic coefficient prediction of airfoils based on deep learning[J]. Acta Aerodynamica Sinica,2018,36(2):294-299.
[16] 范周伟,余雄庆,王朝等. 基于深度神经网络模型的客机总体主要设计参数敏感性分析[J]. 航空学报, 2021,42(4):524353. FAN Zhouwei,YU Xiongqing,WANG Chao,et al. Sensitivity analysis of main design parameters of passenger aircraft based on deep neural network model[J/OL]. Acta Aeronautica et Astronautica Sinica,2021,42(4):24353.
[17] 廖鹏,姚磊江,白国栋,等. 基于深度学习的混合翼型前缘压力分布预测[J]. 航空动力学报,2019,34(8):1751-1758. LIAO Peng,YAO Leijiang,BAI Guodong,et al. Prediction of hybrid airfoil leading edge pressure distribution based on deep learning[J]. Journal of Aerospace Power,2019,34(8):1751-1758.
[18] TAO J,SUN G. Application of deep learning based multi-fidelity surrogate model to robust aerodynamic design optimization[J]. Aerospace Science and Technology,2019,92:722-737.
[19] LI Y,JIANG J,SHAO Y,et al. Fast Hyperparameter optimization of deep neural networks via ensembling Multiple Surrogates[J]. arXiv,2018:1811.02319.
[20] ZHU G,ZHU R. Accelerating hyperparameter optimization of deep neural network via progressive multi-fidelity evaluation[R]. Nanjing:Advances in Knowledge Discovery and Data Mining,2020:752-763.
[21] CANG R,HOPE Y,YI R. One-shot generation of near-optimal topology through theory-driven machine learning[J]. Computer-Aided Design,2018,109.
[22] CONNOR S,TAGHI M. A survey on image data augmentation for deep learning[J]. Journal of Big Data,2019,6(1):1-48.
[23] AUGUSTO D,AURORA P. On the use of fitness landscape features in meta-learning based algorithm selection for the quadratic assignment problem[J]. 2020, 805:62-75.
[24] 闫雷鸣,严璐绮,王超智,等. 基于句式元学习的Twitter分类[J]. 北京大学学报,2019,55(1):98-104. YAN Leiming, YAN Luqi,WANG Chaozhi,et al. Sentence style meta learning for twitter classification[J]. Acta Scientiarum Naturalium Universitatis Pekinensis,2019,55(1):98-104.
[25] 王璐,潘文林. 基于元学习的语音识别探究[J]. 云南民族大学学报,2019,28(5):510-516. WANG Lu,PAN Wenlin. Speech recognition based on meta-learning[J]. Journal of Yunnan University of Nationalities,2019,28(5):510-516.
[26] FINN C,ABBEEL P,LEVINE S. Model-agnostic meta-learning for fast adaptation of deep networks[J]. 2017,70:1126-1135.
[27] PAN J,LIU C,WANG Z,et al. Investigation of deep neural networks (DNN) for large vocabulary continuous speech recognition:Why DNN surpasses GMMS in acoustic modeling[J]. IEEE,2012,7196(8):301-305.
[28] XU Y,MA J,LIAW A,et al. Demystifying multi-task deep neural networks for quantitative structure-activity relationships[J]. Journal of Chemical Information & Modeling,2017,57(10):2490-2504.
[29] 孙文珺,邵思羽,严如强.基于稀疏自动编码深度神经网络的感应电动机故障诊断[J]. 机械工程学报,2016,52(9):65-71. SUN Wenjun,SHAO Siyu,YAN Ruqiang. Induction motor fault diagnosis based on deep neural network of sparse auto-encoder[J]. Journal of Mechanical Engineering,2016,52(9):65-71.
[30] YAN L,ZHOU T. An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems[J]. arXiv,2019:1911.08926.
[31] XIONG F,XIONG Y,CHEN W. Optimizing Latin hypercube design for sequential sampling of computer experiments[J]. Engineering Optimization,2009,41(8):793-810.

基于元学习的多可信度深度神经网络代理模型

Meta-Learning Based Multi-Fidelity Deep Neural Networks Metamodel Method

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	程洪鑫, 李璐祎, 周长聪. 基于Kriging模型和代理抽样的分布参数不确定性灵敏度分析方法[J]. 机械工程学报, 2024, 60(8): 370-383.
[2]	孟原, 史宝军, 张德权. Kriging-高维代理模型建模方法研究与改进[J]. 机械工程学报, 2024, 60(5): 249-263.
[3]	俞水, 吴晓, 郭鹏, 王志华. 基于首次穿越PDF自适应估计的时变可靠性分析方法[J]. 机械工程学报, 2024, 60(5): 264-275.
[4]	吕利叶, 鲁玉军, 王硕, 刘印, 李昆鹏, 宋学官. 代理模型技术及其应用：现状与展望[J]. 机械工程学报, 2024, 60(3): 254-281.
[5]	赵峰, 胡伟飞, 李光, 邓晓豫, 刘振宇, 郭云飞, 谭建荣. 基于混合指标自适应采样代理模型的多目标优化设计方法[J]. 机械工程学报, 2024, 60(13): 81-91.
[6]	胡伟飞, 廖家乐, 郭云飞, 鄢继铨, 李光, 岳海峰, 谭建荣. 基于物理信息神经网络的时变可靠性分析方法[J]. 机械工程学报, 2024, 60(13): 141-153.
[7]	范小宁, 王凯, 余畅. 基于约束边界抽样的起重机金属结构可靠性优化[J]. 机械工程学报, 2023, 59(8): 288-298.
[8]	常建涛, 乔子萱, 孔宪光, 杨胜康, 罗才文. 多维非线性特征重构与融合的复杂产品工期预测方法[J]. 机械工程学报, 2023, 59(6): 295-309.
[9]	李春明, 孙晓霞, 张涛, 舒成龙, 张帅, 宋学官. 基于组合代理模型的变海拔工况车辆动力总成流动性能优化[J]. 机械工程学报, 2023, 59(4): 135-144.
[10]	陆晨浩, 陈耀, 何若琪, 范维莹, 冯健. 基于深度神经网络的四折痕锥形折纸结构设计[J]. 机械工程学报, 2023, 59(21): 167-176.
[11]	林京, 焦金阳. 可解释机械智能诊断技术的研究进展与挑战[J]. 机械工程学报, 2023, 59(20): 215-224.
[12]	何庆, 利璐, 李晨钟, 王平, 谢斯. 基于Kriging模型的在役高速列车悬挂参数近似贝叶斯估计[J]. 机械工程学报, 2023, 59(12): 139-148.
[13]	张文鑫, 吕震宙. 一种新的自适应Kriging法停止准则及其在涡轮盘疲劳寿命可靠性中的应用[J]. 机械工程学报, 2022, 58(6): 263-273.
[14]	余晓霞, 邓蕾, 汤宝平, 夏乙, 李琪康. 小样本条件下元学习门控神经网络的齿轮退化趋势预测研究[J]. 机械工程学报, 2022, 58(3): 149-156.
[15]	陈炜镒, 胡伟飞, 方健豪, 姜宏伟, 刘振宇, 谭建荣. 考虑雨滴侵蚀的风力发电机叶片涂层疲劳寿命优化设计方法[J]. 机械工程学报, 2022, 58(17): 2-15.