支持形态自定义和自感知的可变形界面设计方法

doi:10.3901/JME.2023.11.084

机械工程学报 ›› 2023, Vol. 59 ›› Issue (11): 84-93.doi: 10.3901/JME.2023.11.084

• 特邀专栏:智能产品交互设计 • 上一篇下一篇

扫码分享

支持形态自定义和自感知的可变形界面设计方法

周子洪¹, 尤伟涛¹, 陆胤瑜¹, 郗宇凡¹, 王可幸¹, 王冠云¹, 孙凌云^1,2

1. 浙江大学-南方电网人工智能创新联合研究中心杭州 310027;
2. 浙江大学计算机辅助设计与图形学国家重点实验室杭州 310027

收稿日期:2021-12-01 修回日期:2022-06-22 出版日期:2023-06-05 发布日期:2023-07-19
通讯作者: 尤伟涛(通信作者),男,1989年出生,博士,特聘研究员,主要研究方向为智能设计和审美计算。E-mail:weitao_you@zju.edu.cn
作者简介:周子洪,男,1997年出生,博士研究生。主要研究方向为人机交互和智能产品设计。E-mail:zihongzhou@zju.edu.cn;陆胤瑜,男,2000年出生,硕士研究生。主要研究方向为智能产品设计。E-mail:buliugu6@outlook.com;郗宇凡,女,2000年出生,硕士研究生。主要研究方向为可持续设计。E-mail:xiyufan_echo@163.com;王可幸,女,1998年出生,硕士研究生。主要研究方向为工业设计。E-mail:wakexi328@163.com;王冠云,男,1987年出生,博士,研究员,博士研究生导师。主要研究方向为数字化设计与制造、实体交互界面、4D打印等。E-mail:guanyun@zju.edu.cn;孙凌云,男,1981年出生,博士,教授,博士研究生导师。主要研究方向为人工智能、设计智能、信息与交互设计。E-mail:sunly@zju.edu.cn
基金资助:
科技创新2030-“新一代人工智能”重大资助项目(2018AAA0100703)

Shape-changing Interface Design with the User-defined Form and Self-sensing Capability

ZHOU Zihong¹, YOU Weitao¹, LU Yinyu¹, XI Yufan¹, WANG Kexing¹, WANG Guanyun¹, SUN Lingyun^1,2

1. Zhejiang University-China Southern Power Grid Joint Research Centre on AI, Hangzhou 310027;
2. State Key Laboratory of CAD & CG at Zhejiang University, Hangzhou 310027

Received:2021-12-01 Revised:2022-06-22 Online:2023-06-05 Published:2023-07-19

摘要/Abstract

摘要： 可变形界面是一种新型的人机交互界面，智能时代下如何应用新兴技术推动可变形界面朝着智能化方向发展已成为人机交互界面研究的前沿。通过分析可变形界面的三大构成元素及其相互关系，明确了感知载体与交互方式是促进智能技术融入可变形界面的重点。围绕界面形态的自定义设计，探索了将模块化惯性传感器网络融入可变形界面的设计过程；围绕界面形态的自感知设计，探索了利用机器学习的方法实现界面形变动作的精准识别，并设计了相应的快速原型平台以提高设计效率。为评估自感知设计方法的有效性，就不同形态界面的形变动作识别结果做了对比分析；为评估形态自定义设计方法的可用性，就传感器网络的实时性、传感器硬件能耗以及硬件成本做了实验和讨论。最后，给出了可变形界面在柔性可穿戴设备、智能交互玩具、新型输入控制设备三个领域中的设计实例，表明上述设计方法能够有效开发具有相应形态功能的可变形界面，并推动可变形界面的应用场景朝更加多元化的方向发展。

关键词: 可变形界面, 传感器网络, 机器学习, 快速原型

Abstract: Shape-changing interface is a new-type human-computer interface. In the era of intelligence, how to apply emerging technologies to promote the development of shape-changing interfaces towards intelligence has become the research frontier of the human-computer interface. By analyzing the three components of the shape-changing interface and their interrelationships, it is clear that the perceptron and the interaction are the key points to promote the integration between the intelligent technologies and the shape-changing interface. For the shape-changing interface design with the user-defined form, the process of integrating the inertial sensor network into the shape-changing interface based on the modular design method is investigated. For the shape-changing interface design with the self-sensing capability, a general process of accurate deformation recognition based on machine learning is introduced and a rapid prototyping platform is developed to improve the design efficiency. To evaluate the effectiveness of shape-changing interface design with the self-sensing capability, a comparative analysis is made on the results of deformation recognition in different interfaces. To evaluate the availability of the shape-changing interface design with the user-defined form, three experiments are conducted on the real-time performance of the sensor network, the energy consumption of the sensor hardware, and the hardware cost. Finally, three design examples of the shape-changing interface in the fields of flexible wearable devices, smart interactive toys, and new-type input control devices are introduced, which show that the above design methods can effectively develop the shape-changing interfaces with corresponding shape and function, and also broaden the shape-changing application areas towards more diversified scenarios.

Key words: shape-changing interface, sensor network, machine learning, rapid prototyping

中图分类号:

TH122
TP391

周子洪, 尤伟涛, 陆胤瑜, 郗宇凡, 王可幸, 王冠云, 孙凌云. 支持形态自定义和自感知的可变形界面设计方法[J]. 机械工程学报, 2023, 59(11): 84-93.

ZHOU Zihong, YOU Weitao, LU Yinyu, XI Yufan, WANG Kexing, WANG Guanyun, SUN Lingyun. Shape-changing Interface Design with the User-defined Form and Self-sensing Capability[J]. Journal of Mechanical Engineering, 2023, 59(11): 84-93.

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: http://www.cjmenet.com.cn/CN/10.3901/JME.2023.11.084

http://www.cjmenet.com.cn/CN/Y2023/V59/I11/84

参考文献

[1] RASMUSSEN M K，PEDERSEN E W，PETERSEN M G，et al. Shape-changing interfaces:a review of the design space and open research questions[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2012:735-744.
[2] STURDEE M，EVERITT A，LINDLEY J，et al. Visual methods for the design of shape-changing interfaces[C] //Human-Computer Interaction - INTERACT 2019. Cham:Springer International Publishing，2019，337-358.
[3] PAN Y. Heading toward artificial intelligence 2.0[J]. Engineering，2016，2(4):409-413.
[4] WANG G，QIN F，LIU H，et al. Morphingcircuit:an integrated design，simulation，and fabrication workflow for self-morphing electronics[J]. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.，2020，4(4):1-26.
[5] WANG G，YAO L，WANG W，et al. Xprint:a modularized liquid printer for smart materials deposition[C] //Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2016:5743-5752.
[6] LARSON C，SPJUT J，KNEPPER R，et al. A deformable interface for human touch recognition using stretchable carbon nanotube dielectric elastomer sensors and deep neural networks[J]. Soft Robotics，2019，6(5):611-620.
[7] ZHU Z，PARK H S，MCALPINE M C. 3d printed deformable sensors[J]. Science Advances，2020，6(25):eaba5575.
[8] CHAMPAGNE A，PRITCHARD B，DIETZ P，et al. Investigation of a novel shape sensor for musical expression[C]//The International Conference on New Interfaces for Musical Expression，2021. https://nime.pubpub.org/pub/bu2jb1d6.
[9] GONG J，SEOW O，HONNET C，et al. Metasense:integrating sensing capabilities into mechanical metamaterial[C]//The 34th Annual ACM Symposium on User Interface Software and Technology. New York，NY，USA:Association for Computing Machinery，2021:1063-1073.
[10] OLWAL A，STARNER T，MAININI G. E-textile microinteractions:augmenting twist with flick，slide and grasp gestures for soft electronics[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2020:1-13.
[11] TAHOUNI Y，QAMAR I P S，MUELLER S. Nurbsforms:a modular shape-changing interface for prototyping curved surfaces[C]//Proceedings of the Fourteenth International Conference on Tangible，Embedded，and Embodied Interaction. New York，NY，USA:Association for Computing Machinery，2020:403-409.
[12] SHAHMIRI F，CHEN C，WAGHMARE A，et al. Serpentine:a self-powered reversibly deformable cord sensor for human input[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2019:1-14.
[13] SCHMITZ M，STEIMLE J，HUBER J，et al. Flexibles:deformation-aware 3d-printed tangibles for capacitive touchscreens[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2017:1001-1014.
[14] OLWAL A，MOELLER J，PRIEST-DORMAN G，et al. I/O braid:scalable touch-sensitive lighted cords using spiraling，repeating sensing textiles and fiber optics[C]//Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. New York，NY，USA:Association for Computing Machinery，2018:485-497.
[15] ZHONG C，WAKKARY R，ODOM W，et al. Transtexture lamp:composing a deformable device as a computational whole[C]//Designing Interactive Systems Conference 2021. New York，NY，USA:Association for Computing Machinery，2021:1291-1300.
[16] ALEXANDER J，ROUDAUT A，STEIMLE J，et al. Grand challenges in shape-changing interface research[C]//Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2018:1-14.
[17] 米海鹏，王濛，卢秋宇，等. 实物用户界面:起源、发展与研究趋势[J]. 中国科学(信息科学)，2018，48(4):390-405. MI Haipeng，WANG Meng，LU Qiuyu，et al. Tangible user interface:origins，development，and future trends[J]. Science in China (Information Sciences)，2018，48(4):390-405.
[18] BOEM A，TROIANO G M. Non-rigid hci:a review of deformable interfaces and input[C]//Proceedings of the 2019 on Designing Interactive Systems Conference. New York，NY，USA:Association for Computing Machinery，2019:885-906.
[19] 高一聪，曾思远，冯毅雄，等. 支持4D打印的可控变形结构设计研究进展[J]. 机械工程学报，2020，56(15):26-38. GAO Yicong，ZENG Siyuan，FENG Yixiong，et al. Review of design of programmable morphing composite structures by 4D printing[J]. Journal of Mechanical Engineering，2020，56(15):26-38.
[20] MILLER R B. Response time in man-computer conversational transactions[C]//Proceedings of the December 9-11，1968，Fall Joint Computer Conference，Part I. New York，NY，USA:Association for Computing Machinery，1968:267-277.
[21] TSENG T，MURAI Y，FREED N，et al. PlushPal:storytelling with interactive plush toys and machine learning[C]//Interaction Design and Children. New York，NY，USA:Association for Computing Machinery，2021:236-245.

支持形态自定义和自感知的可变形界面设计方法

Shape-changing Interface Design with the User-defined Form and Self-sensing Capability

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	杨毅晟, 李明, 杨泽源, 闫喜强, 严思杰, 丁汉. 数字技术赋能风洞全生命周期体系[J]. 机械工程学报, 2026, 62(7): 97-113.
[2]	李桂伟, 曹祺原, 王嘉庆, 赵一航, 吴文征. 先进材料增材制造研究与发展[J]. 机械工程学报, 2026, 62(3): 15-45.
[3]	詹远新, 林勤龙, 刘洋, 高英, 吴剑明, 张嘉振. 面向Ti-6Al-4V合金增材制造的机器学习研究进展[J]. 机械工程学报, 2026, 62(3): 86-103.
[4]	陈瑞, 董明, 任明, 张崇兴, 王若谷. 锂离子电池内部温度检测与估计研究进展[J]. 机械工程学报, 2026, 62(2): 367-384.
[5]	刘海波, 邓平, 迟庆宇, 刘天然, 刘阔, 李特, 黄祖广, 刘行健, 薄其乐, STEVEN Y LIANG, 王永青. 智能加工中数据使能技术与应用[J]. 机械工程学报, 2026, 62(2): 407-444.
[6]	丁梦珂, 高东岳, 顾海洋, 王博睿, 周国泰, 武湛君. 基于BO-CNN模型的复合材料疲劳损伤超声导波监测方法[J]. 机械工程学报, 2025, 61(24): 38-47.
[7]	柴馨雪, 陶奕良, 祖洪飞, 徐灵敏, 李秦川. 基于误差模型与机器学习结合的2PRU-PSR并联机器人误差补偿方法[J]. 机械工程学报, 2025, 61(21): 192-203.
[8]	牛帅, 仝晓萌, 蔡茂林, 李毅波, 岳烜德. 面向智能制造的数控加工工艺重用关键技术：系统回顾[J]. 机械工程学报, 2025, 61(20): 301-317.
[9]	徐自力, 高京京, 覃曼青, 何孟夫. 基于混合机器学习模型的两级加载下金属材料的剩余疲劳寿命预测方法[J]. 机械工程学报, 2025, 61(12): 73-82.
[10]	张赛凡, 李博, 轩福贞. 激光选区熔化过程声发射信号的降噪与分类预测方法[J]. 机械工程学报, 2024, 60(6): 163-176.
[11]	牛静宜, 鲁思伟, 张倍宁, 杨春成, 李涤尘. 变组分复合材料3D打印工艺中机器学习算法对工艺参数预测有效性研究[J]. 机械工程学报, 2024, 60(21): 263-274.
[12]	褚端峰, 刘鸿祥, 高博麟, 王金湘, 殷国栋. 车辆队列预测巡航控制研究综述[J]. 机械工程学报, 2024, 60(18): 218-246.
[13]	刘建琴, 胡桂菘, 乔金丽, 郭晓. 基于双路神经网络的TBM实时性能预测研究[J]. 机械工程学报, 2024, 60(16): 43-53.
[14]	何洪文, 王浩宇, 王勇, 李双歧. 基于实车运行数据的锂离子电池健康状态估计[J]. 机械工程学报, 2023, 59(22): 46-58.
[15]	张墅野, 段晓康, 罗克宇, 许孙武, 张志昊, 陈捷狮, 何鹏. 基于机器学习电子封装互连材料研究现状[J]. 机械工程学报, 2023, 59(22): 222-233.