支持形态自定义和自感知的可变形界面设计方法

doi:10.3901/JME.2023.11.084

机械工程学报 ›› 2023, Vol. 59 ›› Issue (11): 84-93.doi: 10.3901/JME.2023.11.084

• 特邀专栏:智能产品交互设计 • 上一篇下一篇

扫码分享

支持形态自定义和自感知的可变形界面设计方法

周子洪¹, 尤伟涛¹, 陆胤瑜¹, 郗宇凡¹, 王可幸¹, 王冠云¹, 孙凌云^1,2

1. 浙江大学-南方电网人工智能创新联合研究中心杭州 310027;
2. 浙江大学计算机辅助设计与图形学国家重点实验室杭州 310027

收稿日期:2021-12-01 修回日期:2022-06-22 出版日期:2023-06-05 发布日期:2023-07-19
通讯作者: 尤伟涛(通信作者),男,1989年出生,博士,特聘研究员,主要研究方向为智能设计和审美计算。E-mail:weitao_you@zju.edu.cn
作者简介:周子洪,男,1997年出生,博士研究生。主要研究方向为人机交互和智能产品设计。E-mail:zihongzhou@zju.edu.cn;陆胤瑜,男,2000年出生,硕士研究生。主要研究方向为智能产品设计。E-mail:buliugu6@outlook.com;郗宇凡,女,2000年出生,硕士研究生。主要研究方向为可持续设计。E-mail:xiyufan_echo@163.com;王可幸,女,1998年出生,硕士研究生。主要研究方向为工业设计。E-mail:wakexi328@163.com;王冠云,男,1987年出生,博士,研究员,博士研究生导师。主要研究方向为数字化设计与制造、实体交互界面、4D打印等。E-mail:guanyun@zju.edu.cn;孙凌云,男,1981年出生,博士,教授,博士研究生导师。主要研究方向为人工智能、设计智能、信息与交互设计。E-mail:sunly@zju.edu.cn
基金资助:
科技创新2030-“新一代人工智能”重大资助项目(2018AAA0100703)

Shape-changing Interface Design with the User-defined Form and Self-sensing Capability

ZHOU Zihong¹, YOU Weitao¹, LU Yinyu¹, XI Yufan¹, WANG Kexing¹, WANG Guanyun¹, SUN Lingyun^1,2

1. Zhejiang University-China Southern Power Grid Joint Research Centre on AI, Hangzhou 310027;
2. State Key Laboratory of CAD & CG at Zhejiang University, Hangzhou 310027

Received:2021-12-01 Revised:2022-06-22 Online:2023-06-05 Published:2023-07-19

摘要/Abstract

摘要： 可变形界面是一种新型的人机交互界面，智能时代下如何应用新兴技术推动可变形界面朝着智能化方向发展已成为人机交互界面研究的前沿。通过分析可变形界面的三大构成元素及其相互关系，明确了感知载体与交互方式是促进智能技术融入可变形界面的重点。围绕界面形态的自定义设计，探索了将模块化惯性传感器网络融入可变形界面的设计过程；围绕界面形态的自感知设计，探索了利用机器学习的方法实现界面形变动作的精准识别，并设计了相应的快速原型平台以提高设计效率。为评估自感知设计方法的有效性，就不同形态界面的形变动作识别结果做了对比分析；为评估形态自定义设计方法的可用性，就传感器网络的实时性、传感器硬件能耗以及硬件成本做了实验和讨论。最后，给出了可变形界面在柔性可穿戴设备、智能交互玩具、新型输入控制设备三个领域中的设计实例，表明上述设计方法能够有效开发具有相应形态功能的可变形界面，并推动可变形界面的应用场景朝更加多元化的方向发展。

关键词: 可变形界面, 传感器网络, 机器学习, 快速原型

Abstract: Shape-changing interface is a new-type human-computer interface. In the era of intelligence, how to apply emerging technologies to promote the development of shape-changing interfaces towards intelligence has become the research frontier of the human-computer interface. By analyzing the three components of the shape-changing interface and their interrelationships, it is clear that the perceptron and the interaction are the key points to promote the integration between the intelligent technologies and the shape-changing interface. For the shape-changing interface design with the user-defined form, the process of integrating the inertial sensor network into the shape-changing interface based on the modular design method is investigated. For the shape-changing interface design with the self-sensing capability, a general process of accurate deformation recognition based on machine learning is introduced and a rapid prototyping platform is developed to improve the design efficiency. To evaluate the effectiveness of shape-changing interface design with the self-sensing capability, a comparative analysis is made on the results of deformation recognition in different interfaces. To evaluate the availability of the shape-changing interface design with the user-defined form, three experiments are conducted on the real-time performance of the sensor network, the energy consumption of the sensor hardware, and the hardware cost. Finally, three design examples of the shape-changing interface in the fields of flexible wearable devices, smart interactive toys, and new-type input control devices are introduced, which show that the above design methods can effectively develop the shape-changing interfaces with corresponding shape and function, and also broaden the shape-changing application areas towards more diversified scenarios.

Key words: shape-changing interface, sensor network, machine learning, rapid prototyping

中图分类号:

TH122
TP391

周子洪, 尤伟涛, 陆胤瑜, 郗宇凡, 王可幸, 王冠云, 孙凌云. 支持形态自定义和自感知的可变形界面设计方法[J]. 机械工程学报, 2023, 59(11): 84-93.

ZHOU Zihong, YOU Weitao, LU Yinyu, XI Yufan, WANG Kexing, WANG Guanyun, SUN Lingyun. Shape-changing Interface Design with the User-defined Form and Self-sensing Capability[J]. Journal of Mechanical Engineering, 2023, 59(11): 84-93.

参考文献

[1] RASMUSSEN M K，PEDERSEN E W，PETERSEN M G，et al. Shape-changing interfaces:a review of the design space and open research questions[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2012:735-744.
[2] STURDEE M，EVERITT A，LINDLEY J，et al. Visual methods for the design of shape-changing interfaces[C] //Human-Computer Interaction - INTERACT 2019. Cham:Springer International Publishing，2019，337-358.
[3] PAN Y. Heading toward artificial intelligence 2.0[J]. Engineering，2016，2(4):409-413.
[4] WANG G，QIN F，LIU H，et al. Morphingcircuit:an integrated design，simulation，and fabrication workflow for self-morphing electronics[J]. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.，2020，4(4):1-26.
[5] WANG G，YAO L，WANG W，et al. Xprint:a modularized liquid printer for smart materials deposition[C] //Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2016:5743-5752.
[6] LARSON C，SPJUT J，KNEPPER R，et al. A deformable interface for human touch recognition using stretchable carbon nanotube dielectric elastomer sensors and deep neural networks[J]. Soft Robotics，2019，6(5):611-620.
[7] ZHU Z，PARK H S，MCALPINE M C. 3d printed deformable sensors[J]. Science Advances，2020，6(25):eaba5575.
[8] CHAMPAGNE A，PRITCHARD B，DIETZ P，et al. Investigation of a novel shape sensor for musical expression[C]//The International Conference on New Interfaces for Musical Expression，2021. https://nime.pubpub.org/pub/bu2jb1d6.
[9] GONG J，SEOW O，HONNET C，et al. Metasense:integrating sensing capabilities into mechanical metamaterial[C]//The 34th Annual ACM Symposium on User Interface Software and Technology. New York，NY，USA:Association for Computing Machinery，2021:1063-1073.
[10] OLWAL A，STARNER T，MAININI G. E-textile microinteractions:augmenting twist with flick，slide and grasp gestures for soft electronics[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2020:1-13.
[11] TAHOUNI Y，QAMAR I P S，MUELLER S. Nurbsforms:a modular shape-changing interface for prototyping curved surfaces[C]//Proceedings of the Fourteenth International Conference on Tangible，Embedded，and Embodied Interaction. New York，NY，USA:Association for Computing Machinery，2020:403-409.
[12] SHAHMIRI F，CHEN C，WAGHMARE A，et al. Serpentine:a self-powered reversibly deformable cord sensor for human input[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2019:1-14.
[13] SCHMITZ M，STEIMLE J，HUBER J，et al. Flexibles:deformation-aware 3d-printed tangibles for capacitive touchscreens[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2017:1001-1014.
[14] OLWAL A，MOELLER J，PRIEST-DORMAN G，et al. I/O braid:scalable touch-sensitive lighted cords using spiraling，repeating sensing textiles and fiber optics[C]//Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. New York，NY，USA:Association for Computing Machinery，2018:485-497.
[15] ZHONG C，WAKKARY R，ODOM W，et al. Transtexture lamp:composing a deformable device as a computational whole[C]//Designing Interactive Systems Conference 2021. New York，NY，USA:Association for Computing Machinery，2021:1291-1300.
[16] ALEXANDER J，ROUDAUT A，STEIMLE J，et al. Grand challenges in shape-changing interface research[C]//Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2018:1-14.
[17] 米海鹏，王濛，卢秋宇，等. 实物用户界面:起源、发展与研究趋势[J]. 中国科学(信息科学)，2018，48(4):390-405. MI Haipeng，WANG Meng，LU Qiuyu，et al. Tangible user interface:origins，development，and future trends[J]. Science in China (Information Sciences)，2018，48(4):390-405.
[18] BOEM A，TROIANO G M. Non-rigid hci:a review of deformable interfaces and input[C]//Proceedings of the 2019 on Designing Interactive Systems Conference. New York，NY，USA:Association for Computing Machinery，2019:885-906.
[19] 高一聪，曾思远，冯毅雄，等. 支持4D打印的可控变形结构设计研究进展[J]. 机械工程学报，2020，56(15):26-38. GAO Yicong，ZENG Siyuan，FENG Yixiong，et al. Review of design of programmable morphing composite structures by 4D printing[J]. Journal of Mechanical Engineering，2020，56(15):26-38.
[20] MILLER R B. Response time in man-computer conversational transactions[C]//Proceedings of the December 9-11，1968，Fall Joint Computer Conference，Part I. New York，NY，USA:Association for Computing Machinery，1968:267-277.
[21] TSENG T，MURAI Y，FREED N，et al. PlushPal:storytelling with interactive plush toys and machine learning[C]//Interaction Design and Children. New York，NY，USA:Association for Computing Machinery，2021:236-245.

支持形态自定义和自感知的可变形界面设计方法

Shape-changing Interface Design with the User-defined Form and Self-sensing Capability

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张赛凡, 李博, 轩福贞. 激光选区熔化过程声发射信号的降噪与分类预测方法[J]. 机械工程学报, 2024, 60(6): 163-176.
[2]	褚端峰, 刘鸿祥, 高博麟, 王金湘, 殷国栋. 车辆队列预测巡航控制研究综述[J]. 机械工程学报, 2024, 60(18): 218-246.
[3]	刘建琴, 胡桂菘, 乔金丽, 郭晓. 基于双路神经网络的TBM实时性能预测研究[J]. 机械工程学报, 2024, 60(16): 43-53.
[4]	何洪文, 王浩宇, 王勇, 李双歧. 基于实车运行数据的锂离子电池健康状态估计[J]. 机械工程学报, 2023, 59(22): 46-58.
[5]	张墅野, 段晓康, 罗克宇, 许孙武, 张志昊, 陈捷狮, 何鹏. 基于机器学习电子封装互连材料研究现状[J]. 机械工程学报, 2023, 59(22): 222-233.
[6]	王栓程, 杨冰, 廖贞, 肖守讷, 康国政, 阳光武, 朱涛. 金属材料疲劳短裂纹萌生与扩展研究综述[J]. 机械工程学报, 2023, 59(16): 32-53.
[7]	华丰, 王亚森, 金骏阳, 袁烨. 一种数据隐私保护下多方无损线性模型学习方法[J]. 机械工程学报, 2023, 59(12): 17-27.
[8]	管寅昕, 杨吉祥, 丁汉. 考虑控制电信号中间量的机床运动控制误差数据驱动建模方法[J]. 机械工程学报, 2022, 58(23): 178-187.
[9]	詹梅, 董赟达, 翟卓蕾, 樊晓光, 石志鹏, 安强. 塑性成形快速数值仿真方法的研究进展[J]. 机械工程学报, 2022, 58(16): 2-20.
[10]	刘源, 魏世忠. 数据驱动的钢铁耐磨材料性能预测研究综述[J]. 机械工程学报, 2022, 58(10): 31-50.
[11]	宰红斌, 刘建国, 唐保国, 马建国, 上官明霞, 单荣荣. 基于WSN的输电线路状态监测与数据采集跨层优化方法^*[J]. 电气工程学报, 2021, 16(3): 161-169.
[12]	赵健, 邓志辉, 朱冰, 常婷婷, 陈志成. 基于RBF网络滑模的电动助力制动系统液压力控制[J]. 机械工程学报, 2020, 56(24): 106-114.
[13]	孙博华, 邓伟文, 吴坚, 李雅欣. 虚拟随机车路场下驾驶人驾驶能力机理分析[J]. 机械工程学报, 2020, 56(16): 166-180.
[14]	胡晓松, 陈科坪, 唐小林, 王斌. 基于机器学习速度预测的并联混合动力车辆能量管理研究[J]. 机械工程学报, 2020, 56(16): 181-192.
[15]	高扬, 穆继亮, 何剑, 陈鸿, 穆锦标, 安站东, 赵新平, 丑修建. 煤机设备无线自供电状态监测系统[J]. 机械工程学报, 2020, 56(13): 41-49.