Shape-changing Interface Design with the User-defined Form and Self-sensing Capability

doi:10.3901/JME.2023.11.084

Abstract

Abstract: Shape-changing interface is a new-type human-computer interface. In the era of intelligence, how to apply emerging technologies to promote the development of shape-changing interfaces towards intelligence has become the research frontier of the human-computer interface. By analyzing the three components of the shape-changing interface and their interrelationships, it is clear that the perceptron and the interaction are the key points to promote the integration between the intelligent technologies and the shape-changing interface. For the shape-changing interface design with the user-defined form, the process of integrating the inertial sensor network into the shape-changing interface based on the modular design method is investigated. For the shape-changing interface design with the self-sensing capability, a general process of accurate deformation recognition based on machine learning is introduced and a rapid prototyping platform is developed to improve the design efficiency. To evaluate the effectiveness of shape-changing interface design with the self-sensing capability, a comparative analysis is made on the results of deformation recognition in different interfaces. To evaluate the availability of the shape-changing interface design with the user-defined form, three experiments are conducted on the real-time performance of the sensor network, the energy consumption of the sensor hardware, and the hardware cost. Finally, three design examples of the shape-changing interface in the fields of flexible wearable devices, smart interactive toys, and new-type input control devices are introduced, which show that the above design methods can effectively develop the shape-changing interfaces with corresponding shape and function, and also broaden the shape-changing application areas towards more diversified scenarios.

Key words: shape-changing interface, sensor network, machine learning, rapid prototyping

CLC Number:

TH122
TP391

ZHOU Zihong, YOU Weitao, LU Yinyu, XI Yufan, WANG Kexing, WANG Guanyun, SUN Lingyun. Shape-changing Interface Design with the User-defined Form and Self-sensing Capability[J]. Journal of Mechanical Engineering, 2023, 59(11): 84-93.

References

[1] RASMUSSEN M K，PEDERSEN E W，PETERSEN M G，et al. Shape-changing interfaces:a review of the design space and open research questions[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2012:735-744.
[2] STURDEE M，EVERITT A，LINDLEY J，et al. Visual methods for the design of shape-changing interfaces[C] //Human-Computer Interaction - INTERACT 2019. Cham:Springer International Publishing，2019，337-358.
[3] PAN Y. Heading toward artificial intelligence 2.0[J]. Engineering，2016，2(4):409-413.
[4] WANG G，QIN F，LIU H，et al. Morphingcircuit:an integrated design，simulation，and fabrication workflow for self-morphing electronics[J]. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.，2020，4(4):1-26.
[5] WANG G，YAO L，WANG W，et al. Xprint:a modularized liquid printer for smart materials deposition[C] //Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2016:5743-5752.
[6] LARSON C，SPJUT J，KNEPPER R，et al. A deformable interface for human touch recognition using stretchable carbon nanotube dielectric elastomer sensors and deep neural networks[J]. Soft Robotics，2019，6(5):611-620.
[7] ZHU Z，PARK H S，MCALPINE M C. 3d printed deformable sensors[J]. Science Advances，2020，6(25):eaba5575.
[8] CHAMPAGNE A，PRITCHARD B，DIETZ P，et al. Investigation of a novel shape sensor for musical expression[C]//The International Conference on New Interfaces for Musical Expression，2021. https://nime.pubpub.org/pub/bu2jb1d6.
[9] GONG J，SEOW O，HONNET C，et al. Metasense:integrating sensing capabilities into mechanical metamaterial[C]//The 34th Annual ACM Symposium on User Interface Software and Technology. New York，NY，USA:Association for Computing Machinery，2021:1063-1073.
[10] OLWAL A，STARNER T，MAININI G. E-textile microinteractions:augmenting twist with flick，slide and grasp gestures for soft electronics[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2020:1-13.
[11] TAHOUNI Y，QAMAR I P S，MUELLER S. Nurbsforms:a modular shape-changing interface for prototyping curved surfaces[C]//Proceedings of the Fourteenth International Conference on Tangible，Embedded，and Embodied Interaction. New York，NY，USA:Association for Computing Machinery，2020:403-409.
[12] SHAHMIRI F，CHEN C，WAGHMARE A，et al. Serpentine:a self-powered reversibly deformable cord sensor for human input[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2019:1-14.
[13] SCHMITZ M，STEIMLE J，HUBER J，et al. Flexibles:deformation-aware 3d-printed tangibles for capacitive touchscreens[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2017:1001-1014.
[14] OLWAL A，MOELLER J，PRIEST-DORMAN G，et al. I/O braid:scalable touch-sensitive lighted cords using spiraling，repeating sensing textiles and fiber optics[C]//Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. New York，NY，USA:Association for Computing Machinery，2018:485-497.
[15] ZHONG C，WAKKARY R，ODOM W，et al. Transtexture lamp:composing a deformable device as a computational whole[C]//Designing Interactive Systems Conference 2021. New York，NY，USA:Association for Computing Machinery，2021:1291-1300.
[16] ALEXANDER J，ROUDAUT A，STEIMLE J，et al. Grand challenges in shape-changing interface research[C]//Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York，NY，USA:Association for Computing Machinery，2018:1-14.
[17] 米海鹏，王濛，卢秋宇，等. 实物用户界面:起源、发展与研究趋势[J]. 中国科学(信息科学)，2018，48(4):390-405. MI Haipeng，WANG Meng，LU Qiuyu，et al. Tangible user interface:origins，development，and future trends[J]. Science in China (Information Sciences)，2018，48(4):390-405.
[18] BOEM A，TROIANO G M. Non-rigid hci:a review of deformable interfaces and input[C]//Proceedings of the 2019 on Designing Interactive Systems Conference. New York，NY，USA:Association for Computing Machinery，2019:885-906.
[19] 高一聪，曾思远，冯毅雄，等. 支持4D打印的可控变形结构设计研究进展[J]. 机械工程学报，2020，56(15):26-38. GAO Yicong，ZENG Siyuan，FENG Yixiong，et al. Review of design of programmable morphing composite structures by 4D printing[J]. Journal of Mechanical Engineering，2020，56(15):26-38.
[20] MILLER R B. Response time in man-computer conversational transactions[C]//Proceedings of the December 9-11，1968，Fall Joint Computer Conference，Part I. New York，NY，USA:Association for Computing Machinery，1968:267-277.
[21] TSENG T，MURAI Y，FREED N，et al. PlushPal:storytelling with interactive plush toys and machine learning[C]//Interaction Design and Children. New York，NY，USA:Association for Computing Machinery，2021:236-245.