2023-2024
In Progress
Teamwork: Yue Zhu, Zhiyuan Zhou, Jinlin Miao
Advisor: Dr. Haipeng Mi, Yijie Guo
While traditional methods in art education primarily focus on observation and imitation, they often fail to adequately support the imaginative and creative capacities of children. This paper introduces TangibleNegotiation, a novel child-AI co-creation system that integrates Tangible User Interfaces (TUIs) with Generative AI technologies to enhance imagination cultivation in child art education. Leveraging the capabilities of swarm user interfaces (SUIs) for both visualization and dynamic narrative, this system embeds an LLM-based agent for conversational and motion planning alongside real-time image-to-image generation. TangibleNegotiation offers a comprehensive pipeline with four interactive modalities: Pre-creation Tips, Real-time Conversation, Real-time Artwork Rendering, and Final Artwork Generation, each designed to foster an engaging and interactive learning environment. A pilot study involving semi-structured interviews with four elementary school art teachers suggests that the system effectively enhances children’s engagement and stimulates their imagination through dynamic, real-time artistic feedback. The findings highlight the potential of combining SUI with generative AI to make art education more accessible, inclusive, and effective in fostering artistic imagination.
Video Demo
Yue Zhu, Zhiyuan Zhou, Jinlin Miao, Haipeng Mi, and Yijie Guo. 2024. TangibleNegotiation: Probing Design Opportunities for Integration of Generative AI and Swarm Robotics for Imagination Cultivation in Child Art Education. In Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’24). Association for Computing Machinery, New York, NY, USA, 66–70. https://doi.org/10.1145/3675094.3677586