Creativity in VR

This ICMI’19 Best Paper Runner Up article presents a speech and gesture-based intercommunication paradigm for creative design tasks in Virtual Reality. The multimodal approach showed several advantages, including better perceived usability, an increased sense of a mental state of complete immersion and absorption in an activity (flow) and a higher sense of presence in VR (presence).


Virtual reality (VR) has always been considered a promising medium to support designers with alternative working environments. However, graphical user interfaces tend to distribute visual attention between the user interface and the manipulated target objects, which can hinder the creative process. This article suggests an interaction paradigm based on language and gestures for creative design tasks in VR. We developed a multimodal toolbox (MTB) for VR-based design applications and compared it to a typical unimodal menu-based toolbox (UTB). The comparison uses a design-oriented use case and measures flow, usability and the feeling of presence as relevant characteristics for a VR-based design process. The multimodal approach (1) resulted in a lower perceived task duration and a higher reported feeling of flow. The approach (2) offered a higher intuitive use and a lower mental workload, being no slower than a UTB. The approach (3) also produced a greater sense of presence. Overall, our results confirm significant benefits of the proposed multimodal interaction paradigm and the developed MTB for important features of design processes in VR.

Here you can download the paper.

Here you can find information about the chair and here you can go directly to its director, Prof. Marc Erich Latoschik.

About the author

Written by:

"Paint that object yellow": Multimodal Interaction to Enhance Creativity During Design Tasks in VR, In Proceedings of the 21st ACM International Conference on Multimodal Interaction (ICMI '19), pp. 195-204. New York, NY, USA2019. Best Paper Runner-Up ????

Leave a Reply

Your email address will not be published. Required fields are marked *


Economy Kategorie-Icon


View category
Feuilleton Kategorie-Icon


View category
Meetings & Social Kategorie-Icon

Meetings & Social

View category
Science Kategorie-Icon


View category
Training & Education Kategorie-Icon

Training & Education

View category