While performance based animation has been widely used in film and game production, we apply the similar technique for the recreational/artistic performance purposes to bring users to experience real-time natural interaction with virtual character. We present a real-time system which allows a user to interact with a synthetic virtual character animated based on the performance of a dancer hiding behind the scene. The virtual character responds to the user as if she can “see” and “listen” to the user. The character motion can further elicit computer generated animations based on motion detection. By presenting only the virtual character animated by our system to the observing audiences within an immersive virtual environment, we create the natural interaction between the virtual world and real world.
