Interactive Character Animation by Learning Multi-Objective Control

Interactive Character Animation by Learning Multi-Objective Control

Kyungho Lee1    Seyoung Lee1    Jehee Lee1

1 Seoul National University   

Recurrent neural network learned basketball rules and skills for interactive character animation.

We present an approach that learns to act from raw motion data for interactive character animation. Our motion generator takes a continuous stream of control inputs and generates the character¡¯s motion in an online manner. The key insight is modeling rich connections between a multitude of control objectives and a large repertoire of actions. The model is trained using Recurrent Neural Network conditioned to deal with spatiotemporal constraints and structural variabilities in human motion. We also present a new data augmentation method that allows the model to be learned even from a small to moderate amount of training data. The learning process is fully automatic if it learns the motion of a single character, and requires minimal user intervention if it deals with props and interaction between multiple characters.


Kyungho Lee, Seyoung Lee, and Jehee Lee. 2018.
Interactive Character Animation by Learning Multi-Objective Control.
ACM Trans. Graph. 37, 6, (SIGGRAPH Asia 2018)

Download Paper (4.0 MB)


Presented at SIGGRAPH ASIA 2018 in Character Animation session.

Download Slide (205 MB)

Source code


Demo video

Download Video (292 MB)    


This research was supported by the MSIP(Ministry of Science, ICT and Future Planning), Korea, under the SW STARLab support program (IITP-2017-0536-20170040) supervised by the IITP(Institute for Information communications Technology Promotion.