Technical Papers Fast-Forward
Neural State Machine for Character-Scene Interactions
Event Type
Technical Papers Fast-Forward
Registration Categories
TimeSunday, 17 November 201919:14 - 19:15
LocationGreat Hall 1&2
DescriptionWe propose Neural State Machine, a novel data-driven framework to guide characters to achieve goal-driven actions with precise scene interactions.
Even a seemingly simple task such as sitting on a chair is notoriously hard to model with supervised learning. This difficulty is
because such a task involves complex planning with periodic and non-periodic motions reacting to the scene geometry
to precisely position and orient the character.
Our proposed deep auto-regressive framework enables modeling of multi-modal scene interaction behaviors purely from data.
Given high-level instructions such as the goal location and the action to be launched there,
our system computes a series of movements and transitions to reach the goal in the desired state.
To allow characters to adapt to a wide range of geometry such as different shapes of furniture and obstacles,
we incorporate an efficient data augmentation scheme to randomly switch the 3D geometry while maintaining the context
of the original motion. To increase the precision to reach the goal during runtime, we introduce a control scheme that
combines egocentric inference and goal-centric inference.
We demonstrate the versatility of our model with various scene interaction tasks such as sitting on a chair,
avoiding obstacles, opening and entering through a door, and picking and carrying objects generated in real-time
just from a single model.