Projects Last Projects

Motion Transfer, the task of reenacting the image of a person according to the movement of another, is an active research field in computer vision. Recent methods achieve realistic looking results in controlled scenarios. Yet it is difficult to obtain similar results for complex, crowded, in-the-wild scenes while integrating the reenacted person into the target scene. In this work, a novel workflow is proposed to tackle this challenging scenario, which we name Scene-Aware Motion Transfer (SMT). Our workflow harnesses a set of models, each attaining state-of-the-art results in its respective field, and is divided into two major stages. First, a novel person tracking pipeline is used to separate each unique identity from the crowd. Then the tracking results are utilized for a targeted single-person motion transfer, resulting in a fully automatic workflow that can handle complex videos. An extensive evaluation is presented to show the quality and robustness of the results in different scenarios.

Please, see demo clip.