To date, the main methods of creating realistic animation of virtual creatures are skeletal animation and the infamous Motion Capture (mocap). However, experts are constantly in search of new lower quality, but cheaper ways. And, quite possibly, this may help the neural network. One of them recently trained to recognize the movements of people and transfer them to virtual characters. Perfect for any video. Even videos from YouTube.
For the development of responsible scientists from the Laboratory of research in artificial intelligence at the University of California at Berkeley (USA) and the University of British Columbia (Canada). The new algorithm is called DeepMimic was created in early 2018, but the full presentation was held only recently at the conference SIGGRAPH Asia in 2018. According to the authors,
“Our approach is based on a thorough analysis and evaluation of certain poses and events. Neural networks with deep machine learning can perform qualitative analysis of any movements from publicly available sources such as YouTube. It has a huge potential for the development of tools for a quick and simple animation, just by requesting the video with the desired behavior on the Internet. Our algorithm is able to explore a wide range of skills including acrobatics and martial arts.”
The algorithm works quite simply: the system of “watching” a video in which is performed a certain action. Then the movie is broken down frame by frame and for each frame created a three-dimensional model. After that all models are “connected”, and a special subroutine monitors the ligaments between them and removes artifacts and smoothes out the “choppy” animation.
To evaluate the performance of the algorithm, you can demo video is available below. But I want to add that the result was really impressive, provided that it is not, of course, pre-prepared videos, and random videos from the Internet.
To discuss new technology to create animation and other news, you can in our chat in Telegram.
The neural network was trained to copy movements of people from videos on YouTube
Vladimir Kuznetsov