Neural networks are able to create the face of non-existent people and even improve the quality of the video. Technology continues to evolve, and very quickly — it seemed impossible, but now removed the video people can be controlled using the computer keyboard. This was made possible thanks to the work of the team of developers of Facebook AI Research, which trained a neural network to recognize a moving person in the video, change the background image and control its movement. This is a must see — fortunately, developers have shared a detailed video.
Based on the technology invented by the developers of Facebook, are two of the neural network algorithm — Pose2Pose and Pose2Frame to migrate objects from one video to another. They also used the algorithm DensePose that turns two-dimensional image captured on video of a man in a three-dimensional model.
So, to turn an ordinary video into a computer game, DensePose algorithm creates a 3D model of a specific person. The algorithm then transforms Pose2Pose entry with the animation of a moving person in a video with a black background. At this stage the neural network is able to accept keyboard and move “game character” into the right direction. Finally, the algorithm Pose2Frame creates the final image with moving human and the corresponding animation.
The video also can be seen other examples of the neural network. Of course, we can also see many artifacts in the image, but the fact of the possibility to control the man from the video is very impressive. It is likely that this technology will find application in the gaming industry — how useful neural networks in the field of games and movies, read in our special article.
Impressed whether you are a new neural network from Facebook? Your opinion feel free to share in the comments or in our Telegram chat.