DensePose

I agree with your take on this. Facebook has done considerable work with their Detectron / Dense Pose projects which can UV unwrap human bodies using AI. (https://github.com/facebookresearch/DensePose) Another is the smpl / smplify-x project (https://github.com/vchoutas/smplify-x) Further, by utilizing generated (synthetic) data with 3d scanned human models, or generating your own with something like makehuman or daz3D, you should be able to find a way to UV unwrap a distorted sphere. I've had a lot of success with generating synthetic data like this with Blender 3D and training various neural networks (Mask RCNN / Deeplab for example) with various combinations of this synthetic and real datasets.

It's something in between the two possibilities you describe. Each human pixel in the image is labeled with an index and two coordinates: x, y (u and v are the traditional names, but think of 2D x, y coordinates) The index specifies which patch that pixel is on, and the x, y coordinates specify where in the patch the pixel is on. This is for a pre-specified set of patches that cover a human mesh. See https://github.com/facebookresearch/DensePose/blob/master/no... for more detail. So, no, it does not extrapolate the full mesh, but also for all human pixels, you are getting 3D information.

Yes. From https://github.com/facebookresearch/DensePose/blob/master/GE...: > DensePose should automatically download the model from the URL specified by the --wts argument

"real-time approach for mapping all human pixels of 2D RGB images to a 3D surface-based model of the body." "Our work shows that one can efficiently compute dense correspondences between 2D RGB images and 3D surface models for the human body. Unlike common works in human pose estimation that operate with 10 or 20 human joints (wrists, elbows, etc), this work accounts for the entirety of the human body, defined in terms more than 5000 nodes. The resulting speed and accuracy of our system accelerates connections with augmented and virtual reality. Earlier works on this problem would require computation in the order of minutes, initialization by an external system e.g. for human joint localization, while being particularly brittle. DensePose operates at multiple frames per second on a single GPU and can handle tens or even hundreds of humans