DODonHopkins·Feb 3, 2022
The wonderful thing is that since Blender is totally scriptable in Python, and Python is the lingua franca of machine learning, you can seamlessly and efficiently use Blender to do all kinds of cool things with machine learning and AI!
Both creating ML-enhanced content creation and editing tools of course, but also synthesizing images, videos, 2D and 3D content for ML training.
For example, you can use using GPU-accelerated ray tracing and procedural character animation in photorealistic environments to train body tracking and pose estimation networks, since you get the precise "ground truth" and pixel-perfect semantic segmentation for free without enlisting Mechanical Turk to tediously and unreliably annotate millions of video frames.
And you can use randomization and retexturing to plug in different body types, skin tones, hair styles,
Read the rest
clothing, etc, to train it to work with diverse sets of people, in a wide range of environments, camera angles, and lighting conditions.
BlenderAI : Artificial Intelligence in Blender:
https://download.blender.org/documentation/bc2008/BlenderAI....
Blender for AI Devs, pt. 1: Overview:
https://www.youtube.com/watch?v=UtRxC9HPGd0
Blender pipeline to generate images for deep learning (BlenderProc) - Maximilian Denninger:
https://www.youtube.com/watch?v=1AvY_iS6xQA
>Using blender to generate realistic images for ML training:
>This is a two parts talk. It is based on the papers "3D Scene Reconstruction from a Single Viewport" presented at ECCV 2020 and the "BlenderProc" paper. The speaker is the main author of both papers. This is the recording of part 2.
>Recording of part 1:
https://youtu.be/cGGm3Vjdp8s
>References to everything covered in the talk:
https://www.reddit.com/r/2D3DAI/comments/jbkjdy/references_f...
>Lecture abstract: We present BlenderProc, which is a modular procedural pipeline, helping in generating real looking images for the training of convolutional neural networks. These can be used in a variety of use cases including segmentation, depth, normal and pose estimation and many others. A key feature of our extension of blender is the simple to use modular pipeline, which was designed to be easily extendable. By offering standard modules, which cover a variety of scenarios, we provide a starting point on which new modules can be created.
>arxiv:
https://arxiv.org/abs/1911.01911
>git:
https://github.com/DLR-RM/BlenderProc