stable-dreamfusion

Dreamfusion -- text-to-3d seems like it might be useful here eventually: https://dreamfusion3d.github.io/ (once successfully opensourced, see https://github.com/ashawkey/stable-dreamfusion) Rigging also looks like it could have a decent AI/DNN solution: https://arxiv.org/pdf/2005.00559.pdf

Magic3D looks like an improvement on DreamFusion [1], so it's sad to see that the code and models are not being made public. What is public right now is StableDreamFusion [2]. It produces surprisingly good results on radially symmetrical organic objects like flowers and pineapples. You can run it on your own GPU or a colab. Or, if you just want to type a prompt into a website and see something in 3D, try our demo at https://holovolo.tv [1] https://dreamfusion3d.github.io/ [2] https://github.com/ashawkey/stable-dreamfusion [3] https://holovolo.tv

What we really need is a MUD that is fed through StableDiffusion -> NeRF and outputs a 3D map for you to play in https://github.com/ashawkey/stable-dreamfusion

Key step in generating 3D – ask Stable Diffusion to score views from different angles: for d in ['front', 'side', 'back', 'side', 'overhead', 'bottom']: text = f"{ref_text}, {d} view" https://github.com/ashawkey/stable-dreamfusion/blob/0cb8c0e0...

There's a small gallery of success and failure cases here [1]. It certainly doesn't look as good as the original, yet. I wonder if that's due to the implementation differences noted, less cherry picking in what they show, or inherent differences between Imagen and Stable Diffusion. Maybe Imagen just has a much better grasp of how images translate to actual objects, where Stable Diffusion is stuck more on the 2d image plane. 1: https://github.com/ashawkey/stable-dreamfusion/issues/1