AVavaer·Dec 14, 2023
This looks a fine-tune of the classic zero123 (https://github.com/cvlab-columbia/zero123) I’m excited to check out the quality improvements.
Though 3d model synthesis is one use case, I found the less advertised base reprojection model to be more useful for gamedev at the moment. You can generate a multiview spritesheet from an image, and it’s fast enough for synthesis during a gameplay session. I couldn’t get a good quality/time balance to do the same with the 3d models, and the lack of mesh rigging or animation combined with imperfections in a fully 3d model tends to break the suspension of disbelief compared to what players are used to for full 3d. I’m sure this will change as the tech develops and we layer more AI on top (automatic animation synthesis is an active research area).
If you’re interested in this you might also want to che
Read the rest
ck out deforum (https://github.com/deforum-art/deforum-stable-diffusion) which provides even more powerful camera controls on top of stable diffusion designed for full scenes rather than single objects.