CogVideo

Been working on this for a bit now and would love to know what folks think about solutions like this. Have been hearing that 360-degree videos are much more engaging for E-Commerce brands, and the tech is just starting to get good enough. Would love any and all feedback here! The tech behind is right now is primarily driven by CogVideo: https://github.com/THUDM/CogVideo with some post processing but been working to make it much better and scale up resolution.

Depending on how you're measuring it, video generation will come in less than 12 months, or it's already here. - In the Video Diffusion paper[1] they generate 64x64 16 frame videos, model is not released, but an open source implementation using an imagen-like pipeline exists[2], no model is available publicly. - The CogVideo paper[3], which is essentially a huge transformer model, can generate 480x480 videos, the code for the paper and models are open source[4], but be warned that you need a huge GPU like an A100 to run the damned thing. The future of text to video generation will probably be video diffusion, i.e. using 3D UNets, or more likely a much more optimized version of a UNet. Improvements on diffusion models are happening on a daily basis, and we're probably a couple papers away from having a breakthrough in efficiency that we ca

Check out https://github.com/THUDM/CogVideo - progress is being made on coherent video generation. Characters and dialogue are effectively solved, just look at GPT-3. The entity behind StableDiffusion is also supporting generative music art, so let's see what is coming out of that: https://www.harmonai.org/ We are currently far away from generating a production quality movie with AI, but I don't think it's going to be nearly as long as a lifetime. In my opinion, we'll have high quality AI shorts within the decade.

It's out now : https://github.com/THUDM/CogVideo/blob/main/paper/CogVideo-a...