Video generation models as world simulators
Video generation models as world simulators

Video generation models as world simulators

Created
Feb 23, 2024 03:49 AM
Tags
technology
Type
文章
Date
Feb 15, 2024
Content
The study explores large-scale training of generative models on video data, using a transformer architecture. The model, Sora, can generate high-quality videos. This suggests potential for developing universal physical world simulators.
media
https://openai.com/research/video-generation-models-as-world-simulators
We explore large-scale training of generative models on video data. Specifically, we train text-conditional diffusion models jointly on videos and images of variable durations, resolutions and aspect ratios. We leverage a transformer architecture that operates on spacetime patches of video and image latent codes. Our largest model, Sora, is capable of generating a minute of high fidelity video. Our results suggest that scaling video generation models is a promising path towards building general purpose simulators of the physical world.
 
notion image