VideoPoet by Google Research is a cutting-edge program that transforms language models into powerful video generators. It leverages advanced components like MAGVIT V2 video tokenizer and SoundStream audio tokenizer to convert images, videos, and audio clips into a sequence of discrete codes. These codes, integrated with text-based language models, enable the tool to predict the next video or audio token in the sequence. VideoPoet offers a wide range of generative learning objectives, including text-to-video, image-to-video, video stylization, and more, showcasing its versatility in video synthesis.

VideoPoet stands out for its ability to generate high-quality videos in square or portrait orientation, catering to short-form content needs. With features like multitasking on video-centric inputs, object identity preservation, and interactive video editing capabilities, VideoPoet demonstrates how language models can be harnessed to create videos with remarkable temporal consistency.