Google researchers have unveiled plans for a groundbreaking time-and-space diffusion model known as “Lumiere,” which can transform text or images into realistic AI-generated videos and offers on-demand editing capabilities.
Lumiere’s core feature is its “Space-Time U-Net architecture,” which enables the generation of realistic, diverse, and coherent motion in videos with a single pass through the model. The researchers explained that the model combines spatial and temporal down- and up-sampling while leveraging a pre-trained text-to-image diffusion model. This approach allows Lumiere to directly create full-frame-rate, low-resolution videos from textual descriptions or still images with prompts.
Users have drawn parallels between Lumiere and ChatGPT, noting that it offers similar capabilities but for text and image-to-video generation, stylization, editing, and animation.
While other AI video generators exist, such as Pika and Runway, what sets Lumiere apart is its novel single-pass approach to handling the temporal data dimension in video generation. Social media platforms have been buzzing with excitement about this development, with users calling it an “incredible breakthrough” and anticipating a transformative year in video generation.
Lumiere was trained on a vast dataset comprising 30 million videos and text captions, enabling it to generate 80 frames at 16 frames per second. However, the source of the data used by Google to train the model has not been disclosed, raising concerns related to copyright and AI.
The world of AI has witnessed numerous copyright infringement-related lawsuits, particularly concerning the use of copyrighted content during model training. One notable case involved The New York Times filing a lawsuit against Microsoft and OpenAI, the creator of ChatGPT, alleging the “illegal” use of its content for training purposes.
As Lumiere represents a significant advancement in AI-generated video technology, it remains to be seen how copyright and intellectual property concerns will be addressed in the evolving landscape of generative AI models.
![Pinterest](https://pinterest.com/pin/create/button/?url=https://cryptoshrypto.com/google-unveils-lumiere-ai-generated-video-from-text-and-images/&media=https://cryptoshrypto.com/wp-content/uploads/2024/01/photo_2024-01-25_23-24-16.jpg&description=Google researchers have unveiled plans for a groundbreaking time-and-space diffusion model known as "Lumiere," which can transform text or images){kind=link}