Google Lumiere text-to-video generator is powered by AI

Getting your Trinity Audio player ready...

Google (NASDAQ: GOOGL) has announced a new artificial intelligence (AI) model designed to generate realistic videos with real-time editing functionalities.

Called “Lumiere,” Google’s researchers describe the new offering as a time-and-space diffusion model capable of turning text and image inputs into videos. Lumiere demonstrates proficiency in synthesizing videos in what industry experts describe as a “state-of-the-art” offering.

AI-generated videos have been criticized over their lack of realism, data limitations, bias, and fine-grained control, casting doubt over the future of the offering.

 “We introduce Lumiere — a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion — a pivotal challenge in video synthesis,” the paper read.

However, Google attempts to solve the challenge by introducing a Space-Time U-Net architecture to produce the temporal duration of the video. Previous video generation models generate keyframes before proceeding with a temporal super-resolution, a procedure fraught with several downsides.

“By deploying both spatial and (importantly) temporal down- and up-sampling and leveraging a pre-trained text-to-image diffusion model, our model learns to directly generate a full-frame-rate, low-resolution video by processing it in multiple space-time scales,” the report explained.

Use cases for Google’s new model include video edition, inpainting, and stylized generation, but it still comes with its own flaws. While Lumiere offers full-frame-rate videos, generated videos are of low resolution, but its versatility gives it an edge over other industry players.

Comparisons with video generators like Pika and Runway place Lumiere ahead of the pack, with early testers hailing the offering as the future of video generation.

Google’s researchers trained Lumiere with nearly 40 million text and video captions, but the exact sources for the data sets are unknown, as copyright issues continue to threaten the future of the emerging technology.

Solving AI copyright debacle

Since generative AI became mainstream, several AI developers have been dragged to court for infringing on the copyrights of creators in training their large language models (LLMs). Google, Meta (NASDAQ: META), OpenAI, and Anthropic are putting up their legal defenses in courts across the U.S., denying any wrongdoing and arguing fair use.

Several theories have been advanced to solve AI’s copyright issues, including a proposed integration of models with blockchain technology. Google says it will protect AI users against copyright claims stemming from third parties by bearing responsibility for legal risks.

In order for artificial intelligence (AI) to work right within the law and thrive in the face of growing challenges, it needs to integrate an enterprise blockchain system that ensures data input quality and ownership—allowing it to keep data safe while also guaranteeing the immutability of data. Check out CoinGeek’s coverage on this emerging tech to learn more why Enterprise blockchain will be the backbone of AI.

Watch: Artificial intelligence needs blockchain