Stability AI, the creators of the famous Stable Diffusion image generator, has just introduced their AI-powered tool for videos. Stable Video Diffusion is now available in a limited version for researchers. The company promotes it as a state-of-the-art system, capable of generating high resolution videos.
“This next-generation generative AI video model represents an important step in our journey toward creating models for everyone of every type,” Stability AI said in a statement. Stable Video Diffusion is released in the form of two image-to-video models: each of them is capable of generating 14 and 25 frames, at customizable frame rates between 3 and 30 frames per second.
It is based on the Stable Diffusion image model, which gained it popularity and some lawsuits for using copyrighted data for its training. Stability AI explained that The training of the new video system went through three phases– Text-to-image pre-training, video pre-training and high-quality video fine-tuning.
The company claims that its tool was superior to commercial competitors RunwayML and Pika Labs in user preference studies. Stability AI explained that It is available as an open source model at GitHub and clarified that it is not yet intended as a commercial solution or for public use. The limited version for researchers will allow them to refine quality and safety aspects before its general release.
Stability AI will launch a web platform to generate videos
Stability AI highlights that Its video model can be easily adapted to various tasks later. For example, multi-view synthesis from a single image with fine-tuning on multi-view datasets. “We are planning a variety of models that build on and extend this foundation, similar to the ecosystem that has been built around Stable Diffusion,” he said in the statement.
The company also made available a waiting list of a new web platform, which will include a text-to-video interface. “This tool will showcase practical applications of Stable Video Diffusion in numerous sectors including advertising, education, entertainment and more,” he added.
Stability AI launched Stable 3D at the beginning of the month. It is an AI-powered application that generates textured 3D objects for game modeling and development platforms. This tool is available in private preview for customers. The company highlighted in its Blog which is designed to allow non-experts to generate draft-quality 3D models “in minutes.”
The Stable Diffusion copyright controversy continues to take its toll. One of his vice presidents, Ed Newton-Rex, resigned this month as director of audio at Stability AI. He disagreed with the company’s position that training with copyrighted works is considered “fair use.” This position won him a court fight with Getty Images, one of the most important photography agencies in the world.
Leave a Reply