On July 13, "Miraculous Adventures in the Mythical World," China's first original fantasy micro-drama generated by artificial intelligence (AI), premiered on the short video platform Kuaishou. Five days earlier, Douyin had debuted "Future Revelations of Sanxingdui," the first sci-fi short drama created using AI, powered by its proprietary "Dream" AI model.
Within a week, Douyin and Kuaishou had both released AI-generated short dramas, signaling a clear intention to invest in this cutting-edge technology.
So, how are these AI-generated short dramas made? And what impact will the increasing use of AI video models have on the industry? We spoke with Chen Kun, director of "Miraculous Adventures in the Mythical World," and Wang Pengfei, head of Kuaishou's Visual Generation and Interaction Center, to explore these questions.
AI's Role in Short Drama Production
Traditionally, producing a short drama involves multiple stages: scriptwriting, team formation, actor selection, filming, and post-production. Chen Kun explained that AI does not participate in the entire process but primarily assists in the mid-production phase. Scriptwriting and post-production still require human involvement.
During scriptwriting, AI tools like GPT can provide creative ideas, but experienced screenwriters are still needed to develop fully realized stories.
In the mid-production phase, or the actual filming stage, AI can be used to generate visuals without relying on human actors or green screens. This eliminates the risk of "actor collapses." "Miraculous Adventures in the Mythical World" was produced entirely through a process of "text-to-image" and "image-to-video."
According to Chen Kun, the most labor-intensive part of the process is "text-to-image" and "image-to-video." A single AI tool cannot handle the entire process. Applications like "Midjourney" support the "image-to-video" phase, while "Kaleidoscope" and "Runway" provide assistance during the "image-to-video" stage.
The production team of "Miraculous Adventures in the Mythical World" began preparations in the first quarter of this year and only had a first version of the film in May. Due to technical limitations at the time, they were not satisfied with the final effect and kept revising it. In June, the "Kaleidoscope" AI model was released, and Chen Kun's team collaborated with them, leading to a complete overhaul of the film. 99% of the frames were generated by the "Kaleidoscope" model.
Regarding the impact of the revamp, Chen Kun expressed surprise at some of the generated scenes. For example, there is a scene where the god Gonggong emerges from the mountain and stands up, a task that other model software found challenging to generate.
While AI excels at creating special effects, it still has technical shortcomings in areas such as character performance, action interaction, and scene consistency. For instance, it is still difficult for AI to replicate subtle facial expressions. In "Miraculous Adventures in the Mythical World," a scene where a fish turns into a bird can only be done using a first and last frame technique.
Compared to traditional film and television, AI film and television find it challenging to express complex narratives through vivid performances. Many action scenes require montage editing. In both "Miraculous Adventures in the Mythical World" and "Future Revelations of Sanxingdui," the scenes used in the series are generally around 5 seconds long.
Viewers can sense the "cut up" nature of the films during viewing. Currently, the post-production process still relies heavily on human editors, and voiceovers require "voice gods" to accurately express complex emotions. AI-generated music and sound effects can already be used directly in short dramas.
In summary, the application of AI in the film and television industry is still in its early stages, with a gradual climb ahead. Wang Pengfei of Kuaishou informed us that they are continuously upgrading, iterating, and updating their AI models, and the effects today might be different from those a week later.
Can AI-Generated Short Dramas Become a New Industry?
In February, the release of OpenAI's "Sora" AI-generated video model sparked discussions about how AI could disrupt the traditional film and television industry. Although "Sora" is not yet open to the public, AI's involvement in the film and television industry is increasing, and filmmakers are starting to utilize AI technology.
AI has the potential to significantly improve efficiency and reduce costs in the industry. Under the traditional film and television model, producing a short drama requires a team of over a hundred people, whereas it can now be done with just ten or so people. Chen Kun revealed that, depending on the project's price and production quality, the use of AI can reduce costs by 1/n (where n is at least 4).
Chen Kun's Star Xian Culture has transformed from a traditional film and television company into an AI-native film and television company, focusing on AI-generated content. Bona has established the "AI Production Center," aiming to leverage AI technology to develop content IP.
During the development of the sci-fi short drama "Future Revelations of Sanxingdui," Bona needed a product to provide AI technical support, while Douyin's "Dream" AI sought to showcase its video capabilities. The two parties came together seamlessly. Over five months, they produced 13 AI short dramas, totaling 52 minutes. Thus, the two major industry trends, AI and short dramas, have found each other.
Similarly, Kuaishou's AI short dramas followed this path. According to the head of Kuaishou's entertainment business, Chen Yiying, there are now 270 million users watching short dramas on Kuaishou daily, consuming around 2 hours of content. The existing Star芒 short drama content supply is insufficient. In this situation, either more creators need to be added, or the efficiency of existing creators needs to be increased. AI is the solution to both of these paths.
Chen Kun's actions and Kuaishou's direction are highly aligned. Together, they produced the fantasy short drama "Miraculous Adventures in the Mythical World." Chen Kun believes that AI-generated film and television are in their early stages, and it is essential to choose the industry's pain points for creation. Within traditional film and television, the fantasy and sci-fi genres are limited by high costs and long production cycles for CG special effects, hindering their development. AI, on the other hand, can achieve this more easily.
As for whether AI-generated short dramas can grow into a new industry, Chen Yiying's response is cautious. She states that the scaling of content like fantasy and sci-fi in AI-generated short dramas depends on two factors: first, whether C-end users are interested in it, and second, whether there is a business model. Only when both these conditions are met can it be said that it might be a new industry.
Currently, "Miraculous Adventures in the Mythical World" has updated one episode, with 950,000 views. "Future Revelations of Sanxingdui" has updated six episodes, accumulating 10.72 million views, but it has not yet gone viral. This is also related to the platform's promotional efforts. Both dramas are free to watch and have not yet started commercialization.
At this stage, the "first foray" into fully AI-generated short dramas serves a greater purpose in showcasing AI technology, allowing the industry to see the limits of current AI technology and the potential for the future.
Chen Kun refers to AI-generated short dramas as a "new species," and its future development is uncertain. Just as short videos first emerged, no one knew what the final business model for live streaming sales would be. Standing today, we cannot see the final outcome of AI-generated short dramas, but it is right to keep pushing forward.