In the blink of an eye, AI art and illustration have gone from bizarre curiosity to a powerful, sometimes worrying force. With AI-generated images having fooled judges to win art and photo competitions, it seemed only a matter of time before the technology turned to video, and sure enough, it has.
An LA-based studio has just created what's probably the first anime film made using AI. That sounds like something of a landmark achievement, but for all the talk about democratising creativity, a lot of people aren't very impressed (see how to use DALL-E 2 for more on text-to-image generation).
Corridor Digital's (opens in new tab) Rock, Paper, Scissors is a seven-minute 'anime' that tells the epic story of two princes battling for a throne. But actually, it's not an anime because it wasn't animated. Rather an anime look was created using AI. The studio didn't use generative AI to create the imagery from scratch but a transformative AI to turn live action into what looks like a cartoon.
The process they developed to do so is ingenious and fascinating to follow (see the behind-the-scenes video below). So far, it's been difficult to apply generative AI to video because the noise generated in the diffusion process that's used to create each frame leads to a disjointed result. Corridor Crew found ways to overcome a bunch of hurdles, using new techniques to freeze the noise and apply character styles to keep things consistent. They then applied the deflicker plugin from DaVinci Resolve.
Is the result any good? Hell, no. It's terrible. The 'making of' video is a lot more watchable than the final anime itself. As many mention on Twitter, the film recalls bad rotoscoping. It has an uncanny valley feel with lots of vacant-looking expressions, inconsistent frames and an overall "air of wrongness." And that's before we come to the acting, which just feels like parody.
"The constantly morphing hair, inconsistent dead eyes, highlights randomly popping into shadows, all built on the work of humans who laboured over their craft to get the details right," someone tweeted (opens in new tab). "Not only is this a terrible terrible idea, but it actually hurts my eyes to look at it, real animators can do an amazing job, just hire them!" someone else wrote.
Congratulations, you just found out rotoscope... Max Fleischer did this 80 years ago, he didn't have AI but a team of skilled, hard-working animators.The AI rotoscope used all assets from Vampire D and other anime, which are well animated by animators. pic.twitter.com/mst1cacDYTFebruary 28, 2023
Of course, we should think of this as merely an initial experiment and another milestone on the way to nobody quite knows what. It's something that nobody's done before, and firsts often aren't very good. Corridor is encouraging others to use their process, and it will no doubt be refined and lead to better results. In the meantime, it's clear that Niko and Dean had an absolute ball screaming their lungs out and hamming it up in the acting.
But the Corridor Crew's endeavours will also raise concerns. Just last month Netflix copped a lot of flak for using AI to create the backgrounds in a short anime film, purportedly to "help the industry" amid a labour shortage. Corridor says it aims to "democratise creativity", which is a phrase that gets thrown about a lot in connection with AI. But it's a strange way to put it, making it sound like animators are running an elitist despotic cabal that the rest of us are unfairly denied entry to because of our lack of skill and talent.
'Democratise' is starting to sound more like a synonym for 'make cheaper'. For some people that could be great, Small studios could eventually make passable anime on the cheap without having to hire animators. Small bands could make their own animated music videos. For now though, it looks like it might also mean cheapening the quality of results.
Read more:
