Some of the most incendiary works of AI-enabled film, however, exist outside both commercial and nonprofit spheres. The wide breadth of playful mashups and advocacy-oriented PSAs that populate social media and video-sharing platforms serve as R&D for the film and TV industry (and at times offer studios fresh talent), as well as advancing trenchant social critiques. Deepfake satire, in particular, constitutes a form of high-concept culture jamming, mocking duplicitous tech entrepreneurs and authoritarian leaders.
The proliferation of “generative” text-to-image and text-to-video software is pushing AI into a new phase. Instead of looking at the visuals from models like DALL-E 2, Midjourney, and Stable Diffusion as fixed and isolated objects, we could consider how these tools might operate as part of a larger creative process.
Synthetic imagery is sure to become central to preproduction. Screenwriters will be able to use AI-generated imagery for their pitch decks to evocatively establish the mood and feel of a project and position it within a larger genre. Likewise, concept artists will benefit from the back-and-forth tweaking of prompts and visual outputs as they flesh out a film’s narrative arc in the early stages of storyboarding. Generative AI might also expand the “previs” process of transforming flat images of material environments and character interaction into 3D approximations of scenes.
Generative image software also offers artists new ways to connect with audiences. For example, German technologist Fabian Stelzer engaged with Midjourney, Stable Diffusion, and DALL-E 2 to create a core set of images for a sci-fi fantasy about the discovery of a malicious salt substance in outer space. He also used GPT-3 to assist with the script and voice-generators Murf and Synthesia for the narration. Stelzer then published short sequences of this film, SALT, on Twitter (what he calls “story seeds”), along with a call for followers to vote on further developments in the unfolding narrative. The result is a crowdsourced essay film that takes some loose formal and thematic inspiration from Chris Marker’s landmark experimental essay film, La Jetée (1962).
Generative AI also offers a way to blend graphics and performance, breathing life into genres such as the music video. Belfast-based computational artist Glenn Marshall won the Jury Award at the Cannes Short Film Festival for his AI-driven film, The Crow (2022). Marshall started with Duncan McDowall’s captivating video of Dorotea Saykaly dancing in a black shawl in a decaying industrial building. He then fed frames of the video to OpenAI’s CLIP, a neural network model. The result is a work of clever style transfer, where every live-action frame is given a painterly treatment that transforms the dancer into a crow-like figure, encouraging viewers to reflect on the relationship between human and nonhuman animals, technological process and aesthetic experience.
Part of what is exciting about these projects is their strangeness and messiness, the ways they flaunt their own fabrication. These projects point to the productive frictions of mixed-media and cross-platform practices.
Much of the utopian rhetoric around generative AI has to do what it means for the individual maker. Never before has an amateur or seasoned professional been able to build such an elaborate project on such a small budget in such a short amount of time. This line of thinking resonates with the long history of mythmaking within Silicon Valley and Hollywood, two places that love and love to loathe their geniuses.
The more promising dimensions of this software and the debate surrounding its use, however, have less to do with visions of reinvigorated auteurism than the fact that it requires a constellation of skilled labor, not only to make it, but also to deploy it in innovative fashion. We might see a broader reframing of authorship as VFX supervisors—not to mention the computer scientists, concept artists, engineers, and animators they work with—become increasingly responsible for the movements and expressions of the characters on screen, as well as the look and feel of the world they inhabit. It remains to be seen whether this is an inherent feature of this new media or a first stage, much like the early stages of cinema itself.