Off center, wayward, slightly off path.... Rettberg and Luers discuss their longrunning encounters with writers, artists, computational film makers and other multidisciplinary "people who come to the electronic literature community, and it’s not only writers, but also artists, visual artists, and you find everyone has a similar kind of wayward path."
SR: What is post-human cinema and how is AI changing the way we make movies? Today I’ll be talking with Will Luers, an artist who’s been working on a collaborative project making AI cinema. Welcome, Will.
WL: Hi, Scott. How are you?
SR: Good. And you also teach at Washington State University in Vancouver.
WL: Yes, it’s an undergraduate program, Creative Media and Digital Culture. It was envisioned and is run by Dene Grigar, who was president of ELO for some time, and it’s through her that I came into this community.
SR: And I know that you have also been teaching a course about creativity in AI.
WL: Yes, I just finished that. We have special topics classes, and I had proposed an AI class. Dene wanted me to teach a class called "AI in the Arts." We go through the different modalities of generative AI and really get students to open up creatively to new processes and integrate it into what they are doing in their other classes. And to be critical, but also not to think of it in a scary way.
SR: Well, that’s great. And I think that sort of course is the kind of thing that we’re going to be seeing a lot of in the future. At least I hope so, that we’re mainly not having art courses and writing courses where people talk about the dangers of AI and then just dismiss it.
WL: That is happening. I have also tied this to an open education resource I created (with a grant) that is based on the class and hopefully some faculty will pick it up and see how they can use AI in the classroom. Just watching my students using these tools, they realize they can just do so much more. They are learning new skills. That’s the important thing, they’re learning coding, they’re learning to create images and other media.
SR: Great. Well, I want to talk more about AI, and particularly about the projects that you’ve been making recently with Mark Amerika, with AI cinema. But first, I’d like to sort of dial back a little bit and just about your journey as an artist, an author, an educator and researcher. So maybe just give us a little bit about where you came from, your earlier projects and the direction that you moved across your career.
WL: Sure. I was drawn to film early on, I guess in my teens. I took a Super 8 class in high school and learned to cut film, and I got into it, and later I got into video as well. But I was really more drawn to the experimental work. The video artist Bill Viola, when I was in my 20s, was big for me. But then, with Reagan, all the funding for the arts dried up...
SR: Yeah.
WL: And so, I traveled the world a bit. I taught English in Japan, Czech Republic, and I came back and applied to film school. And film school in the 90s was sort of that moment where... there was this thing called independent film.
SR: Yeah, Jarmusch
WL: Jarmusch and Spike Lee and all those folks, you know, and others, of course. So I went to film school at Columbia University in New York. While there, I kept pushing to move away from film and cutting on film, and to think about video. I had this idea that we were going to all be using video, not film. I mean, it wasn’t only my idea, but it was frowned upon by a lot of people. So, in the 90s, after finishing, I concentrated on screenwriting. I was writing screenplays, waiting tables or catering. I was still interested in film history and theory and teaching that at Parsons. Around the late 90s, with the dot-com bust and all that, I got the hint that people were going to do video on the web. I had some successes with screenwriting, but it didn’t really lead to anything financial. I remember I put a QuickTime video on a web page, and I was just testing out this whole thing that became video blogging. I got to know a global community of video bloggers before YouTube. Now, that was an extraordinary period. This is, you know, just after the dot-com—
SR: This is like early 2000.
WL: Its like 2000, 2001. People didn’t know what to do with the web anymore, but the artists knew what to do. So you had a community of people sharing their experimental videos, and it was amazing to think that if you make a little video and you get it screened in front of 100 people, that was supposed to be really good. But now you could get hundreds and hundreds of people to see it online. So, it was sort of an obvious thing. And I then learned how to do web design and I found my way to the 2008 ELO Conference. I think it was because Mark Amerika was giving the keynote, and I had been paying attention to what he was doing, really from the 90s, because he was doing this kind of storytelling with media that felt like what cinema could be.
SR: One of his projects was called Film—
WL: FILMTEXT, yeah. I went to the ELO (Electronic Literature Organization) Conference in Vancouver. I was living in Portland, and I met you, I met Mark, and I met Dene and everyone else that’s connected with this community. Dene actually hired me to teach, so I started teaching at that program, got more involved with ELO and ebr—
SR: The electronic book review.
WL: By the way, electronic book review, I was reading that early on. I think in the late 90s, because you just search terms and you’d find all that stuff, you know.
SR: It’s kind of a chronicle.
WL: Nobody was doing anything like that.
SR: Yeah.
WL: So anyway, so that was how I got into the community, and I think what’s exciting is, you meet people who come to the Electronic Literature Organization, and it’s not only writers, but also artists, visual artists, and you find everyone has a similar kind of wayward path.
SR: Slightly off path.
WL: Yeah, a lot of it has to do with the time and the opening of the web, but you’re trying to figure out, not what’s next, but what’s the openings of art forms and things like that, and it was really exciting for me, and I think for everybody, to be part of that.
SR: You, kind of early on, framed your work as digital cinema, experimental cinema.
WL: I ended up calling it computational cinema because I did learn to use JavaScript. I know you do a lot of computational video, as well. That was really interesting to me. Interactive cinema has not taken off, you know. It’s not a thing really, but... it’s still a thing.
SR: It’s an unknown thing.
WL: Yeah. But it’s also, I guess my interest is in, from my teaching and studying of film language, the way that images work - like a long shot, medium shot, close up - I saw that language in Olia Lialina’s My Boyfriend Came Back from the War, in a simple way.
SR: One of the first narrative artworks on the web.
WL: She came from a film program, so that was all exciting for me. I had a narrative bent, but still very experimental, and the net art works kind of combined with it. So, I got interested in randomness and things that were breaking up clear structures. I became interested in multiple paths, not only multiple paths of narrative, but also like multiple screens going on at the same time. The spatial montage that Lev Manovich talked about.
SR: And you’ve done a lot of collaboration over the years, particularly quite a bit with Hazel Smith and Roger Dean
WL: I met Hazel at an ELO Conference, at Brown I think, and we had some similar interests. She wrote about the New York Poets, and I was really into the New York Poets. We talked about poetry and then she and her husband, Roger Dean, a sound artist, invited me to do some work with them, and I think we’ve made four works total. The fourth was a smaller one with Roger, Hypnagogia.
SR: Five, I’m just counting on here.
WL: Oh my God. Okay.
SR: Including one that won The Robert Coover Award.
WL: That’s right, novelling. That was a more cinematic kind of work. Not really narrative, but it was a changing interface of video clips, and it was reflecting the experience of reading a novel and how you’re in an in-between space. That was the idea.
SR: It was a great piece. I think I was one of the judges or something like that.
WL: Well, thank you.
SR: Maybe we should talk about TDR first, The Digital Review, because you’ve been the managing editor for electronic book review for eight—
WL: Since 2018.
SR: So, for six years, and you’re sort of cycling out of that a little bit, although you’ll still be involved in ebr, and partially because you’ve now started up this new online magazine, going into the third year now, something that was a lot of peoples' dream for a long time, The Digital Review, which is more of a born digital peer reviewed journal.
WL: Let me just go back a little bit, because ebr was going through a transition, and I had an interest in digital publishing. I was teaching web publishing, and I approached Joe Tabbi, the founding editor, about moving ebr to WordPress. I took all this work on just because I really wanted ebr to last. And it got me in more deeply into digital publishing. Then Joe said he had a domain name called The Digital Review, and we should think about what we could do with that. So, TDR is based on a domain name. ebr, in the early days, in the 90s, really was multimodal writing, right? It had experimental essayistic and academic kinds of writing. And that kind of work had been lost...you would be able to tell me when ebr had to transition to more formal standards for publishing... around 2004, 2005 maybe?
SR: I can’t remember.
WL: Okay, something like that.
SR: But yeah, maybe even a bit later, thinking about the structures of academic publishing and making sure that, in a way, that these things were getting circulated in the academic environment and counting for people’s tenure case and things like that.
WL: Yeah. A lot of that kind of writing was lost. And so, I talked with the not only Joe but the other editors about what we can do to make this new Digital Review a kind of sibling publication that allowed scholars and artists to explore a different kind of writing. You know, it’s framed as multimodal essays, born digital essays, but that means you can still make art.
SR: There are maybe a couple others, like Kairos.
WL: And BeeHive.
SR: Yeah, in the past, but that had sort of gotten lost. So, you set up a separate editorial board to kind of spin this off.
WL: Yeah.
SR: Bringing together, I think, very sharp criticism in digital media along with actual digital art and digital narrative practice.
WL: I wrote a grant to get things going, and I had Holly Slocum, who was one of my students and works with Dene in the Electronic Literature Lab now. She helped me design the site and get it going, and the idea of the publication - The Digital Review - is that I find people, sometimes they come to me, to edit an issue focused on an area of their own research, or you know, an art practice or whatever. And so we’ve had issues on digital performance, digital humanities practice. The first one that I did was Digital Essayism, which is sort of going back to those forms of essay writing that involved computation or media.
SR: Are you going to do a couple issues a year, or...?
WL: It’s been once a year, but I’m finding that there are people who propose issues that are smaller, curated things rather than the submission process. So, the next one after our AI issue, which I edited, is going to be Player Stories, edited by Jentery Sayers, which is going to include walkthroughs of games, that kind of thing. It’s kind of loose in terms of what’s next. I mean, if it turns into biannual that’s fine. I think it works because the editors already have a structure in place, and they just need to get the idea going and get other people to contribute. I’ve worked it out so that it’s not a huge amount of labor for people to take it on.
SR: Yeah, and I’m excited, not only because I actually have a piece in it that I’ve been working on for a while, but I’m excited about this issue that has just come out a couple months ago, what did you end up—
WL: Well, the call had had a title "augmenting creativity", but I wanted the AI in it, so "AI-augmented Creativity" is the title.
SR: Excellent.
WL: Yeah, yeah. I think you suggested putting AI in—
SR: For search engine optimization.
WL: It’s all over the place right now, but you gotta do it.
SR: Let’s talk about that issue and some of the work that’s in it because it looks like there’s going to be quite a bit of different forms of experimentation.
WL: There are narratives like yours, Alan Bigelow’s and Mez Breeze, you know, experimental narratives. Then there are works that are using AI to probe certain kinds of data, say from nature, that can reflect back on our relationship to technology and living world, that kind of thing. And there’s someone who has written a kind of log about using AI to write a story, it’s sort of a—
SR: Auto ethnography.
WL: Auto ethnography, yeah.
SR: Who’s that?
WL: Lisa Farrell. So, there’s a range of—
SR: There’s the songbird—
WL: By Alinta Krauth.
SR: This is one of the projects that are sort of animal translators.
WL: Yeah, getting the audio from birds to produce language in some way. In AI, that’s an interesting whole area... of speaking with animals through translation. So, there’s a range of work and some of it is direct narratives. Yours has a long essay which is great. It explores the process of using these tools.
SR: And Jhave’s is kind of a video essay.
WL: Yeah, Jhave’s is a featured work where he has nine videos that get very philosophical about what AI means for the human.
SR: Does it have Posthuman Cinema?
WL: No, no, no, I can’t put my own work in it.
SR: Oh yeah. I’m not opposed to it when it's such a core, amazing piece. But, let’s turn to that piece, the work that you’ve been doing with Mark Amerika.
WL: And Chad Mossholder is our sound designer.
SR: Mark Amerika has been doing this work, he sounds like he sort of inspired your voyage into working in this environment. He beat The Unknown to be one of the first major web hypertext projects.
WL: What year was The Unknown?
SR: ‘99. Bobby Arellano, who was on here, and who visited us last year, had the first web hypertext novel. But then along came GRAMMATRON, which was a huge thing, culturally, I’d say. It was in The Whitney Biennial in 99, or something. There was that amazing kind of cluster of things happening at the very end of the 90s. He’s been doing this sort of work that crosses poetics, philosophy, writing, digital cinema, a variety of different forms of remix. So, he’s had this long career and you and him have been in conversation for a long time, and then AI comes along. Tell us a little bit about Posthuman Cinema.
WL: Mark had written a book. I have to admit, early AI work, I didn’t quite get it. Like, creatively, I didn’t get it. But then I read his book My Life as an Artificial Creative Intelligence and I found it fascinating because he talks about how an artist or writer would use these tools to explore and improvise. It’s very much like all of his work, taking a medium and playing with it. So, we talked about doing something with cinema or video because a new tool, RunwayML, came out. My first time playing around with it I was pretty amazed at what it could do, really quite excited about it. And so we exchanged clips trying to figure out where we were going to go for a collaboration.
SR: And these are sending video clips back and forth, using Runway?
WL: Yeah, using Runway.
SR: Which generates these, how long—
WL: Like four seconds or so. You can extend them, but at that point I don’t think we could. It’s funny because as you’re working with these tools, they keep coming out with new features and you’re like, “Well, okay, now there’s lip sync” and there’s all this stuff.
SR: And it integrates stable diffusion and text to speech and all these other—
WL: Yeah. So, we started working. We got this idea of exploring art cinema language - we’re talking about 20th century, black and white, auteur cinema like Hitchcock, Lang, Murnau, Antonioni, Bergman. I mean, those are all male, but we also had Agnès Varda and Věra Chytilová, who is a Czech filmmaker from the 60s, feeding some of our ideas.
SR: Like mid 1960s and 70s experimental cinema.
WL: Yeah. So, Mark’s book reflected on an alien intelligence that awakened our own alien intelligence. In other words, that the machine opened up what’s alien within our own creativity, which is an idea that’s important to him, and I think a lot of artists might agree with that. You know, it’s not you creating, it’s something coming out of you. So, the idea was an alien intelligence finds access to this data set of cinema history and was just using it to explore the human. We definitely went with feminine archetypes, rather than... we were not interested in the gangsters and the cowboys of cinema. We wanted more internal, reflective characters, like in a Bergman film, that kind of thing. We had some initial images that then... really the way it worked was simply we exchanged a set of images, and we see which ones cohere. And then, in order to find some arc, not really narrative arc, but a poetic structure, it’s just a matter of filling in gaps between those initial shots. That’s what’s interesting about this AI cinema, you’re not writing a script ahead of time, you’re writing it as you’re making it and you’re writing with images.
SR: The writing becomes images.
WL: Right. It’s almost as if the prompting is a kind of screenwriting, but you’re doing it with immediacy. And you’re actually responding to what the AI, RunwayML in this case, is giving you, because if it messes up and it’s interesting, it might completely change what the next shot might be.
SR: Yeah. And that glitch, which is something that I’ve done a little bit of work with, and Mark Amerika, I know that’s been a big part of his, but this sort of this moment of transitional media where things haven’t settled, and things are sort of in between things. I think the early history of photography, for example, or the early history of cinema is really exciting because it was all these different, wild mistakes that led to these new—
WL: Like lens flares.
SR: And many of the genres are sort of like lens flares, too, and that they happen momentarily, and then they go away. And they might have been like stereographic photography, for example, that actually was part of the early history of photography and then it just sort of went away before it came back again recently. One of the things I wanted to ask, because you guys are playing with the language quite a bit with the AI as a posthuman identity, but maybe say a little bit about post humanism and how the film reflects that.
WL: The title came sort of midway. Mark suggested Posthuman Cinema. At first I was like, “Well, that’s that doesn’t speak to any kind of narrative,” but it makes complete sense because we were reflecting on this hybrid intelligence that I would say amplifies a lot of your own intuitions, meaning your own human intuitions and instincts, because it’s so immediate, and you can see whether that idea works or not. The whole work is reflecting on itself as a process, as an alien intelligence coming in and actually accessing cinema data to work with you to create something.
SR: And that comes in on your end, through both your thinking about what filmmakers you’re referencing, but then also in the prompts. I assume you are actually referencing specific filmmakers, maybe specific actors or actresses?
WL: Yeah. Less the actors, more the filmmakers. It’s sort of “an early Fritz Lang style” for a certain look. One of them had a "Last Year at Marienbad feel," and exploring what that does in combination with maybe “moody black and white” as an adjective. We didn’t really get into naming film stocks and lenses. Some people do that, but we were just going for a kind of warmer feel of an image. And so, those terms would bring up fascinating results. If you were too direct, if you name certain films, certain scenes or even take a screengrab... it’s too direct.
SR: Yeah. And I don’t know if Runway’s blocking it yet.
WL: No, it wasn’t blocking it. So, for example, there’s a desert in a scene in the last episode where I had actually referenced “John Ford landscape in Utah.“ Now, my wife looked at it and goes, “Well, you just stole from John Ford.” And I’m like, “He stole from the Mojave Desert.” He doesn’t own the Mojave Desert, you know what I mean? Like, yes, all films reference each other!
SR: Well, that’s a kind of post-modern impulsive that’s also part of what you do and what I do and what Mark does too, that’s in the background of the way that we’re thinking all the time.
WL: I mean, how many people have copied Hitchcock again and again and again?
SR: The difference is that you are now saying it.
WL: Yeah, but I bet directors say, “I want a Hitchcock style," you know what I mean?
SR: Yeah, yeah, it just becomes complicated now.
WL: I mean, yes, it is complicated. But I think we’ve always been able to copy other works. Painters copy other painters, I mean direct copies. You can photograph other works. But why do that? Why directly copy a Star Wars film? What you want to do is you want to reference, like... what is a Cassavetes Star Wars film? That kind of mixture.
SR: A mix-up and homage.
WL: Yeah. Then you get to see these new possibilities. So that’s really where we went with it. Mark has his own language model for his writing, which is his own work, a history of all his work. So, he just started generating text and that would come my way, and I’d sort of find my way into it. I would do some writing as well, and then we just sort of shape little episodes, and the beauty of it is that they’re one minute. You can really work within one minute. That was the cutoff.
SR: It’s credit based too, right?
WL: Yeah.
SR: Because there’s a lot of computation.
WL: Yeah. I do have unlimited, though.
SR: Oh, do you?
WL: Well, my university paid for it.
SR: It seems like a lot of money at the start, although when you compare it to when you get into filmmaking, and you compare the cost of that.
WL: We were paying 30$ a month, and we did it maybe two and a half, three months each. That's like $200. I mean, that’s nothing. And we went all over the world. We had a big cast. They didn’t need to eat anything.
SR: It’s amazing, and there’s a lot of critiques of course, but to me, it’s just amazing what writers can do now. That we can write images, that we can write songs. I mean, I know there’s a lot of musicians and filmmakers and graphic designers and artists who are like, "Well, this is eating my lunch.”
WL: Let me mention Chad Mossholder, because after we edited these pieces, we send it to Chad, who’s a professional sound designer. He's worked with Mark on a lot of things. He’s a film buff and really wanted to work with us. And he created the non-AI soundtrack. Well, now we’re working on a project called Dream Factory, a longer piece, and he is getting into AI sound and music tools. And he was freaking out a little bit, because he was worried that this was going to make his whole career disappear. However, what he was doing with it was so much better than what we were doing with it.
SR: Right.
WL: He had an understanding of sound. So, I think that it’s not going to make careers go away necessarily, it’s just that the artists are going to use these tools. Yes, to speed up things, but also to play and explore more.
SR: I always think of working with AI models for text image generation or even text to text generation, that it’s not really an artificial intelligence. It’s this pool of so much of human language and so much human imagery, and then it’s just this sort of weird averaging that also brings in all these biases that are part of the fabric of our culture that we don’t even necessarily see. To me, it’s not necessarily replacing, it’s more this completely different tool / environment / cognitive play mate that’s not a human, definitely, but that is the sort of massive repository of language that we can play with—
WL: I think my more utopian vision of it is like the Tang dynasty in China, that had a flourishing of the arts. There were artists who were known and commissioned, but everyone made poetry, everyone made paintings. The idea of a kind of culture that used art for reflection and for capturing ideas and experiences, and then out of that comes the very special cream of the crop ideas that everyone responds to. I mean, it’s very hard for most people to make a movie.
SR: Yeah.
WL: Much less write a novel. Just the labor and costs involved, and to have time to do it, but if ideas start coming and it’s just part of the culture, you know? There’s a symbiosis, there’s a relationship between the big productions, big artwork, big novels and then how expression works in a more folk vernacular level.
SR: Well, a lot of garage bands are happening now, and all sorts of different fields of art as a result of people’s ability to work with these new tools that are evolving so fast, that, maybe by the time this podcast comes out, there’ll probably be another generation of Runway and Suno, who knows? And this new one, Sora?
WL: Sora. That’s crazy. The results look amazing. I don’t know what it’s really doing.
SR: But, yeah, we haven’t gotten to touch it.
WL: My students, who are all gamers for the most part, I don’t game much, but they’re not looking at art and movies, they’re gaming. So, I think independent games, the garage independent games where teenagers get together, come up with an idea and develop it, that’s totally possible. Full immersive worlds, you know? To me that’s very exciting because you want the richness of culture.
SR: Well, it’s certainly a great time to be a digital artist or electronic literature author probably, I think there’s never been a better time as long as I’ve been doing this.
WL: In my class, AI in the Arts, I had students making work that ten years ago would have been like, I mean, they are copying electronic literature forms, like moving text on the screen and things like that, and they did it without any real knowledge of coding. One can say that that’s like, I don’t know... but the tools get better, I’m sorry.
SR: Yeah, I feel bad for Nick Montfort.
WL: I know.
SR: He has a different perspective on this sort of thing, and part of it is I think he’s spent so much time becoming such a good—
WL: But he also is sort of a craftsman.
SR: For him, the code is an artform.
WL: Absolutely, yeah.
SR: And then, “Oh well, I’m going to spend 20 minutes with ChatGPT and have code that does some similar things.”
WL: I guess he’s looking into limitations of—
SR: Constraints.
WL: Constraints, yeah.
SR: And it’s not that any of that’s going away, it’s just that this discourse about machine generated content and how we work with text generation generators, or computational narrative systems, as he calls them, that's changing pretty quickly. Well, we have to go, but it’s been a pleasure talking with you.
WL: Thank you.
SR: So again, thanks for being here.
WL: Thanks, Scott.
Listen to the full episode of Off Center.
References
Amerika, Mark. 1997. GRAMMATRON. https://markamerika.com/artworks/grammatron-by-mark-amerika
Amerika, Mark. 2002. FILMTEXT. https://markamerika.com/artworks/filmtext-by-mark-amerika
Amerika, Mark. 2022. My Life as an Artificial Creative Intelligence. Stanford University Press. https://www.sup.org/books/title/?id=34987
Arellano, Robert. 1996. Sunshine ‘69. https://bobbyrabyd.github.io/sunshine69//
Bigelow, Alan. 2024. AI: A Love Story. https://thedigitalreview.com/issue04/alanbigelow/index.html
Breeze, Mez. 2024. Por]TrAIts: AI Characters + Their Microstories. https://thedigitalreview.com/issue04/mezbreeze/index.html
Farell, Lisa. 2024. Demons & Ghosts. https://thedigitalreview.com/issue04/lisafarrell/index.html
Johnston, Jhave. 2024. Identity Upgrade. https://thedigitalreview.com/issue04/jhave/index.html
Krauth, Alinta. 2024. The Songbird Speaks. https://thedigitalreview.com/issue04/alintakrauth/index.html
Lialina, Olia. 1996. My Boyfriend Came Back from the War. https://sites.rhizome.org/anthology/lialina.html
Luers, Will and Roger Dean. 2013. Hypnagogia. https://will-luers.com/collage-cinema/hypnagogia.html
Luers, Will, Hazel Smith and Roger Dean. 2016. Novelling. https://collection.eliterature.org/4/novelling
Luer, Will, Mark Amerika and Chad Mossholder. 2023. Posthuman Cinema. https://posthumancinema.com/
Rettberg, Scott. 2024. Fin Du Monde. https://thedigitalreview.com/issue04/scottrettberg/index.html
Resnais, Alain, director. Last Year at Marienbad. Rialto Pictures, 1961.
This research is partially supported by the Research Council of Norway Centers of Excellence program, project number 332643, Center for Digital Narrative and project number 335129, Extending Digital Narrative.