Camille Utterback's physical poetics, re-symbolized.
First, thank you to both respondents for their insightful and kind comments!
To respond:
Matt Gorbet critiques my implication that "poetic" interfaces do not allow users to maintain control of the interaction, stating instead that it is precisely the simplicity and clarity of this control that allow my examples to be successful. I agree that whether a work is artistic or purely functional, the connection between a user's actions and these actions' effects on the system must be clear and immediate. If a user cannot easily understand how his or her actions affect an interactive system, then the interactivity is irrelevant from the user's point of view. The same effect may as well be randomly generated. Certainly there may be conceptual reasons for incorporating information about a user's actions into a piece where no direct mapping is evident. For instance, works that collect data from the world and respond to this data raise a rich set of questions about surveillance, synesthesia, and dynamic systems. These types of works might more aptly be described as "responsive" rather than "interactive" systems, as they do not directly engage an individual via the body-centric interfaces we have been discussing.
Let me clarify my implication that poetic interfaces are somehow "unpredictable." In the examples I cite, how a user triggers a response from the interactive system is quite clear and predictable, but the entirety of the system's response to the user's actions is not predictable. The structure of the interaction is simple and direct, but the user's control over the system is not absolute. This tension is what makes the works compelling. In Small and White's Poetic Garden, for example, it is clear that manipulating one's finger on the touch pad controls the size and position of the blue glow in the waterfall. It is also clear that the blue glow will interfere with the words' trajectory down the waterfall, in a fashion similar to objects in the physical world. The structure of the interaction is clear and predictable. Despite the fact that the user understands the parameters of the interaction, the user cannot always keep the words from "slipping from one's fingers," as the words turn and move in the imaginary current. Additionally, when the user does stop words, they morph into surprising and unpredictable new words. The user can expect the words to morph when stopped, but can never know for sure what the words will change into. When something unexpected happens within clearly defined parameters of interaction, it is not frustrating, but intriguing.
Text Rain employs a similar dynamic of varied responses within a clearly defined structure of interaction. In Text Rain it is readily apparent that holding out one's hand or any other object will stop the letters from falling. Which letters will fall next, and whether one can catch enough letters to form legible words, however, is unpredictable. As with the Poetic Garden, there are also moments when the letters "escape" from one's control by falling or flowing out of one's reach. But because the structure of the interaction is clear, these moments of surprise become pleasurable, not confusing. Both the Poetic Garden and Text Rain are successful in part because the structures of their interactions are clear in the ways that Gorbet describes, but these pieces hold the user's interest because the systems' responses are varied, and not completely determined by the users.
Gorbet also provocatively raises the question, "If these pieces are the haiku of the genre, how might we go about creating the Homer?" The problem with this question is that the goal of the Iliad or the Odyssey is not expressly to question or examine its own literary structure. When we read Homer via text in a book, the book structure is (currently at least) rendered invisible due to its functional status - much like a canvas is rendered invisible in Renaissance painting. As Gorbet points out, the interactive works I cite are more akin to haiku, or perhaps more accurately, to concrete poetry, because they are examinations of form as well as content. The structures of interaction in these pieces are meant to be part of the subject matter, not invisible substrates for the content.
Gorbet asks how might one "turn the page" in Text Rain, but the goal of Text Rain and the other works I cite is not to create a broad system to accommodate many narratives, but to create a form specific to the idea being conveyed. This coupling of form and content is what distinguishes an individual artwork from the development of a medium or formal convention. The questions Gorbet's team wrestled with when developing the Tilty Table - how to move through content, how to "chose a different chapter" - are important questions when developing a functional form. If one were making art out of a Tilty Table however, the question would be instead, what type of emotional content is implied by the tilty-ness of the table, or by the sensation of the text disappearing off the table's edges? Does this physical interaction of tilting and balancing imply a content that is somehow unstable, that shifts as it slides, and that stabilizes as the user balances the text on the table?
Gorbet is right that certain body-centric pieces and interaction design might allow for longer or more involved narrative structures than others. Even in these instances, the structure of the narrative is critical to the content of the work. In See/Saw, my collaborator Adam Chapman developed a narrative with a structure that is hinged to the physical action of the see-saw. The narrative consists of a cyclical audio monologue that loops without a logical beginning or end, as long as users see-saw continuously. For each phrase in the monologue, Chapman also wrote a split narration - a phrase told from a position of power and a phrase told from a position of compromise. When users stop the see-saw on a certain phrase of the monologue - holding one user "up" and one user "down," the audio track fragments into the two phrases - heard by the "up" and "down" participant respectively via speakers embedded near them in the see-saw. The "up" and "down" clips expand on the word that each participant can see projected behind the other participant. Each participant is only party to their own "point of view" - both literally and metaphorically. The narrative Chapman developed cannot be heard or read in a linear sense at all. Its structure as well as its meaning derives from the temporal and spatial interactions with the see-saw.
My initial essay cites works that explore unusual interfaces between physical bodies and symbolic spaces. Adrianne Wortzel, in her response, is right to point out that all my examples deal with interfaces that connect the body to representations via external means. Her examples of works by Stelarc and Kac helpfully round out the discussion by pointing to the opposite, perhaps more controversial, strategy of connecting technology to the body by appending or internalizing that technology. To Wortzel's categories of extending the body, attaching things to it, and ingesting or physically morphing the body, one could add another complication. What happens when our machines or technology internalize us? This is the goal of much AI research, but artists have also probed in this direction. Natalie Jeremijenko's Voice Box pieces are examples of machines that quite literally capture something of us into themselves. My newest project, Potent Objects, also explores this line of inquiry.
Potent Objects, another collaboration with Adam Chapman, does not address bodily engagement with network media, as Wortzel conjectures, but does wrestle with the "dichotomy of control and losing control" she describes. Potent Objects explores our social anxiety about machines that can feel or emote, as well as our complex feelings about the increasing necessity of "interacting" with these machines. The series draws attention to the confused state of "interactivity" (in which users feel both in control and out of control as they adopt a certain mode of behavior in order to cause a machine to respond to them) while at the same time pointing at deeper emotional states implied by the physical grammar of interactivity.
Each Potent Object is a stand-alone object that functions as an interactive and visual pun. The pun or metaphor represented by each object will be illustrated by the object's physical form and the interaction it requires to react. Some objects will try to "learn" about certain emotions or interactive behaviors via onboard cameras or sound recording chips used to capture and incorporate a user's activities into the object's own repertoire of "emotion."
Held, for example, will be a small, sensual, egg-shaped object with an embedded video display. The video screen will cycle through images of past users "holding" the object. When the user picks up this object, a tiny camera activated by the motion will capture a short video clip of the user's hands. When the user puts the object down, the clip of the user's hands will be added to the object's "memory" displayed on the LCD screen. The object embodies and reiterates its memories of being held, while at the same time the visualizing of its memories entice you to hold it again. The object both compels you to hold it, and is "held" in a continual pattern of remembering this holding.
By coaxing a machine to respond to us, we enter into a physical and emotional dialogue with it. When users shake or hold a potent object in their attempt to interact with it, they become partners in the device's project of trying to "feel." Ultimately, the object's humorous "misunderstanding" or failure to truly embody the enacted emotion reinstates the human/machine boundary. Our emotional entanglement with these objects is represented by the traces of our interactivity, which the objects keep in their literal and figurative memory.
What emerges from the collective examples given by Gorbet, Wortzel and myself, is that whether our interfaces with our machines are functional, poetic, external, appended, ingested, or any number of unimagined shapes, this territory is not neutral ground. What is at stake ranges from the formats of new media through which we will read and imagine, to how we will explore the limits and reaches of our physical bodies, to how our information about our bodies will be captured and represented given new technologies. Boundary negotiations between our physical selves and our symbolic structures are not new to the human condition. All the examples I chose for my initial essay explore physical interactions with text or spoken words. Our manipulations of text are a subset of our interfaces and interactions with technology in general. Text is itself our first virtual world - the first technology of representation we have externalized, as well as effectively appended and ingested. Will our new interfaces refigure our relationship to the world and our stories as radically as the technology of writing? The answer is clearly yes. The works discussed here point to the very beginnings of how.
back to The Pixel/The Line introduction