At the Time of Writing: Digital Media, Gesture, and Handwriting

by Maria Angel, Anna Gibbs

[W]riting is indeed an act in league with the past and the future, but it also requires that a body move through the space of the now. The gestures of writing can make the body present as well as absent; they do more than "commit" the word to space; they actually submit space to...the caress (Carrie Noland 209).

In this paper we explore textual forms that are emerging through a reanimated interactivity with technology and environment. We draw on a set of texts that conceptualize and enact an ethos of corporeal performance that is increasingly coming to characterize acts of reading and writing in digital contexts. What we do with our bodies when we read and write with new media and new media interfaces is becoming qualitatively different to our quiet and relatively still engagement in the scenes and scenarios of bookish writing and reading. Gesture is elicited from us when we play with or operate technologies and read texts, and writers seem to be actively programming texts that express behaviors such as movement and sound, and we contend that these changes, apparently minor in their everydayness, are more significant than might first appear.

Historically, for Walter Ong, the animism in oral forms of storytelling, and the reliance of such storytelling on context and the dynamic strategies of performance, was ultimately replaced by a culture of silent reflective reading engendered by the technology of the typographic book. Silent reading as a model for the transmission of knowledge is still a dominant one—the very idea of study is captured in the image of the (reading) thinker in repose immersed in a book, or at the very least, in text. This literate performance of the reproduction of cultural memory and knowledge has become so naturalized (internalized in Ong’s sense) that it has suppressed what the body—the eyes, the restless changes of position, the incline of the head—does in the performance of reading and writing text: that is, its active, affective, and gestural engagement with its environment and its artifacts. We argue that new media technologies reintroduce an animism and dynamism that re-engage the movement and gestures of the body in the scenes of writing and reading, rendering these processes explicitly performative in a way that is intimately involved in the generation of meaning.

For Carrie Noland, the movement and vitality of digital works “recall the corporeal energies that drive inscription” so that “[w]hen letters move, morph, or pulse, they expose digital writing’s nostalgia for the hand, the producer’s creative will to reengage with and express the kinetic impulses of the body” (“Digital Gestures” 218). Digital writing can then be understood to call forth particular kinds of body memories associated with behaviors suppressed or sublimated by typographic cultures of text. First among these is movement (reading and writing require a certain stillness), but strongly associated with it is touch, which may be linked to the capacity to reach and grasp.

What we would call an ethos of touch is evoked either through the representational aspects of the user interface or else through the movement of letters themselves.  Camille Utterback’s Text Rain is paradigmatic here of a whole series of more recent texts in its evocation of words becoming sensory objects that can be played with - in the original installation a reader-participant, standing and moving in front of a video projector, could use her/his body to play with falling letters drawn from a poem about bodies and language. Belén Gache’s Word Toys, on the other hand, works with the representational design of the typographic book which demands to be held and touched in order to be read. Here the reader is able to turn the pages by clicking and dragging the mouse, entering the book either through a portal or through an index, the pages responding visually as well as audibly as they flip or flick over. Similarly, Jason Nelson’s Dreamaphage (Version 2, 2004) uses the representational qualities of the page (more specifically, the medical report), but in an interactive context where the reader must discover, locate and read these in  mobile environments which move text and page in and out of focus (and grasp), so that the textual environment is suggestive of dreamwork, or the feverish, hallucinatory experience brought about by infection. Other works invoke a more generalized vitalism. For example, Dan Waber’s Strings comprises little visual poems about a relationship between two poles of attraction. The “text” is composed of a piece of animate string (a live wire, if you like) which morphs and dances to produce the textual elements of meaning, but in a way that address meaning in its affective and gestural as well as its linguistic or typographic context. In the little poem “argument," the characters yes and no are written through the movement which composes them (recalling the hand in handwriting although you don't see it). In it, yes and no move and stretch between two poles of attachment  simulating the movement of a tug of war, but also the movement of affect and force in an argument which shifts from antagonist to antagonist engaged in “hot” exchange. In “argument 2”, the narrative movement of yes and no is transformed through the introduction of the word or character maybe which hovers over the scene, fading in and out of sight, but is nonetheless there as a potential resolution of the conflict. “flirt” sees the disappearance of yes from the string of language and the deformation of no. In “flirt (cntd)” yes remerges in a teasing dance where the string visually flirts with the reader, showing itself off bit by bit, revealing itself flirtatiously and then rather brazenly seeking attention, before making offstage trailing its feather boa behind it. In this way, Strings uses dynamic form to represent patterns of behaviour that exceed the affordances of typographic writing—typography does its work through the activity of silent reading calling forth incipient gesture and sensory modalities through the internalisation or else abstraction of writing as voice and speech.

Because of their appeal to the senses and sensory novelty, we propose that digital environments have a strong relationship to gestural, sensory and affective modes of communication. New media platforms proliferate the potentials for combining writing with visible, aural, mobile, and tactile modes. As we will argue below with reference to locative writing projects and texts involving gestural “grasp,” this affectivity signals an implicit understanding of cognitive process as integrally involving both affective and sensory modalities strongly related to movement.

We would argue that fundamental to these changes is the manner in which media mediate and alter physical activity, what we do with our bodies in acts of mediation. If “reflective” and “silent” reading and writing is characterized by incipient gesture then with new media we find a different ecology of the body. New media technologies entail and promote movement, action, and gesture: we push buttons, drag and click mice, stroke track pads, tap screens, insert and detach USBs, external drives, monitors, and so on, to say nothing of the manner in which Wii consoles and Xboxs are using whole body movement in their gaming interfaces.

We want to turn now to the way that new media produce and elicit forms of what we call “gestural mimicry.” As we have proposed, reciprocal mimicry involves the interrelation and assemblage between the reader’s or user’s body, technology and/or environment as the body moves through space. On the other hand, we argue that gestural mimicry involves a different form of relationship. Rather than the transformational movement of the body through environmental space, it involves the intensive gestural movements of the body as it interacts with a technology. In gestural mimicry, it is the smaller, intensive movements between body and technology which characterize its distinctiveness. We would want to insist, for example, on the way that digital texts and projects induce repetitive behaviors such as the hand or finger movements associated with typing or mouse movements and track-pads, and on the intensive movement of handwriting. At first glance, these small repetitive gestures may seem fairly innocuous, but we argue that with gestural mimicry it is precisely exhaustive repetition and the attenuation of expansive gesture, the smallness or intensiveness that escapes overt attention, that gives it a certain power. Using the everyday example of the pop-up toaster, Brian Rotman draws attention to the non-explicit and non-intentional effects of technologies, to “their implicit restructuring of time and space, their facilitations of new modalities of self, the work they do behind or beneath or despite the explicitly instrumental or signifying functions they are known by and are introduced to discharge” (437). Just as for Rotman the pop-up toaster “prosthetically extends” the human body and contributes to the formation of new temporal economies associated with capitalism (438), we would add an increasing array of technologies and techniques which reshape habits and restructure modes of awareness. These would include, the “pings” associated with the arrival of email and text messages or missed calls on computers and mobile phones which instill behaviors and modes of attention in the user; the resizing of laptop computers so that they resemble the shape and weight of the paperback book or magazine (the workstation or desk becoming yet another mobile prosthetic attachment, hence the importance of the “fashion accessory” to designers of digital hardware); and, significantly, the varieties of button pushing and device manipulation associated with the use of the control pad in digital gaming (the affects of this conditioning of the human body are yet to be fully realized).

In thinking about new textual behaviors emerging in digital contexts and what their relevance might be, we draw on the work of Twentieth-Century French anthropologist Marcel Jousse, in whose unacknowledged wake Ong, Abram, Donald and others, all theorize orality and its relationship with writing. Jousse understands writing as one of the technologies of an algebraic form of abstraction that occludes more direct “concrete” or mimetic form he terms “mimage”—that is, the corporeal or gestural re-enactment and transmission of the world's energies. Marcel Jousse theorizes gesture or “geste”, as the body's “direct resonance” with the energies of the world, or its “objective registration” of them through engagement with it (15). Jousse calls this process “intussuception.” For him, gesture comprehends all corporeal activity, including that associated with affect (laughter, tears), and even states of mind, while what we ordinarily call “intellectual” is simply an abstraction from gesture (15). For Jousse, then, memory is less a purely cognitive capacity than the result of the body’s organization of expression in relation to its action in and active proprioceptive apprehension of its relation to the world: the up/down, left/right, forwards/backwards, which forms the basis of parallelism in oral style, or the correspondence between the rhythm of breathing and certain traditions of verse, including additive patterns like the one in a particular 1609 version of Genesis cited by Ong, for example. Because gestural re-enactment can be delayed or deferred as the performance of memory, it can also be made conscious and either become formulaic or opened to play—and hence, to art.  As Merleau-Ponty long ago realized, writers don’t just deliver messages: they produce gestures (60), for even forms of sedentary writing involve incipient gesture. Gilles Deleuze and Claire Parnet contend that writing is made less of ideas than of “motor agitation and inertia” (75). From this perspective, the writing body rehearses and recalls or re-enacts, in abstract attenuated form, active relations with the world. As we have argued, this experience of incipience is also at the heart of silent reading, and we now propose that the interactivity and new affordances and vitality of digital texts and contexts is externalizing gesture and making visible the conditions of gestural mimicry in both reading and writing practices.

Writing of the evolution of human cognition, Merlin Donald, like Jousse, has theorized this mimetic capacity as one that involves using human bodies as a complex storage and retrieval devices for the performance of memory. Arguing that “a mimetic act is a manifestation of a highly abstract modeling process,” Donald writes that:

Mimesis is based in a memory system that can rehearse and refine movement voluntarily and systematically, guided by a perceptual model of the body in its surrounding environment, and store and retrieve products of that rehearsal…Purposive rehearsal reveals the presence of a unified self-modelling process, and most importantly, the whole body becomes a potential source of conscious representation. Retrievable body-memories were thus the first true representations, and also the most basic form of reflection, since the mimetic motor act itself represents something: systematic rehearsal refers to the rehearsed act itself, comparing each exemplar with a sort of idealized version of itself. (6)

For Donald, this mimetic system is “supramodal” in that the human body can employ various senses or perceptual systems for modeling input and output (a visual event can be translated into speech, or a tactile sensation rendered by an image). The link between “grasping” a model for performance and purposive rehearsal can be found in the experience of typing. As Steven Connor suggests, typing is like playing the piano, where you don’t just play sequential notes, but chords and runs in which the fingers are faster than the brain, and the ambidextrous mode this requires means that rhythm plays a crucial role in co-ordinating the activities of each hand in relation to the other, creating an automaticity that doubtless produces its own affect (for example, pleasure in fluency and agency) but which also allows concentration on the feeling of the music itself precisely because it is automatic. When it comes to typing, however, the movement of the fingers on their respective hands gives rise not to notes but to particular words with particular meanings, which might then be affectively and imaginatively linked to the gestures that produce them. For Connor, then,

“I” is located firmly on my right hand, and I strike it always with my second finger. “Me” is a snappy and satisfying cooperation between left and right; but "we” belongs to the left. A special atmosphere attaches to words the letters of which are concentrated on the left; the atmosphere of words learned in a foreign language. The word “dream” is almost entirely a left-handed word, as is the word "left” and the word “word.” (np)

The activity of typing conjoins gesture with signification and meaning through the medium of affect in a process that relies on the human “mimetic faculty” described by Walter Benjamin. For Benjamin, this means the capacity to translate between, or abstract immediately, from different forms of sensory experience (so that we can speak of synaesthesia, the intermingling of the senses, or of sensory experience as always taking place in the mode of “mixture” as Michel does—so too Massumi, who places more emphasis on the shifting ratios between sensory modalities).

While the human species’ capacity for gestural mimicry and cross modalisation is rich and varied, we propose that a new ontology is developing around the figure of the hand, particularly in its relation with visibility and forms of knowledge, or what Nigel Thrift calls “perceptual labour” (2008: 98). We see this in the attention that designers are giving not only to the look, but to the “feel” or haptic” quality of technological objects. As both Serge Bouchardon and Barbara Bolt have noted (in different contexts), to “grasp” a concept, an object or a text, is to introduce a human dimension to knowledge through the figure of the hand and its capacity for manipulation and gesture. Nigel Thrift, too, in his work on the computational aspects of contemporary culture, points towards a new relevance for the hand as an age old epistemological figure. For Thrift the hand not only manipulates the world, it touches it, and in emergent contexts he speculates that “touch will increasingly be thought of as a sense which can stretch over large spaces.” He writes that

all manner of entities will be  produced  within an expanded sensory range....[P]aramount amongst these newly touchable entities will be data of various kinds which, through haptic engineering, will take on new kinds of presence in the world... the hand will extend, be able to touch more entities and will encounter entities which are more “touchable,” The set of experiences gathered under “touch” will therefore become a more important sense, taking in and naming experiences which heretofore have not been considered as tactile and generating haptic experiences which have hitherto been unknown. Equally, we might expect that descriptions of tactile sensations like “soft,” “hard,” “rub,” “stroke” and “caress,” “hold,” “shove,” “push,” grasp,” “hit” “strike,” and “seize” will change their meaning. (103-4)

As we argue below, the expanded sensory dimension of the hand is already producing a new epistemology in digital contexts, although perhaps because of its complexity and its acknowledged relationship with the development of the human brain, it has always held a distinctively human place of significance.As Thrift notes, quoting Tallis (2003), a strong case exists for the evolution of human intelligence driven by the hand as the first “ur-tool” (103).  The hand has often been used as a figure of creation, as when Michelangelo famously images the hands of God and “man” almost connecting, the gap between their outstretched fingers suggesting a synaptic transfer, so that the hand, thought (or the idea) and the “spark” of creation are immediately and inescapably conjoined. The figure of the hand is inseparable from the idea of gesture (which clearly involves movement), and instantiates a conception of thought as thinking (i.e. a movement) as a process which is active and creative, rather than passive and contemplative. In gesture the body is not an isolated instigator of movement: rather, movement is always attuned to a milieu to which it responds in different ways. Handwriting retains the indexical trace of gesture (hence the importance we accord to the signature, for example to an artist’s signature on a painting, or on a document where the gesture stands as a sign of legal and ethical assent). Typography and the production of graphical logos each represent attempts in different ways to reproduce the gesture inherent in writing and the making of visual signs. While digital writing must dispense with the indexicality privileged by both the legal and art historical discourses of the western world, it nevertheless returns us to the importance of gesture as a way of thinking.

The immediacy and fluidity of handwriting is an acute site of such desire. Handwriting, after all, as Hensher notes “is what registers our individuality, and the mark which our culture has made on us. It has been seen as the unknowing key to our souls and our innermost nature. It has been regarded as a sign of our health as a society, of our intelligence, and as an object of simplicity, grace, fantasy and beauty in its own right” (np). As Noland comments, the “ductus of the letter is the conduit for corporeal energy; it embodies—in an inscription—the gestures required to form it” (“Digital Gestures” 232), and “the software programs digital poets use provide many opportunities for creating a bond between the writer’s visceral experience of tracing letters and the graphic instantiation of this tracing” (“Digital Gestures” 234).

Writing of poet and painter Henri Michaux’s attempts to supersede alphabetic writing by creating a set of signs that registered the body’s production of “gestes,” Noland points out that Michaux himself, as well as many of his contemporaries, considered the apparently spontaneous realm of the gestural to be more “authentic” than alphabetic script, the handwriting requiring hours of laborious bodily discipline to learn. According to Noland, Michaux practiced sets of movements associated with the production of signs in scripts other than the western alphabet in order to unlearn the routines of movement inculcated in his own body and to open new kinetic possibilities in this same body.  What she makes clear is that crucial to Michaux’s project is “examining meaning’s contours...discovering what meaning looks like, and, more to the point, how meaning feels when being made and unmade” (Agency and Embodiment 135, emphasis in original), and this feeling might not be directly related to the particular meaning being made: rather, the point is the way in which the body is drawn into the process and signals its success so that the process can be learned and iterated. It is not so much that the experience of the author’s body is simply transmitted to the body of the reader through the intermediary of text, as that the reader has to retrace and recapture some of this feeling through the work of reading which re-performs some aspects of writing.   Here we might recall Benjamin’s comment: “perhaps Stone Age man produced such incomparable drawings of the elk only because the hand guiding the implement still remembered the bow with which it had felled the beast” (253). The same preoccupation with the body’s learned routines as repositories of "sleeping" memory which must be awakened through performance is manifested in much digital work, as letters traced out on screen as if by an invisible hand stimulate—perhaps via the mirror neuron system - mimetic impulses in the nerve endings of those watching, especially if they have been subject to the disciplines of these gestural routines themselves. Variations in the scale of the letters and the speed of writing evoke different kinds of experience beyond the rhetorical dimension of emphasis - the slowness and scale of childlike writing, or perhaps sensations of an unaccustomed fluency accompanying a speeded up process of inscription: at any rate, a contagious “micro-kinetic melody” of movement (Dagognet 528). As Alphonso Lingis observes: “a hand is not only an instrument, for seizing and taking. Hands are also organs for exploration” (69).

Here the reader’s attunement to the feeling of meaning-making (the meaning being made is precisely about the conditions under which meaning is made) is key. We are used to thinking of gesture as something continuous with speech, where it works to fill in gaps. Gesture in this optic always gestures to the point where speech fails, and something arises which can’t be articulated—or even thought—in words (Agamben, 78). The gesture as an event in speech then creates a figure for the inexpressible (Bennett). But in digital works, as we will argue, where the independence of text from speech is most apparent, gesture gestures towards communicability: it is constantly foregrounded as a way of thinking by ‘doing things with words’ under suggestion from the work itself, as we read. Here gesture entrains thought, but reading also takes place by doing, by active engagement in the manipulation of text, as in Paul Carter’s idea of "material thinking" (2004), where interaction with materials is the process by which art (sculpture, painting, performance) is made.

At the intersection of performance art, installation and writing, Fiona Templeton’s 1995 text installation Cells of Release at the disused Eastern Penitentiary in Philadelphia, imbricates the architectural space of the building (a series of cells lining both sides of a corridor) with movement through that space and the work of handwriting a single long line of text on a reel of paper made to unwind as she went, working day after day for six weeks to move down the corridor, in and out of each the cells, composing the text, writing without revision, so that the unreeling paper strip followed along her trajectory. The text itself, documented by means of photos of the installation and a transcription of the line in a book of the same name and followed by Templeton’s afterword reflecting on the process of production of the work and her rational for it, is at once referential and reflexive. It references the fact that the project was carried out as a collaboration with Amnesty International whose members write letters to authorities in order to obtain the release of political prisoners, it reflects on the site itself and what happened there, it insists on the here and now of the performance of the writing process and it instantiates the duration of the work of thinking about the unspeakable violence done to particular prisoners or the effects of incarceration on them, the work of composition and sheer physical and mental endurance involved in it, as well as the changes in the writer herself which are inevitably induced by all these things.

Using first person narration that fluctuates between Templeton herself and the prisoners on whose behalf Amnesty advocates (it draws on court records for the latter), the text addresses a “you” who sometimes seems to be actually present at the time of writing (Templeton allowed visitors to witness and read the work in progress) but who is also the reader who comes after at any time to retrace the journey of that continuous unpunctuated line, whether through the space of the prison or through the pages of the book that follows. “Handwriting,” comments Caroline Bergvall, in her essay on Templeton’s work, “commits writing to the circumstances of writing as much as to language” (228). Writing that “gestural and calligraphic skills, long devalued by most western poetry, have found their way back to art through performance and body-related visual arts: blood writing and white ink writing, writing done using body emissions and organic materials; writings that are seen to register bodily movements and highlight taboo manifestations of the human body’s interior/s,” Bergvall links Templeton’s work to a line of often avowedly feminist political performance work by Nancy Spero, Gina Pane, Ana Mendieta, André Serrano, Shirin Neshat and others (228-9). But where, especially in work dating from the 1970s and 80s, it was specifically women’s experience of inhabiting a body marked as female in a world in which the male body was taken as representative of the human, this particular work of Templeton’s foregrounds not gender, but rather the articulation of a relationship between what is culturally inexpressible—the excruciating effects of all the different forms of pain suffered by the prisoners who were once incarcerated in the particular physical site of Templeton’s work (the Eastern Penitentiary)—and the silencing, first by the removal from society effected by internment then by the world’s ignorance or forgetfulness, of the prisoners in other places whose cases have been taken up by Amnesty. In so doing, Templeton’s work connects both past to present, and “here” with a “there” which it brings into an urgent now. It is a performance of memory which it both evokes and transforms through the physical activity of a locative practice uniting walking with writing and the use of this movement to map thinking to space, or more particularly, to place, to the architecture of a (purpose) built environment with its very particular history. Once again, however, the body and its movements mark the site and sign of the intimate relationship, rather than the absolute divide, between digital and print media. Both imbricate bodies, movement and gesture with writing, but where—as in Templeton’s work—movement is by necessity extensive, in written works, digital or otherwise, gesture is more often than not intensive, where textual movement is contained within the incipience of the page, or in digital contexts in the parameters of an a screen or performance space, manipulated either by the play of generative code and/or else by the manipulation of the reader/user. Digital writing, it seems, functions at the confluence of thinking and moving.

Serge Bouchardon writes of electronic work as “moveable, actable or explorable text": many electronic texts are “read” by the reader having to click buttons or work with a mouse or track-pad to reveal text. Some notable literary examples would include Judd Morrissey’s The Jew’s Daughter where text is revealed by brushing the cursor over highlighted text, and Alison Clifford’s The Sweet Old Etcetera which produces the poetry of E.E. Cummings as a visual score which can be played by the reader. Hence, as Bouchardon argues, the materiality of electronic text

cannot be dissociated from the action of the reader. The text on the screen is not only another materialisation of a meaningful form. It is the gesture of the reader which reveals the materiality of the text. One can wonder if the nature of a digital text is not to be manipulable more than to be readable. (5).

The significance of this lies less in outdated debates about interactivity and its limits in artworks, or in discourses of the power, control or freedom of the reader, than in the way electronic works solicit the body’s involvement with reading via gesture, and in the nature of the experience this generates. Bouchardon’s work suggests that this involves not only the reader’s active capacities, but also their failure: for him, the experience of manipulation is characterized not only by grasp, but also, and crucially, by its loss. Here he draws on the way in which Bessy and Chateauraynaud use the concept of “grasp” to describe the embedding, in both persons grasping and things grasped, of moments when the body is engaged in noncognitive experience.

Mari Velonaki’s Circle D: Fragile Balances plays with the idea of grasp and its loss in the way Bouchardon suggests. At first glance, Velonaki’s work might be understood deliberately eliciting nostalgia both for the kind of grasp implied in handwriting via the handling of a pen, and for what we take for the immediacy and intimacy in other forms of (hand)craft that have become so familiar as to constitute a second nature. Her small, carefully crafted and finished wooden boxes atop their matching wooden stand each frame four screens on which handwritten texts (fragments of love letters) are made manifest as the boxes are handled and turned by the viewer. They must be held and handled carefully, however, since the messages tend to disintegrate and become illegible if the movement of the holder is jerky and abrupt. The viewer holding a box must attune herself to a certain rhythm to enable the continuation of the flow of text around and between the boxes and to be able to read it herself, so that interaction involves experimentation with and a play between grasp and loss of grasp.  Here, then, the central role of gestural attunement takes the work beyond the nostalgia of the trace, as the viewer’s movements become an active conduit for communication between the boxes. This work forms one of the iterations in Velonaki’s Fish-Bird Series which deals with incommensurability: the fish and bird are in love with each other but can’t unite, “due to technical difficulties.”

Other works which raise the oscillation of grasp, its achievement and loss in new writing environments, are Jason Nelson’s between treacherous objects and Philippe Bootz’s petite brosse à dépoussiérer la fiction ( small brush to dust off fiction). Both these works in different ways problematize a transparent relationship between perception and gesture through the sensory novelty of control. Nelson’s work is heavily influenced by his interest in games and gaming interfaces. In between treacherous objects the reader/user has to navigate different spatial and mobile  environments littered with commodity objects which recede into the horizon, or else move towards the reader, depending on the controlled motion of the mouse, which is only partially successful in avoiding obstacles as the reader searches for various text clues hidden in the work. Gestural precision does not work, in fact it is smaller more diffuse gesture that seems to control the movement through the text’s environment. Philippe Bootz’s petite brosse à dépoussiérer la fiction plays on the very carefully and minutely executed archaeological act of brushing away earth layer by layer to excavate an object, and especially on the forensic work of dusting surfaces for fingerprints or traces of DNA as the reader must continually brush away an image in order to be able to decipher clues to what might have happened in the different scenarios offered by the text which seems to lie below it. The fact that images must be brushed aside to read text is an aspect of the work that we take up in a later chapter, but important for our point here is the way the work reveals the process of dusting as a fantasy of control in the digital world, and not as a as part of the process of never-ending housework.

On one level, Nelson’s and Bootz’s texts subvert the usual idea of precision in interactive skill, and on another they demonstrate the experience of the complexities of grasp involved in “writing on complex surfaces” (Cayley “Writing on complex surfaces” np), although we would share Johanna Drucker’s view that the surface of the page is already complex, given the ways it imbricates space with temporality. Digital media does, however, introduce a new dimension to the idea of “surface." John Cayley, an eminent poet and theoretician working with  programmable media, argues for the distinctly new experience of “writing on complex surfaces” made possible by digital media. For Cayley, programmable media introduce a complexity of surface and temporal structures that enable the performance of writing in ways only imagined, or abstracted from the space of the written paper page. Drawing on and critiquing Joan Retallack’s concept of the “flatland” of (paper page) writing, Cayley asks us to think about the ways in which new media writing is reorganizing our perceptions of both time and space. Referring to his experimental writing in the CAVE environment at Brown University, where the experience of language is phenomenologically “real” (in the world of CAVE it is possible to perceive letters in a mobile three dimensional context, and more recently to “play” with them), he writes that:

[l]iteral graphic materiality is able to entirely and suddenly transform spatial perception, and at the same time, it creates an entirely new space for itself, for inscription and for reading. It creates the potential for a new experience of language.... Specifically, I believe [it] provides an important vehicle for the investigation of the phenomenology of surfaces, surfaces in general, but also, of course, surfaces of inscription and reading. (Lens 13)

Here Cayley points to the emergence of a new phenomenology of new media writing, which we argue is co-emergent with a different ontology brought into being through the combination of the affordances of new technologies and the way in which they are embodied. Cayley’s CAVE texts involve a new grasp of language and its literalisation in contexts that transform—as he argues—the spaces of language but also, we think, our sense of space and our place within it.

It appears to us, through our analyses of the new surfaces upon which we are inscribing our selves and our cultures, that the importance of grasp and its loss is played out through a dynamic of technical innovation involving transparency and rupture. Gestural movement either sutures together hitherto disparate surfaces so that everything seems transparent and within one’s grasp, or else things evade capture and break apart into irreconcilable differences. JODI's work geogeo is a good example of a text which explicitly engages with the slipperiness of grasp in what we might call the particular surface tensions of digital writing, while simultaneously exposing the political nature of the organization of space into the surfaces we are beginning to take for granted. The work opens with a pointed anecdote (Kare np) about how Steve Jobs’ casual intervention in the search for a series of linked names for new fonts resulted in them being mapped onto the world rather than onto a local train line. The developers initially named the fonts after stops on a local train line. Apparently, for Jobs size mattered, and the story collected on folklore.org is tagged “personality” as well as “software design.” Alongside this excerpt appears a list of the cities that subsequently had fonts named for them (Athens, Monacco, Geneva, Cairo, etc.)—or perhaps it is a list of the fonts that came to be named after cities. The ambiguity is doubtlessly deliberate, as the work reconfigures writing in relation to territory rather than to speech. Selecting one of these proper names will deploy sample text in the relevant font over the Google Map of the terrain for which the font was named, so that the imagination of a particular city is mapped back onto it along with a short text (a word or phrase), scrawled on the surface of the earth. From a particular focal resolution, one might now touch or write the surface of the earth as one would a page or screen. The font sample (phrase) corresponds to handwritten manuscript in its lack of character uniformity and precision because the sample font isn’t bit-mapped in the usual way. Instead, it appeals to, and yet corrupts, the idea of vector mapping, all the more since the letters often follow features of the imaged terrain, such as roads, so as to highlight the way in which the world is already written by human systems, if non-alphabetically. In some ways this is the obverse of the writing of the world on its own surfaces by rivers, canyons, or the trajectories of birds.

But geogeo’s writing is also constrained, as roads follow—at least to some extent—terrain. This work inhabits and works the spaces between different forms or systems of imaging and mapping, and denaturalizes these views (reminding us, as Peter Gendolla writes, of the book of nature (371)) by relativising them in relation to each other, or by creating friction between them. The writing on the imaged terrain in geogeo is both electronic and “handwriting” (or, more properly, it resembles a type of graffiti etched or scratched awkwardly on the surface of the earth). It represents a move to personalize and render proximate the abstraction performed by writing, as well as the abstraction performed by imaging and mapping. The work plays with  telepresence and the smooth traversal of time-space enjoyed by the user of Google Maps where the experience is one of increasing proximity to things as one navigates through the visual field of satellite resolution. From outer space, one can visually reach down and touch the earth at the level of the individual street and follow the pathway set by the line of writing. However, here the coherence and readability of the writing breaks down so that movement in the territory (as opposed to movement over it at a distance) comes at the expense of legibility. We see the move from the striated, optical space of distance and mastery (the commanding view), into the haptic visuality theorized by Laura Marks in which vision is often blurry so that we have to feel our way through space. One might argue that the tendency of web platforms to glitch and freeze might work to elicit the sense of contingency and especially of transience that accompanies mortality, or the alternating seizing and loss of control that characterizes the erotic, though in one way at least the rapid technological development we are currently witnessing might augur the opposite. While in the films Marks discusses, haptic visuality tends to be instantiated in images of sensual experience (like hair-washing, smelling spices or perfumes, and so on) and ultimately bound by narrative, geogeo operates differently calling into account the proprioceptive and haptic economies of a globalizing telepresence.

In geogeo, writing becomes a kind of walking with the fingers, an activity of proximity with a keyboard and prosthetic vision: an abstract mapping of space enabled by Google Maps (trademark registered), and a virtual graffiti or tagging of terrain by JODI. Michel de Certeau describes the way pedestrians who traverse the city create a kind of unconscious script readable only to others from the distance allowed by height, so geogeo can be read as a humorous play on this micro-political idea, a kind of making real of the place-making work of (locative) writing involved in passage through space. In this kind of work one can traverse space with a speed that far surpasses walking, so that the relationship pertaining between time and space is altered, “de-realized” in Stiegler’s sense (110), in contradistinction to the immediacy of access to the particularity of place promised by Google Maps. Here proprioception (the body’s felt awareness of its place and extensive volume in space) as we know it in the everyday experience of walking (Massumi’s “movement vision”) is replaced by a haptic visuality used not to communicate sensory experience by mimesis (as it often is in the quasi-ethnographic films Laura Marks and others discuss, as well as in Velonaki’s work discussed above), but to show us the limitations of our own “mirror vision” in the space of statistical calculation. The speed of interoception is exceeded—and superceded—by the speed of the fingers, and yet, crucially, we discover that anticipation is not enhanced by virtue of this. While geogeo ironically purports to instantiate the universalizing white masculinist vision of smooth transparent access to the world assumed by Steve Jobs’ cosmopolitan and cosmological inscriptive staking of claim. It performs the political work of exposing the fantasy of access to worlds by telepresence, making clear that a certain kind of awareness of detail at street level comes at the expense of another kind of knowledge and purchase gained by overview.

What seems to be emerging, then, in this and other digital media works, is an emphasis on memory as performance, intimately bound up with the gestural and habitual capacities of the human body. Complex surfaces are frequently glossed over by the small habitual consistencies (forms of gestural mimicry) which render temporal and spatial differences homogenous, as in the YouTube-spread contagion of images of deranged cats and laughing babies analysed by Anna Munster as haeccities, or “moments of thisness.” The popularity of these kinds of phenomena contour the affective energy of the present forming a “shared singularity of everydayness”, instances of which are nevertheless generalisable, and can thus comprise what she calls “singular generality.” While Munster sees these as a form of “networked aesthesia,” we would argue that they rather represent a form of networked anaesthesia whose insistence on the moment militates against critical reflection. Against this, the ironies and ambiguities of works that position themselves as part of an aesthetic practice (like geogeo) tend instead to provoke thoughtfulness as they are actively engaged with through the performative work of experiment and play on the part of the reader. Performance is always live, of the present moment, and it therefore demands constant renewal, re-enactment and repetition. Repetition inevitably gives rise to difference, and when this difference is remarked upon, and reproduced, it marks a point of cultural transformation.

Works cited

Abrams, David. (1996) The Spell of the Sensuous, New York: Vintage.

Agamben, Giorgio. (1999). “Kommerell, or on gesture." In Potentialities: Collected essays in philosophy. Trans: Daniel Heller-Roazen. Stanford, CA: Stanford University.

Benjamin, Walter (2002). “The Knowledge That the First Material on which the Mimetic Faculty Tested Itself” [1936]. Selected Writings vol. 3, 1935-1938, eds Howard Eiland and Michael W Jennings. Cambridge, Mass and London: Belknap Press of Harvard University Press. 

Bennett, Jill, (2008). “A Feeling of Insincerity: Politics, Ventriloquoy, and the Dialectics of Gesture.” In Ernst van Alphen, Mieke Bal and Carel Smith (ed.). The Rhetoric of Sincerity. Stanford University Press, Stanford, California, pp. 195 – 213.

Bergvall, Caroline (2008). “Handwriting as a Form of Protest: Fiona Templeton's Cells of Release." In Craig Dworkin, ed, The Consequence of Innovation: 21st Century Poetics. Roof Books, New York. pp. 215-230.

Bessy, Christian, and François Chateauraynaud (1995). Experts et faussaires: Pour une sociologie de la perception. Paris: Editions Métailié.

Bolt, Barbara (2006). “A Non Standard Deviation: Handlability, Praxical Knowledge and Practice Led Research.” Speculation and Innovation: Applying Practice Led Research in the Creative Industries. Brisbane: Queensland U of Technology, http://artsresearch.brighton.ac.uk/links/practiceled/Bolt2005.pdf

Bootz, Philippe, nd. Retrieved March 2012 Petite brosse à dépoussiérer la fiction. http://www.bootz.fr/brosse/brosse.html.

Bouchardon, Serge (2008). “The Aesthetics of Materiality in Electronic Literature.” Conference paper. Electronic Literature in Europe. Bergen: September 11-13, 27 Oct. Retrieved October 2012. http://elmcip.net/critical-writing/aesthetics-materiality-electronic-literature.

Bronowski, Jacob (1973). The Ascent of Man. Boston: Little, Brown and Co.

Carter, Paul (2004). Material Thinking: The Theory and Practice of Creative Research. Melbourne University Publishing.

Cayley, John (2006). “Lens: The Practice and Poetics of Writing in Immersive VR.” Leonardo Electronic Almanac, “New Media Poetry and Poetics” Special Issue, Vol 14, No. 5-6. http://leoalmanac.org/journal/vol_14/lea_v14_n05-06/jcayley.asp.

--, (2005) “Writing on Complex Surfaces.” dichtung-digital 35 (2/2005) <http://www.dichtung-digital.org/2005/2-Cayley.htm>. Originally given as a paper at the 6th DAC (Digital Arts & Culture) Conference, IT University, Copenhagen, 1-3 December.

de Certeau,  Michel  (1984). The Practice of Everyday Life. Trans. Steven Rendall. Berkeley: University of California Press.

Clifford, Alison (2011) [2006], The Sweet Old Etcetera. http://collection.eliterature.org/2/works/clifford_the_sweet_old_etcetera.html

Connor, Steven (1999). Paper given at the Modernism and the Technology of Writing conference. The Institute of English Studies, March 26. Retrieved October 2012. http://www.stevenconnor.com/modhand.htm

Dagognet, François (1992). Etienne-Jules Marey: A Passion for the Trace. Translated by Robert Galeta and Jeanine Herman. New York: Zone Books.

Deleuze, Gilles, and Parnet, Claire (1987). Dialogues. Eds. Hugh Tomlinson and Barbara Habberjam. New York: Columbia UP.

Donald, Merlin (1997). Precis of Origins of the Modern Mind: Three Stages in theEvolution of Culture and Cognition. http://www.bbsonline.org/Preprints/OldArchive/bbs.donald.html

Drucker, Johanna, (2008). “Graphic Devices: Narration and Navigation." Narrative. Vol. 16, No. 2.

Gache, Belén (2006) Word Toys. Anthology of electronic poetry for the internet. Retrieved March 2012. www.findelmundo.com.ar/wordtoys/

Gendolla, Peter (2009). "'No Preexistent World”: On 'Natural' and 'Artificial' Forms of Poetry." In Beyond the Screen: Transfromations of Literary Structures, Genres and Interfaces. Eds Jörgen Schäfer and Peter Gendolla. Bielefeld: Transcript. 365-390.

Hayles, N. Katherine (199). How We Became Posthuman: Virtual Bodies in Cybernetics,Literature and Informatics. Chicago: U Chicago P. 

Hensher, Philip (2012). “Why handwriting matters.” http://www.guardian.co.uk/books/2012/oct/07/missing-ink-handwriting-art-hensher-extract?CMP=twt_gu

JODI. geogeo (2006). http://geogeo.jodi.org

Jousse, Marcel (1925). Études de psychologie linguistique: Le Style oral rythmique et mnémotech- nique chez les Verbo-moteurs. Paris: Gabriel Beauchesne.

Kare, Susan (1983). “World Class Cities.” Folklore.org. http://www.folklore.org/StoryView.py?project=Macintosh&story=World_Class_Cities.txt

Lingis, Alphonso (1996). Sensation: Intelligibility in Sensibility. Atlantic Highlands, NJ: Humanities Press, 

Marks, Laura (2004). “Haptic Visuality: Touching with the Eyes.” The Finnish Art Review, No. 2, 2004

Massumi, Brian (2002). Parables for the Virtual: Movement, Affect, Sensation. Durham and London: Duke University Press.

Merleau-Ponty, Maurice (1974). Phenomenology: Language and Sociology. Ed J O”Neil. London: Heinemann.

Morrissey, Judd (with contributions from Laura Talley) (2000), The Jew’s Daughter. Interactive narrative. Retrieved March 2012. http://www.thejewsdaughter.com/

Munster, Anna (2010). “'Hahaha' – 186,775,728 views. Emerging collectivities of publics, emerging experiences of affect across viral media,” paper presented to the NIEA Experimental Arts Conference, UNSW, 19th August 2010.

Nelson, Jason. (2006). “Between Treacherous Objects." Retrieved March 2012. http://www.secrettechnology.com/between/between.html.

Nelson, Jason (2004). “Dreamaphage” version 2. Retrieved March 2012.  http://www.secrettechnology.com.

Noland, Carrie (2009). Agency and Embodiment: performing gestures/producing culture. Harvard U Press.

--, (2009). "Digital Gestures." In New Media Poetics: Contexts Technotexts and Theories. eds. Thomas Swiss and Adalaide Morris. Cambridge: MIT Press.

Ong, Walter. J. (1982). Orality and Literacy: The Technologizing of the Word. London: Methuen.

Rotman, Brian, (2002). “Corporeal or Gesturo-haptic Writing.” Configurations 10, 2002: 423-438.

Serres, Michel (2009). The Five Senses: A Philosophy of Mingled Bodies. Trans. Margaret Sankey and Peter Cowley. London: Continuum.

Stiegler, Bernard (2009). Technics and Time, 2: Disorientation. Trans. Stephen Barker. Stanford, Calif. :Stanford U Press.

Templeton, Fiona (1997). Cells of Release. New York: Roof Books.

Thrift, Nigel (2008). Non-Representational Theory: Space, Politics, Affect. London: Routledge.

Utterback, Camille, and Romy Achituv (1999). Text Rain. http://camilleutterback.com/projects/text-rain/

Velonaki, Mari (2008). Circle D: Fragile Balances. http://mvstudio.org/work/fragile-balances/

-,- (2004-2009). Fish-Bird Series. http://www.csr.acfr.usyd.edu.au/projects/Fish-Bird/Background.htm

Waber, Dan (1999). Strings.  http://collection.eliterature.org/1/works/waber__strings.html 

Cite this Essay: