Convergent Devices, Dissonant Genres: Tracking the “Future” of Electronic Literature on the iPad

Since its release in 2010, the Apple iPad tablet has launched a new form factor for computing, driven by the chameleon-like interface of direct manipulation provided by its touch screen. The iPad and the many other tablets that have followed in its wake (including the Kindle Fire, the many varieties of Android tablets, and the Windows Surface and touchscreen-centered Windows 8 devices) are transforming our physical relationship with texts by co-opting many of our expectations of print and integrating them with a range of gesture-driven interactive elements. As these electronic “books” have emerged on tablets, ranging in form from clear translations of the print codex to forms nearly impossible to reproduce in print, the iPad and its interface offer the potential to transform our point of entry into electronic literature at the physical level. These tablets represent a break from conventional eBook readers such as Amazon’s Kindle, which in form, scale, and interface sought to reproduce the experience of a bound codex while augmenting only some elements of the text. Tablets thus offer a greater potential than the conventional eReader for defining electronic books as something more than a direct adaptation of print volumes. However, realizing this potential requires abandoning the printed coex as a metaphor and finding new gestures to enter into a story, and in many ways drawing upon the history of interactive texts that electronic literature experiments already represent. As the tablet that started this hardware genre (and the best developed example of a tablet ecosystem, thanks in part to its start as an evolved and rescaled version of Apple’s iPhone) the iPad itself is a site of tension as much as it is a potential point of convergence: tension between faithful remediation of the codex and the breaking of the page, between genres of fiction and genres of play, between turning the page and reinventing the text.

The Virtual Library

When the iPad first launched, it was accompanied by a free version of Winnie the Pooh, translated to the device complete with its color illustrations and page turns. This became an iconic image of the future of the book on convergent devices, conveniently packaged in the Apple iBooks library, which applied the iTunes model of uniformity and instant access to books: in that, it resembles both Amazon’s Kindle store and the Barnes and Noble Nook library. As Daniel Punday observes, while there is an emerging trend towards the eBook as part of a digital library, this is one of the least compelling futures for electronic literature: “no model of the ebook based on the modular library can be entirely satisfying - no matter what DRM restrictions or freedoms are included with it. It is the nature of modular libraries to insist on uniformity of its members” (Punday). The iBooks library, which itself resembles a bookshelf with images of book covers awaiting the user’s touch, is anchored by a gesture that reproduces the feeling of the page turn, complete with the animation of each page sliding to reveal the next. This turning of page is essential enough to the Apple ecosystem of text that Apple patented it, focusing on the “skeuomorphism” that replicates the motion of the folding back of a turning page (Lowensohn). This is a step beyond the page turn and library as metaphor, an idea we are already familiar with from many of our interfaces, including the iconic Windows “desktop” with “files” and “folders” (Johnson).

The process of drawing on previous technology for the development of new interfaces is nothing new: “Every age comes to terms with the latest technology by drawing upon imagery of older and more familiar things” (Johnson 16). The iBooks library doesn’t merely seek to use the printed codex as a metaphor for its translations: it seeks to replicate closely even as it displaces them, as Apple’s page turn patent hearkens towards. Journeying down the tablet rabbit hole of digital codex offered proof enough of Marshall McLuhan’s observation: “the content of a medium is always another medium.” (McLuhan and Fiore). As an initial gesture towards the significance of tablets to redefining the space of electronic books, this first launch was disappointing: it suggested that while the tablet had the potential to break from the eReader model of emulating the physical, it would instead be a place for what Bolter and Grusin have termed remediation. This tension between remediation and reinvention of the book has played out in the evolution of tablets as a media device, and in turn invites us to consider how electronic literature built for touch-screen interfaces might escape the initial imitative intentions of the e-reader.

Early reviews of the first iPad compared it to the Kindle and found it lacking as a reading device, suggesting that if anything the addition of enhancements and aesthetic advances would be distracting, making the iPad only for “casual readers” (Ganapati). Yet only a few months after the iPad debuted, Craig Mod associated the iPad’s emergence with the death of print, suggesting that previous eReaders were well-suited to “formless content”: the type of text where the pagination and font were not integral to the experience, which translates well onto the Kindle’s universalized system of fonts and grayscale screen. Scholars of the history of the book might rightly argue that no text truly acts as formless content. Some books are extremely structured and aware of the page (such as Lewis Carroll’s Alice’s Adventures in Wonderland, a book in which the author planned out each page and indicated every break) while others (such as books printed exclusively as mass market paperbacks) only take form once they are laid out by the printer. But even the pagination and font choices of those mass market paperbacks can impact our perception of what we read. Sven Birkerts has suggested that abandoning this context can have a major impact on our reading: “We may gain an extraordinary dots-per-square-inch level of access to detail, but in the process we will lose much of our sense of the woven narrative consistency of the story. That is the trade-off. Access versus context.” The electronic book by this reading may not have started formless, but it is definitely formless by the time it is digitized.

However, the iPad changed the rules of digital book delivery by offering what Craig Mod terms a “universal container”: “the canvas of the iPad must be considered in a way that acknowledge the physical boundaries of the device, while also embracing the effective limitlessness of space just beyond those edges” (Mod). While the universal aspect might seem overstated (the tablet cannot imitate physical interfaces, and thus is a thoroughly digital container with its own set of limitations) the idea that tablets have brought form back to the electronic book brings the potential for physicality and artistry back to digital texts.

The Formless Screen?

Admittedly, the laptop or personal computer monitor seems to offer many of the same aspects of form: however, such screens distance the interface and human interaction from the text itself. Over twenty years ago, Steven Johnson observed that most interfaces of digital devices are still anchored to remediation and translation of metaphors drawn from physical or organic sources, an observation that holds true for many platforms today. But he also speculated further on the future of interfaces:

The most profound change ushered in by the digital revolution will not involve bells and whistles or new programming tricks. It will not come in the form of a 3-D Web browser or voice recognition or artificial intelligence. The most profound change will lie with our generic expectations about the interface itself. We will come to think of interface design as a kind of art form—perhaps the art form of the next century. (213)

With the introduction of the iPad and its highly mutable interface as a primary site for the consumption of literature, we are seeing Johnson’s prediction of the revolution of interface design as art coming true. The revolution is not happening in the iBooks library, or in any app still anchored to skeuomorphism and the tedious experience of a replicated, animated page flip for stories that need not be bound to any confinement as restrictive as a physical printed page. The transformation of the interface is happening outside the confinement of these library apps as the device is becoming a home for experimentation in electronic literature in the less restrained spaces of interactive content. Alexander Galloway’s definition of an interface as “not a thing” but “an effect” guides us in perceiving this interface revolution, as the hardware and software interfaces of tablet computing are not merely “significant surfaces” but thresholds of media experience (35-36). Tablets make this concept of the threshold particularly visible as the space between object and interface is impossible to distinguish. The object on the touch screen presents interactive elements that the user accesses through direct manipulation, acting simultaneously as interface and object.

The dual interface and text of an interactive object on tablets can collectively be labeled as an application, or in Apple’s parlance an “app.” Applications are pieces of software tailored to a particular platform and installed by the user as a self-contained experience. The independence of one application from another means that each application can have its own interface and form—the hardware and operating system forms Mod’s “universal container,” while the code and design of the application gives body to a digital text or experience. In Software takes Command, Lev Manovich defines software as “a layer that permeates all areas of contemporary societies” (15). This definition draws our attention to the app as a piece of software whose identity as software has been obscured by Apple’s marketing campaign: the switch to “apps” over installations more directly controlled by users has serious implications for who controls the production, distribution, and preservation of software objects, and in turn implications for the social impact of software.

This rebranding is not limited to Apple: Android’s Market, now the Google Play Store, and the Windows Store both similarly imply a dependency between hardware and software that is usually only seen in more specialized markets like console gaming. This level of control by the hardware or operating system manufacturers (depending on the platform) over the software space is shaping the potential of tablets and mobile phones as devices for creative work. Some of this reflects different capabilities of different tablets, as available physical hardware buttons, processing and graphics power, connective ports, internal sensors, and other functionality differ from device to device and platform to platform. Thus even though most tablets have similar affordances, with the touch screen as the primary interface, applications are often encountered in a closed ecosystem of a particular company. Apple’s highly developed applications store and history of support for marketing and distribution of applications make the iPad a primary space for experiments with touchscreen-driven electronic books, even though there is rarely a necessary hardware reason for choosing the iPad.

It is thanks to this market trend that I will be focusing on iPad-based applications for an examination of the current state of the touchscreen interactive book. The intimacy of the iPad as a device and its successful integration into educational environments and family home use cannot be separated from this success. The iPad is marketed as a more personal computing experience than the “personal” computer has ever conveyed, in part due to its scale and portability place it alongside the bound notebook or journal as an intimate media device. The form factor of the iPad and other tablets allows for a shared reading experience and directness, along with an integration of the tablet into a range of environments. The shift of the screen from the laptop profile to the form of a notepad or book changes the relationship with the medium. Our typical models of digital interfaces ask the user to manipulate an object removed from the display—the mouse and keyboard require a mental leap, as the action in physical space corresponds to but does not intersect the corresponding digital movement. By contrast, the iPad touchscreen’s directness allows for conscious manipulation, and a hands-on approach to storytelling elements that can offer responsiveness well beyond the current model of triggering animation or sound, which I will examine in several iPad works. The gesture-based system of the iPad means that no set pattern of buttons, keys, or input needs to be defined: the interface can continually change, not only to imitate past interfaces or other media from a telephone rotary dial to the flipping of pages, but to extended tactile experiences beyond simulation. Lori Emerson rightly cautions against taking the interface of technology (which she defines as the “intermediary layer between reader and writing” for granted—and assuming that any interface, whether the touch of the iPad screen or the shuffling of the pages of the book, is truly invisible ignores the significance of the platform (Emerson). At the same time, the interface of the iPad is difficult to grapple with precisely because of its intended self-camouflage: in most applications it fades to the background, and the user is only aware of it when it intrudes, recalling Galloway’s notion of the interface as a threshold that we are only sometimes conscious of entering.

The Apple App Store doesn’t have an explicit market section for finding electronic literature, and thus it is through interface and intention alone that texts can be grouped into this categorization. Within the App Store categorizations as of October 2014, most apps are grouped either as “Apps” or “Games.” Interactive books with similar interfaces and capabilities can be found under both of those categories, as well as in many sub-headings including Education, Entertainment, and of course Books. The “Books” label is most frequently given to works that either have no interactive component but choose to distribute through an interface outside of Apple’s restrictive iBookstore or to books with specific interactive components, such as the My Little Pony picture books. Examining applications within these categories demonstrates the range of terminology that gets applied to these types of texts: for instance, the “My Little Pony” apps use the label “interactive adventure” despite their placement in the book section. Within this space, it’s clear that producers of apps are still negotiating the expectations of media genres and the space between interactive book and game.

Finding E-Lit on the iPad

This convergence echoes the traditions of hypertext fiction past, suggesting that the iPad and its kin might be the ideal platforms for the type of experimentation long associated with electronic literature as an expressive (if not mainstream) form. As Paul LaFarge observes:

I believe that the promise of hypertext fiction is worth pursuing, even now, or maybe especially now. On the one hand, e-books are beginning to offer writers technical possibilities that, being human, we’re going to be unable to resist. On the other, the form fits with life now. So much of what we do is hyperlinked and mediated by screens that it feels important to find a way to reflect on that condition, and fiction, literature, has long afforded us the possibility of reflection. Just as the novel taught us how to be individuals, 300 years ago, by giving us a space in which to be alone, but not too alone — a space in which to be alone with a book — so hypertext fiction may let us try on new, non-linear identities, without dissolving us entirely into the web. (LaFarge)

Where do we see the first stirrings of this mainstreaming of electronic literature? In the many children’s apps growing in popularity and innovation since the emergence of the iPad, which in their designation as “interactive” books are embracing the hybridity between reading and play and with it suggesting a future for literary convergence drawing on new metaphors for interaction.

Commercial enhanced books often draw on existing franchises. Typical of this model, the “Toy Story” Read-Along app (Disney, 2010) combines a number of features that are present in most Disney apps and a number of other franchises. The images presented evoke the film, and text is presented and highlighted as a narrator reads aloud in the “read-along” mode. Most pages have several forms of interactive content: each can be pared down to black and white for a “paint” mode, including one in which the user simply touches the screen to reveal the “right” colors. Some pages include music (such as the “Strange Things” song from the movie, complete with video and narrated text) while others incorporate games. Typical games break out of the page and offer minigames that fit the context of the narrative moment. In one, the player controls the fall of parachuting army men on their way to investigate the coming birthday presents by moving the iPad itself back and forth to guide their movement around obstacles. While different companies incorporate these levels of interactivity differently, the fundamental conceits of basic “activity book” packaging combined with elements of other genres (particularly games) are relatively consistent across offerings including Dr. Seuss, My Little Pony, Thomas & Friends, Dora the Explorer, and Sesame Street, to name only a few of the franchises with high visibility on the app store and the top rankings in the “Books” category.

Most of these applications are already affiliated with transmedia franchises, and thus have multimedia elements (such as Toy Story’s video clips, or Dora the Explorer: Where is Boots? (Viacom International, 2012) animated “surprises,” activated by exploring the page) to draw upon. Other interactive books in this category are more direct remediations of print. A characteristic example in this category is the Dr. Seuss: The Lorax app (Oceanhouse Media, 2012). The application features three modes: read to me, read it myself, and auto play. The “Read it Myself” mode feels the most like the printed book, although the text is removed from its position in the image and the reader can click on any visual element to see the corresponding word. For instance, clicking on the street causes the word “street” to pop up on the screen. Audio is added corresponding with the text, such as the sound of crows during the first scene of a deserted area outside town. The metaphor of the page is essentially preserved, and the only feeling of animation comes from the shift of focus as the frame shifts inward and outward through the image. The “Read to Me” and “Auto Play” modes offer more modality-transcendent elements. “Read to Me” moves page by page and highlights words as a voice narrates, while “Auto Play” moves through the entire work as a simple animation complete with narration. Such apps have attracted the attention of educators thanks to the still-unclear impact of these additional elements on reading skills (Schugar, Smith and Schugar). Some studies have suggested that the addition of interactive and digital elements increase engagement and particularly extend connections thanks to the reader’s manipulation (Larson), a conclusion not unlike the debates over interaction and immersion in electronic literature.

While these interactive picture books are emerging as a significant genre of their own, with their own set of conventions, the broader category of interactive books on the iPad is divided primarily between works with similar educational intention and more literary and experimental elements. In the latter category, the earliest example was Alice for iPad (Atomic Antelope, 2010), a launch-day app that added motion-responsive graphics to the reading experience of Alice’s Adventures in Wonderland. As an illustrated text, Alice was a natural choice for adaptation. Likewise, one category of literature that could in no way be reduced to the grayscale text-only format of previous e-readers has been particularly influential on the iPad: comics. Major comic publishers have invested in digitizing portions of their library, including Marvel comics. The interfaces for these comics are often awkward, as they primarily recreate the static page, or offer panel by panel navigation that ignores the significance of context and scale necessary to follow the story. Such digital comics draw attention to the original materiality of the comic as well as the impact of remediation, as Darren Wershler notes: “which screens we view digital comics on…should be a matter of concern….rather than viewing digital comics as bridges between a source and a destination, the point is to consider them as an aggregate in flux” (128).

Such digital translations brought to the screen can in some cases be more appropriate to the interface than the original print, as with Jason Shiga’s Meanwhile (adapted to iOS by Andrew Plotkin, 2011), a “choose your own adventure” style graphic novel best viewed not as pages but as a giant interlinked comic spawning from the choice of chocolate versus vanilla ice cream through a series of unlikely consequences. The ease of adaptation of this complex text is perhaps in part tied to the connection Craig Mod suggests between art books (often seen as fundamental to print, due to their integration of design and meaning) and the iPad, which has played out in several texts. Rob Ryan, a papercut artist, recently collaborated on a digital edition of his work released as an iPad app. The Invisible Kingdom (Random House, 2014) takes the papercut illustrations of the physical work and places them in motion. The consequences of the screen as portal are even more striking in comics built with the new format in mind. Most notably in recent work, Erik Loyer’s Upgrade Soul (2012, ongoing installments on iOS) is an example of a born-digital comic where panels are designed not for the page but for moving in and out of the focus window of the iPad viewer. The animation of panels recalls the genre of motion comics, but unlike the artificial animation of originally static art in Ryan’s The Invisible Kingdom or the Dr. Seuss picture book adaptations, is designed with this multimodality in mind.

The most successful of these applications suggest that the iPad offers not so much a “universal container” as a multimodal interface for convergent media experiences that are still emerging. These new forms do owe a strong debt to previous media—and indeed, Jesse Stommel suggests that the iPad’s affordances might make it “the perfect container for conceptual poetry,” as “everything on the iPad is next to everything else…the fact that we use our finger to control the iPad suggests (at least figuratively) that pages 1 and 5 are not only next to one another but are in direct physical contact,” which is appropriate to the “hyperlinked” nature of conceptual / metonymic writing. Stommel’s suggestion is particularly intriguing because conceptual poetry pre-exists the iPad and tablets, but that does not preclude tablets being an ideal delivery system for previously print experiments. Adaptations of printed poetry such as The Waste Land (Faber / Touch Press, 2011) offer a different experience of the text, as Andrew Klobucar suggests in his reading of the app:

Neither text nor program, the poem now exemplifies a significantly different relationship to information than it did in 1922 – one characterized by the contemporary viewer’s innately ambiguous relationship to programmable language in general. Viewing digital culture through the touchscreen, we find ourselves thus momentarily free as electronic consumers of Eliot’s original suspicions of the ultimate threat information supposedly presented to cultural history’s singular, continuous sense and respect of authority. (Klobucar)

In this case, the interface on the work has the potential to transform our relationship to its foundational text, thanks to the same sense of the work as a simultaneous happening (rather than an organized linear codex) that Stommel’s reading of the iPad implied.

Interfacing with Poetics

Current poets are experimenting with the potential of this interface. Jason Edward Lewis’s PoEMMs—“Poems for Excitable [Mobile] Media"—use different manipulations to construct poetry. The interactive poem Speak (2012, iOS) offers a galaxy of letters, and the reader’s finger acts as a beacon pulling letters to form the text of a poem. The intimacy of the work is in the connection between text and finger: when the reader moves away from the screen or releases the letter, the words disperse into the wandering constellations, not re-emerging until the reader’s touch is once again focused on the screen. A similar feeling of motion and touch (though less of narrative) emerges from Joerg Piringer’s app abcdefghijklmnopqrstuvwxyz (2010, iOS), which again offers the reader the potential to compose through touching letters. In this case, the letters—once unleashed from the confines at the borders of the screen—have a will of their own, bouncing and colliding in the open space to create a cacophony of sounds based on the chosen letters and their trajectories. Such works highlight the immediacy of contact that is present in all interactive books designed for the touchscreen, if only occasionally exploited so directly by the creator: “With a stylus or touch screen we can come into very direct, although virtual, contact with the word, contact that is much more immediate and intimate than using a typewriter, which means that these devices once again establish an immediate relation between the body (in fact, the hand) and the word” (Strehovec). Poetic experiments of this kind are the clearest exploitation of the nature of the tablet interface and the dynamics of the personal “universal” container.

While the Apple ecosystem (and the ludicrous act of patenting an animated version of a physical page turn) may still hold some anchors of the interface as metaphor and remediation of printed codex and physical bookshelves, the potential of the iPad as a revolutionary device is in the experimentation with interface it allows—and, as the tablet format continues to be explored by a range of platform developers, interactive books are finding homes throughout personal touchscreens. The interface, to recall Emerson’s warning against the “interface-free”, is not transparent: its active physicality is essential to its potential to redefine our interactions with books. The early examples of interactive books as “applications” explored here promise new forms of electronic literature driven by play with gestures, and the exploration of narrative space that those gestures enable—an exploration that moves in directions far beyond the turning of a page. The replacement of point-and-click with touch demands new metaphors for interaction, drawn from remediating and reconsidering interactive fiction, games, gamebooks, and other genres not easily fused on a single page.

Works Cited

Atomic Antelope. Alice for the iPad. 3 April 2010. 4 February 2012. <https://itunes.apple.com/us/app/alice-for-the-ipad/id354537426?mt=8>.

Birkerts, Sven. "Resisting the Kindle." 2 March 2009. The Atlantic. 5 October 2014. <http://www.theatlantic.com/magazine/archive/2009/03/resisting-the-kindle/307345/2/>.

Disney. Toy Story Read-Along. 1 April 2010. 4 August 2011. <https://itunes.apple.com/us/app/toy-story-read-along/id364376920?mt=8>.

Emerson, Lori. "Activist Media Poetics: Electronic Literature Against the Interface-free." Modern Language Association National Convention. Seattle. <http://loriemerson.net/2012/01/12/activist-media-poetics-electronic-literature-against-the-interface-free-mla-2012/>. 2012.

Galloway, Alexander. The Interface Effect. Cambridge: Polity Press, 2012.

Ganapati, Priya. "Apple iPad Raises the Stakes for E-Readers." 29 January 2010. Wired. 5 September 2013. <http://www.wired.com/2010/01/apple-ipad-ereaders/>.

Johnson, Steven. Interface Culture. San Francisco: Basic Books, 1997.

Klobucar, Andrew. "Against Information: Reading (in) the Electronic Waste Land." 2 June 2013. Electronic Book Review. 6 August 2014. <http://www.electronicbookreview.com/thread/electropoetics/wasteland>.

LaFarge, Paul. "Why the book's future never happened." 4 October 2011. Salon. 12 November 2011. <http://www.salon.com/2011/10/04/return_of_hypertext/>.

Larson, Lotta C. "Digital Readers: The Next Chapter in E-Book Reading and Response." The Reading Teacher: 64(1) (2010): 15-22.

Lewis, Jason. Speak. 2012. 6 July 2014. <https://itunes.apple.com/app/speak/id406078727>.

Lowensohn, Josh. "Apple granted design patent for turning pages." 16 November 2012. CNET. <http://news.cnet.com/8301-13579_3-57551317-37/apple-granted-design-patent-for-turning-pages/>. 2013 12 February.

Manovich, Lev. Software takes Command. New York: Bloomsbury Academic, 2013.

McLuhan, Marshall and Quentin Fiore. The Medium is the Massage. New York: Random House, 1967.

Mod, Craig. "Books in the age of the iPad." March 2010. @craigmod. 1 February 2014. <http://craigmod.com/journal/ipad_and_books/>.

Piringer, Joerg. abcdefghijklmnopqrstuvwxyz. 2010. 9 June 2014. <http://joerg.piringer.net/index.php?href=abcdefg/abcdefg.xml>.

Punday, Daniel. "Ebooks, Libraries, and Feelies." 2 February 2010. Electronic Book Review. 9 January 2012. <http://www.electronicbookreview.com/thread/criticalecologies/modular>.

Schugar, Heather Ruetschlin, Carol Smith and Jordan Schugar. "Teaching With Interactive Picture E-Books in Grades K-6." The Reading Teacher: 66(6) (2013): 615-624.

Shiga, Jason. Meanwhile. 13 May 2011. 9 April 2012. <http://itunes.apple.com/us/app/meanwhile/id373198654>.

Stommel, Jesse. "Books on the iPad: Will the Words Still Touch?" 10 March 2010. Pity Poor Flesh. 5 December 2013. <http://pitypoorflesh.wordpress.com/2010/03/10/books-on-the-ipad-will-the-words-still-touch/>.

Strehovec, Janez. "E-Literary Text in the Nomadic Cockpit." 5 April 2014. Electronic Book Review. 19 July 2014. <http://www.electronicbookreview.com/thread/electropoetics/cockpit>.

Wershler, Darren. "Digital Comics, Circulation, and the Importance of Being Eric Sluis." Cinema Journal: 50(3) (2011): 127-134.

Cite this Essay: