Through performances and readings configured across multiple screens, Phillippe Bootz conveys practices of constrained writing (and interrupted reading) into multiply mediated poetic dimensions.
1.Structure of the work
1.1 Scenic configurations
Created in 2015, Elpenor exists in two languages: English and French. It can be shown in installation, played in performance or published in a virtual machine. Its readable side develops on 2 separate screens. The interface is displayed on one screen, a text changing in time is on the other one. The interface consists of an image. Moving the mouse on the image changes it. The audience is immersed in a changing musical atmosphere.
In the installation and performance configurations, the text is projected onto a large screen behind the small screen on which the interface is displayed.
In the installation configuration (figure 1), the reader only has a mouse to act on the interface. He can sit or stand in front of the screen. Standing is the best position: the installation can then present itself as a moment of passage and not as a break. It is important, in any case, that the configuration of the screens prohibits her from reading the text displayed on the big screen while she is manipulating the mouse.
In the performance (Figure 1), the readers are seated facing the projection screen, at a distance of a few meters from it, and the small screen is placed on a base table in front of the big screen. This may be the screen of the netbook on which the work is performed. The performer sits in the shadows, among the audience, in the front row, in front of the interface that he remotely manipulates with a radio mouse. Unlike most performances, for which the attention of the public focuses on the performer, in the performance of Elpenor, the performer stays in the background. The public sees the screen of the interface but perceives with difficulty what happens on it, because of the ratio between the size of this screen and the distance to which it is located. The low position of this screen even makes its perception more difficult. The public is therefore in the situation of a multi-screen screening, whose configuration disrupts conventional screening devices. Classically, screens are positioned side by side and all have the same presence as in the 1927 configuration of Napoleon by Abel Gange, or, in digital poetry, the multi-screen version of the last performance that Judd Morissey performed in Bergen in 20081http://www.uib.no/fg/digitalkultur/23568/electronic-literature-europe-seminar et https://vimeo.com/3080936. In Elpenor, multi-screening introduces a significant discomfort into perception; either the person cannot perceive what is happening on the small screen and focuses only on the big screen, truncating the reading experience, or the screening arrangement forced her attention to defocus from the main screen to the small screen. In one configuration, information that continues to appear on the main screen can be lost, and in another, attention switches between two aesthetic spaces that are totally disconnected.
The virtual machine version opens 2 windows: the first one, which displays the interface, opens fully on the main screen of the computer, which forces the reader to position the second window on the second screen. Here too, the manipulation of the interface prohibits or hinders the simultaneous reading of the displayed text.
Whatever configuration is chosen, it is difficult, if not impossible during a screening simultaneously to perceive the plastic dimension of the interface and the linguistic dimension of the text. It is also difficult, in interactive reading (at an installation or on virtual machine), to simultaneously act and read. This structural embarrassment of reading places this work in the series of "uncomfortable reading poems" that I have realized since 2007.
The visual interface is composed of a superposition of figurative images that the mouse digs while moving. The reader can quickly understand this process, among others because the series of images regularly changes. Then again, the interface does not specify that the work is generative and it does indicate how the manipulation acts on the narrative flow. This work is part of the broader "aesthetics of frustration" that characterizes a large number of French works in digital poetry since the 1980s.
1.2 Work's composing principle
1.2.1 Primary works
Elpenor is built from 3 pre-existing works that are unstructured and restructured: Xavier Hautbois's musical Night effects and Philippe Bootz's 2 narrative texts.
a) Night effects
This piece describes the impression that one can feel while observing a city during the night from the top of a tower. It uses 2 different programs: an orchestra developed in Pure Data and a score in Iannix.
The Iannix score (Figure 2) consists of a set of curves drawn in a 3D space on which sliders move at different speeds. Plots are located around the curves. Iannix regularly sends the position of the cursors to the orchestra. It also sends a message to the Pure Data Orchestra when a cursor encounters a plot.
The Pure Data program2The exact behavior of the piece will be published in a paper to be published in the across journal. (Figure 2) uses the cursor positions to manage vocoder states. The contact information of a cursor with a plot triggers a bell sound created by additive synthesis. The vocoders only use 3 samples: a cello sound lasting one second, a whisper in a musical atmosphere of 50 seconds and the lapping of a river of 14 seconds. Due to the digital processing of sounds, these samples can be recognized only at certain times. The whisper can be heard only at the beginning of the work and the sound of the water only at the end. In other words, the "primitive" sounds of the piece can only be heard in specific geometric configurations of the Iannix score.
Thus, the musical piece, though programmed, is not random. The curves in Iannix are traveled in the same way at each execution, so that the set of data sent to Pure Data is the same at each execution. Night effects is a closed form with a beginning, a development, and an end that are well defined.
b) Narrative texts
The text shown in Elpenor is generated from 2 separate texts (Figure 3), each describing a walking tour through the Spanish nature, one to Lake Sanabria, the other in Galicia. These two hikes took place in a landscape with mountains and water. A dog accompanied the hiker in each of them.
c) The photos
Two sets of four photos compose the visual interface that can be modified with the mouse. The photos of the first set were taken during the hike in Sanabria, those of the second one during the hike in Galicia. Each set has a photo of the dog present during the hike, a photo showing sky and water, another showing mountain and a fourth showing, for Sanabria, only water, and for Galicia, only sky.
1.2.2 Transformation of the primary works
a) generalized deconstructing
Elpenor works on a principle of deconstructing / restructuring. All that is deconstructed is recomposed, starting with the perceptive multimedia space: the visual is projected onto two screens that are not in continuity. The figurative dimension is present only in the visual interface, the textual dimension only on the projection screen and the plastic dimension is distributed between the two screens. The simulcast projections play the same double role: to disrupt the main semiotic code (figurative for one, linguistic for the other) and impose a reading independent temporality that is different on each screen.
The temporality of the work is indeed split into several temporalities that interfere. The temporality of the generation of the text is not that of its animation, which is displayed differently from the slow temporality of the modification of its background. The temporality of the interaction with the photos of the interface is still different. The photos also undergo an automatic process of very slow reconstitution that allows the work to last infinitely.
Moving the mouse, the reader acts on the sound, the graphic, and the textual dimensions, which do not evolve coherently. Interactive reading activity is then itself composed of several roles that are difficult to combine but that the reader can alternate in time. She can privilege the sound composition of the generated music (instrumentalist role), the graphic composition that the manipulation produces on the visual interface (graphic designer role), the generation of numerous texts (reader role), or simply to arise playful challenges to manipulate the interface (player role). There is no "bad reading" or "forbidden reading" but these various readings produce different perceptions of the multimedia event.
The work is also deconstructed into 4 programs. The interface, programmed in logo3The programming language integrated to Macromedia Director MX 2004, communicates with the text generator also programmed in logo, and with the Iannix score which communicates with Pure Data in the same way as in Night effects. Iannix communicates the information of collision between the triggers and the sliders to the text generator.
The primary works constitute the material of a new multimedia writing. Works are analyzed and decomposed according to 5 concepts: water (lake or sea), sky, water and sky association (air), earth, life.
b) Deconstructing of the narrative texts, second writing
Each narrative text is broken down into blocks of contiguous sentences (lexia in the Barthes sense4Barthes, Roland. S/Z. points. Paris: Le Seuil, 1970.) that fall under one of the five concepts. These blocks are numbered and classified according to the 5 concepts (figure 6). In a second step, each lexie is considered as the mold of a "Hole Phrase". The Hole Phrase Generator algorithm is the oldest algorithm in digital literature. It was invented by Christopher Strachey in 1952 for his love letter generator. Its principle is simple: a sentence constitutes a mold in which certain words or syntactic functions are replaced by a hole that the program fills during the generation process by "picking" items in a list of expressions or words. It thus reproduces the grammatical mechanism of the language: the mold constitutes a syntagmatic axis and the vocabulary lists are paradigmatic axes. This algorithm ensures grammatical consistency but the semantic dimension must be managed at the level of vocabulary lists.
Elpenor uses the algorithm in a recursive way: the paradigmatic lists can themselves contain molds that have their own associated paradigmatic lists. We thus create a generative tree structure, with an a priori infinite depth, which results in generated texts whose length increases with the depth of the tree. Elpenor also introduces a stylistic component by making parts of this tree optional.
The initial narrative texts are finally only present by the first-level molds of the data file. These molds are indexed by the number of the lexia from which they are built. The generative engine includes the logic of the generation, it is programmed, while the tree of the Hole Phrases constitutes a dataset written in a text file.
The example (Figure 7) is a part of the tree generating the concept "water" for the hike in Sanabria, the ID of the lexia are indicated in the list of possible structures "structures" (here a single hole phrase, randomly chosen among the list of 7 lexia proposed in the list "nom_moules"). Thus, the text of the mold "moule6" tells us that the sixth lexie of the list (S8) corresponds to the text "Suddenly the blue stands out from the green. Listless in the surrounding torpor, the lake. A lake without a ripple or quiver. A lake as still as the rocks lying there." The parts bracketed together in the mold are randomly used or not by the generator accordingly. They are optional. The parts in square brackets are the holes: they are replaced by an element of a list. The first hole – "Suddenly the blue stands out from the green" – is replaced by an item from the list "liste1" randomly chosen (the items are separated by a dollar). We see that another item – "Not the one that falls, the one that lengthens" – is itself a hole that will be filled by an item from "liste2". In short, the shortest text that can produce this part of the tree is "Listless in the surrounding torpor, the lake. A lake without a ripple", And the longest is "An elongated blue, more elongated, just blue on waking is stretching here.Listless in the surrounding torpor, the lake. A lake without a ripple or quiver. A lake as still as the rocks lying there."
c) Control of the generated text
Photos stacked in a given order make up the visual interface.
The movement of the mouse creates a hole in the visible layer and thus reveals the layer below. The control of the work consists in digging the pile of photos. It is not possible to completely delete a photo because the program randomly regenerates the holes. The interface then sends to the generative text program the percentage of each concept visible in the visual that corresponds to the percentage of each displayed level. These percentages are used by the generative program to trigger the generation of a text on the given concept, relative to the place displayed in the visual, if they cross a numerical threshold, positive or negative. These percentages also manage the duration of the generated text display, so that the visual superimposes texts that gradually fade more or less quickly.
The visual regularly switches between the pile relating to the Sanabria Lake and the one relating to Galicia. The generated text is therefore relative in turn to one place and the other. This visual switch also reverses the weight of water and sky in the generated text since these concepts have opposite layers in the stacks of photos. The generator also produces a text on the generic concept of the Elpenor syndrome at each changeover.
The program generates a text relating to a distant bell sound each time Iannix sends to it the information of contact between a cursor and a plot.
d) Iannix score Deconstructing
Music treats certain concepts in an iconic way. For example, the concept "life" corresponds to whisper, the concept "water" to lapping. The other concepts are similarly treated by the musical matter; the concept "water and sky" is translated by a phase shift: a beat on the violin sound for low values of the concept, a distortion of this sound for strong values. The concept "earth" is translated by the introduction of a reverb, and the concept "sky" by the predominance of the violin on the iconic sounds (lapping and whispers).
Each concept is represented in the score by a single curve. One can thus modify continuously the sound presence of the concept. For example, lapping is very present when the cursor of the yellow curve is close to its origin. By contrast, if the cursor is close to the end of the curve, this sound is no longer present. The curve makes it possible to make quantitatively musical the percentage of the concept "water" in the visual interface. This also explains why the switchover between the two places is very marked in the sound at the beginning of the Elpenor performance, when the image level 1 covers almost all of the field. The cursor of the yellow curve then suddenly switches from a position close to the origin of the curve when the Sanabria stack is visible in the interface (water concept) to a position close to the other end when the Galician stack is displayed in the interface (sky concept) and vice versa.
The data that Iannix receives from the visual interface therefore imposes non-random specific configurations on the cursor positions. This produces sound patterns that do not exist in the Night effects music because most of the slider configurations made possible in Elpenor never end up in the score roll in Night effects.
Although the Iannix score is managed in a totally controlled and thought-out way, according to a semantic form, the sound it produces is itself unpredictable, because it depends, at every moment, at once on the positioning data sent by Iannix, and on the state in which the vocoders and other programmed instruments are set at that moment. In particular, the disconnection between Iannix and Pure Data would not stop the music; the music simply would not change anymore. Elpenor therefore transforms Night effects into an open form.
2 Theoretical reflections
2.1 Reception deconstructing
The form of the work is much more complex than what is revealed on the screen at any moment. The interactive reading situation is referred to as "lecture étroite"5This expression could be translated as "narrow reading" but it is better to only use the French name of the concept, because different translators translate in different terms these names,” that creates confusion. in the procedural model I have developed over the past twenty years to account for the behavior of digital works. It is only one type of reception among others, irreplaceable but incomplete. It is schematized at the structural level in the model (Figure 8) by the loop between the "reader" position (the reader in "lecture étroite"), the "transitoire observable" (the multimedia material perceptible during execution in installation, performance or on a private computer) and the apparatus (the technical device of the work). Other modalities are necessary to access the aesthetic dimensions not perceptible in "lecture étroite". These modalities are referred to as "meta-readings" in the procedural model. A meta-reading corresponds to a combination of the access modes numbered 1 to 7 in the diagram.
Meta-readings are instrumented accesses. They require an instrument to access the properties they aim to reveal6Several digital documents come with this article. They correspond to instrumented meta-readings of the work. They are also accessible from the website https://www.bootz.fr/elpenor. This article, for example, constitutes, in the diagram of figure 8, a "secondary discourse" a meta-reader analyst (us as authors of the article) produced, accessed, and read using modality 7. It gives access to conceptual information about the work and to its long form that governs its complete behavior. It also gives access to fragments of aesthetic elements such as a capture of the score and extracts of primary texts. The reading instrument used in this mode is reading a printed text (or printable if you read it online).
The performer of the work is also a meta-reader. It seems, for whoever observes her that she is in "lecture étroite" because she acts on the interface like anyone else interacting with the work in the installation version. This is true at the functional level of manipulation, but not at the level of access. Contrary to the "lecture étroite", the performance uses modalities 2 and 5. She never read the generated text as she would do in "lecture étroite". The performer simply checks regularly (modality 5) that it is dense enough. She may try, additionally to manage the sound production in a musically interesting way. She does this by knowing the graphic configurations to favor or to avoid (modality 2) and she uses the visual look at the interface (modality 5) to access these configurations and to manage the temporal dynamics. One can then regard the performer as an "informed instrumentalist", in a midway role between instrumentalist and composer.
Other meta-readings require other types of instruments. For example, access to the Iannix score behavior within Elpenor is required to very slightly modify the launch program of the work (modality 2) because this one normally hides the score. It was also necessary to make a video capture of this score during an execution of the program of the work, using external software (modality 3). The instrument for this meta-reading thus consists of a specific configuration of the visual different from that which prevails in "lecture étroite", and a video recording.
One can also explore in meta-reading the generative dimensions of the work, both on the text and on the music. It is necessary to replace the program of the interface by another that allows one manually to impose the values of the concepts (modality 2) and send them to the other programs of the work as the normal interface does. The meta-reader has then just to observe the generated text (modality 5). Primary works, by contrast, cannot be accessed in this way while they fully participate in the aesthetic of the work. In the procedural model, they constitute "author's documents" and can only be accessed according to modalities specific to these documents, globally identified as modality 1 in the model. It is by comparing them to the behavior they have in Elpenor that they actually become aesthetic components of this work; it is not the material but the context that constitutes them as physical parts of the work. In this meta-reading, the modality 1 is therefore completed by another reception mode, especially the "lecture étroite" of Elpenor. These primary works are therefore ghosts in Elpenor, they are present in hollow although they deeply structure the work.
2.2 A metaphoric "lecture étroite"
The "lecture étroite" in installation or on a virtual machine to be read in private as well as spectatorial reading in performance, build an emotional feeling that is not present in any other access modality. Nevertheless, neither mode can provide access to an understanding of how the work works. In this sense, they situate a metaphorical reception of the Elpenor syndrome, which is to say: a confusion among several spaces. We experience a weakened version of the installation when we wake up in one place thinking that we are in another place. The switch between Sanabria and Galicia, widely present in the multimedia event perceived in "lecture étroite", reproduces this state of confusion that manifests itself in several ways in the "transitoire observable" (change of musical atmosphere, profound modification of the visual interface, change of referent of the generated texts). This state is not narrated; it is given to experiment, to be lived. The activity of "lecture étroite" it itself a metaphor of the confusion experienced in the syndrome. This use of the "lecture étroite" activity as a component of the work is characteristic of the aesthetics of frustration. In this aesthetic, the activity of "lecture éttroite" is an internal sign to the work intended for a meta-reader (modalities 4 and 5) always conceptually present in the Reader Model of the work, even when it is not physically embodied (for example, while reading the work in a private environment). The meta-reading of the "lecture étroite" here makes it possible to identify which role the reader plays at a given moment. Thus, in the aesthetics of frustration, we do not "read" a work, we "read into" the work.
2.3 How far can we dig the sentence?
The text generator tree is a work in progress. Some parts of this tree are transformed into new holes with time, and new items are progressively added to the lists of expressions. Moreover, the addition of new molds to the expression lists gradually increases the depth of the tree. As the combinatorial algorithm generates a text following a branch beginning with the mold of an initial lexia, this process amounts to the lexia's "digging".
This treatment of the primary narrative texts raises literary questions: what becomes a dug text when the depth of the branches increases? How far can one "dig" a text without losing its specificity?