GRIOT's Tales of Haints and Seraphs: A Computational Narrative Generation System

GRIOT's Tales of Haints and Seraphs: A Computational Narrative Generation System

2008-02-19

D. Fox Harrell considers what is computational about composition, and describes the GRIOT system for generating literary texts.

D. Fox Harrell:

The gloss provided by Ben Underwood echoes, rather than disputes, the approach taken in the work described here. Various critiques of the capacity of computing technologies to represent many everyday aspects of human cognition, much less consciousness, are well known. The work here acknowledges such limitations and embraces critical perspectives of AI such as provided by Searle, Winograd and Flores, Agre, and others. The goal of the GRIOT system is not to model consciousness. It is not full system autonomy or machine competence at a Turing-test style for story generation.

2008-10-12
Ben Underwood:

The conversion of the previously noncomputational into the computational evokes the debates over AI that, in his book The Mystery of Consciousness, John Searle succinctly summarizes and participates in. Searle’s position is that computational processes cannot simulate consciousness.

2008-02-22

Within American culture there exist familiar depictions where Pan-African spiritual traditions such as Haitian Vodou or Brazilian Candomblé are presented as “evil.” For example, the trickster Orixá Exu is conflated with the devil in some persecutorial Christian ideologies. By the same token, there exists within some African-based cultures the notion of the “white devil,” or technology as the “unnatural fruit of Babylon.” Demonization occurs from both sides of the dialectic, though power distribution is not equitable between them.

Tales of miscegenational diabolic power are common in contemporary cultural media as diverse as film, comics, popular music, and computer games: human mothers gave birth to Dante, the son of the demon knight Sparda in Capcom’s Devil May Cry games series, Alucard (“Dracula” reversed) the son of Dracula in Konami’s Castlevania game series, and Blade, the jazz trumpet-playing vampire hunter whose blood was tainted by a vampire’s feasting on his mother at birth in Marv Wolfman’s Tomb of Dracula comics series (Wolfman and Colan 1973). In the 1980s the Rastafarian hardcore/punk rock group Bad Brains described themselves as “Fearless Vampire Killers” as their singer intoned:

The bourgeoisie had better watch out for me. All throughout this so-called nation, we don’t want your filthy money, we don’t need your innocent bloodshed. We just wanna end your world. Well my mind’s made up. Yes, it’s time for you to pay, better watch out for me. I’m a member of the F.V.K.

Imagery from the dark romance genre edged up against issues of identity, social inequity, and cultural misrepresentation is a trope with a long history (Harrell 2005).

These are considered sensitive sociological and humanistic issues; more rarely are they considered cognitive issues, but seldom would anyone consider them computational issues (Ibid., 5). The GRIOT computational narrative system utilizes techniques suitable for representing meaning and expression such as the thoughts in the paragraph above. GRIOT is a computer program developed to implement systems that output narratives in response to user input. The first case studies built using GRIOT generate interactive poetry. GRIOT is based on an approach to computational narrative that builds on research from cognitive linguistics on metaphor and conceptual blending (Lakoff and Turner 1989; Fauconnier and Turner 2002), sociolinguistics on how humans structure narrative (Labov 1972; Linde 1993; Goguen 2003), computer science on algebraic semantics and semiotics (Goguen and Malcolm 1996; Goguen 1999).

A focus in the development of the GRIOT system was on developing computational techniques suitable for representing an author’s intended subjective meaning and expression. Special attention was given to the generation of new concepts on the fly (by conceptual blending), and representing them for use in interactive and generative narrative artwork. An algorithm for conceptual blending called ALLOY lies at the heart of GRIOT. Gilles Fauconnier and Mark Turner’s (2002, 8) conceptual blending theory describes the processes by which concepts are integrated, guided by “uniform structural and dynamic principles” both unconsciously in everyday thought and in more complex abstract thought such as in literary arts or rhetoric. A basic component of the theory is a conceptual space. Conceptual spaces, building upon Fauconnier’s theory of mental spaces (Fauconnier 1994), are sets of “elements” and relations, “relatively small, transient collections of concepts, selected from larger domains for some purpose at hand, such as understanding a particular sentence” (Goguen and Harrell 2004).

A computational model of conceptual blending requires that conceptual spaces be formalized in a manner amenable to algorithmic generation, integration, media representation, and manipulation via user input. In GRIOT, conceptual spaces and mappings between them are represented using Joseph Goguen’s algebraic semiotics (Ibid., 13). In algebraic semiotics the structure of complex signs, including multimedia signs (e.g., a film with closed captioning), and the blending of such structures are described using semiotic systems (also called sign systems) and semiotic morphisms (mappings between sign systems).

This does not imply a belief that meaning can be reduced to mathematical formalization; on the contrary, the underlying theories in cognitive linguistics assert that meaning is considered to be contextual and dynamic, and has a basis in embodied human experience. This means that meaning is “actively constructed by staggeringly complex mental operations” such as conceptual blending (Ibid., 8). Furthermore, meaning depends upon the fact that humans exist “in a world that is inseparable from our bodies, our language, and our social history” (Varela, Thompson and Rosch 1991). These underlying assumptions about the nature of meaning and the use of formalization are some of the characteristics that distinguish GRIOT from other work in poetry and narrative generation. Some implementation details for GRIOT can be found in Varela, Thompson and Rosch’s The Embodied Mind: Cognitive Science and Human Experience and details for ALLOY can be found in Joseph Goguen and my “Style as Choice of Blending Principles.” Figure 29.1 depicts the architecture of the GRIOT system and figures 29.2 and 29.3 illustrate the process of writing in GRIOT as an author and “reading/interacting” with GRIOT as a user.

The first poetic system implemented using GRIOT is entitled “The Girl with Skin of Haints and Seraphs.” The system generates prose poems about a girl with skin of angels and demons in response to user input (via keywords as in figure 29.3 or a graphical user interface as depicted below) that triggers the use of concepts from themes of Europe, Africa, skin, whiteness, devils, and angels. A poetic system is not the individual output of one execution of GRIOT, but rather the code that generates a variety of poems algorithmically. Templates and granular fragments of poetry organized by narrative clause type are combined with the output of ALLOY to result in poems that differ not only in how the templates are selected and configured, but in the meaning being expressed by the blended concepts.

Figure 29.2 provides a brief description of how a poetic system designer can input templates and theme domains (sets of relations that describe a theme). Concepts are combined according to principles that produce “optimal” blends. Typically this optimality results in “common sense” blends, but for particular poetic effects different, “dis-optimal” criteria can be utilized. In “The Girl with Skin of Haints and Seraphs,” each poem invokes similar core concepts and themes, and additional meaning emerges from the differences between the varying output poems.

The output from the “Girl with Skin of Haints and Seraphs” is meant to evoke the idea that identity is not based on static categories and classifications, but is rather dynamically changing and contingent upon social situations. A dynamic identity must take into account immediate social context. In the African Diaspora there are many artistic traditions that negotiate the disjunction between self-identity and social identity, between historical, traditional identities and identities of resistance. Dynamic improvisation and call-and-response structures are familiar aspects of pan-African narrative forms as diverse as the delta blues, Charles Mingus’s calling out of the segregationist governor of Arkansas in “Fables of Faubus,” the penetratingly satirical fiction and poetry of Ishmael Reed, hip-hop freestyle rhyming, and the African Brazilian martial art and dance Capoeira Angola (Downey 2005; Pequeno 2000).

Written prose poetry (Lehman 2003) and its more recent descendant flash fiction (Thomas, Thomas, and Hazuka 1992) (“short short” stories that encapsulate full narrative arcs within extremely abbreviated word counts) have not traditionally incorporated these techniques. On-the-fly improvisation has not often been incorporated for the simple reason that the nature of the printed text medium is not dynamically reconfigurable. Computational media have dynamic information structure and feedback loops built into the nature of the medium. The output of “The Girl with Skin of Haints and Seraphs,” as shown in figure 29.3, combines prose poetry that is dynamically reconfigurable and founded in African and African-American vernacular traditions of signification (Gates 1988) with the use of algebraic techniques to construct imaginative metaphors on the fly.

I think of this work as development of improvisational texts (active media). Concepts and metaphors can change fluidly as a user engages a piece. From my vantage point, the cultural objects of most interest are GRIOT, ALLOY, and poetic systems themselves, not each individual poem or narrative as a cultural object on its own. The theoretical approach and technology used in GRIOT are not specifically suited to the subject matter of identity and improvisation; it is meant as a general framework for generating content by means of blending, and allowing narrative structure to be reconfigurable at the conceptual level.

The system has also been used with a graphical interface (depicted below) to generate haibun, a combination of prose and haiku poetry that is often used to narrate personal everyday experiences (Basho 1967), and beat poetry in a live performance with free jazz musicians (Harrell and Goguen 2005).

My longer-term project involves using this technical and theoretical framework as a basis for creating further computational narrative artworks where in addition to textual input, users can interact with graphical or gamelike interfaces. This user interaction will still drive the generation of new metaphors and concepts, but along with text will also result in blends of graphical and/or audio media.

References: Literature

Bad Brains (1982). Bad Brains (LP). ROIR.

Basho, Matsuo (translated by Noboyuki Yuasa) (1967). The Narrow Road to the Deep North and Other Travel Sketches. New York: Penguin Classics.

Downey, Greg (2005). Learning Capoeira: Lessons in Cunning from an Afro-Brazilian Art. Oxford: Oxford University Press.

Fauconnier, Gilles (1994). Mental Spaces. New York: Cambridge University Press. (Originally published 1985. Cambridge, MA: MIT Press).

Fauconnier, Gilles and Mark Turner (2002). The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books.

Gates, Jr., Henry Louis (1988). The Signifying Monkey: A Theory of African-American Literary Criticism. New York: Oxford University Press.

Goguen, Joseph (1999). “An Introduction to Algebraic Semiotics, with Application to User Interface Design.” In Computation for Metaphors, Analogy and Agents, edited by Chrystopher Nehaniv. Berlin: Springer-Verlag.

Goguen, Joseph (2003). “Notes on Narrative.”

Goguen, Joseph and D. Fox Harrell (2004). “Style as Choice of Blending Principles.” In Style and Meaning in Language, Art, Music and Design, Proceedings of a Symposium at the 2004 AAAI Fall Symposium Series, Technical Report FS-04-07. October 21-24, 2004. Washington DC: AAAI Press.

Goguen, Joseph and Grant Malcolm (1996). Algebraic Semantics of Imperative Programs. Cambridge: MIT Press.

Harrell, D. Fox (2005). “Shades of Computational Evocation and Meaning: The GRIOT System and Improvisational Poetry Generation.” Forthcoming in Proceedings of the Digital Arts and Culture Conference. November 30-December 3, 2005. Berlin: Springer-Verlag.

Harrell, D. Fox and Joseph Goguen (2005). (Music by Turetzky, Bertram, Borgo, David, and Goguen, Ryoko.) The Griot Sings Haibun. Performance at the California Institute for Telecommunications and Information Technology, Center for Research in Computing and the Arts. La Jolla, California (October 28, 2005). The full lineup for The Griot Sings Haibun was as follows: D. Fox Harrell: GRIOT system design and implementation, The Griot Sings Haibun polypoem. Joseph Goguen: Poetry performance, The Griot Sings Haibun polypoem, November Qualia (the poem that formed the template for the text of The Griot Sings Haibun). Bertram Turetzky: Contrabass. David Borgo: Saxophones and flutes. Ryoko Amadee Goguen: Piano.

Labov, William (1972). “The Transformation of Experience in Narrative Syntax.” In Language in the Inner City. Philadelphia: University of Pennsylvania.

Lakoff, George and Mark Turner (1989). More Than Cool Reason - A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press.

Lehman, David, editor (2003). Great American Prose Poems: From Poe to the Present. New York: Scribner Book Co.

Linde, Charlotte (1993). Life Stories: The Creation of Coherence. Oxford: Oxford University Press.

Mingus, Charles (2000). Charles Mingus Presents Charles Mingus (compact disc). Candid Records. (Original session November 1960.)

Pequeno, Mestre João (2000). Uma Vida de Capoeira. Self-published.

Reed, Ishmael (1988). New and Collected Poems. New York: Atheneum Books.

Thomas, James, Denise Thomas, and Tom Hazuka, editors (1992). Flash Fiction: Very Short Stories. New York: W.W. Norton & Co.

Varela, Francisco J., Evan Thompson, Eleanor Rosch (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.

Wolfman, Marv and Gene Colan (1973). Tomb of Dracula 10. New York: Marvel Comics.

References: Games

Castlevania: Symphony of the Night. Konami. 1997.

Devil May Cry. Capcom. 2001.