The Quest for Life and Intelligence in Digital Puppets.

Abstract

Performers and artists wishing to make collaborative improvisations using three-dimensional computer graphics will encounter the following difficulty: the animation process lacks the functionality required for spontaneous, serendipitous, real-time interaction. While human motion capture makes such real-time interaction and the corresponding spontaneity possible, it lacks the expressivity required for non-realistic characterisation. My practice-based research study proposes digital puppetry as a solution to this problem. My focus is on credibility as opposed to realism: the expectation is that the characters will behave in a puppet-like manner as opposed to manifesting the aesthetics – in terms of both movement and visual design – found in human actor-based motion capture and three- dimensional animated cartoons. The practical element is motivated by three imperatives: to improvise, to collaborate and to perform live. The primary question this study sets out to answer is: Is it possible to achieve the same spontaneity and animation[1] in digital puppets as it is with tangible puppets and, if so, what are the mechanisms involved? Unable to find a three-dimensional computer graphics digital puppetry software and hardware solution, I have devised what I call the GLOPPID[2] method, where GLOPPID is an acronym for Goniometric[3] Live Organic Performance Puppetry Improvisation Digitalia. The GLOPPID method comprises an artistic approach and a practical solution in the form of a Human Digital Puppetry Interface. It uses off-the-shelf three-dimensional computer animation software, which I have incorporated into a pipeline customised to suit my creative process. This pipeline is configured to transform ready-made computer graphics models into digital puppets that can be used as collaborators, thereby allowing the performer to experience the same kind of spontaneity as is possible in physical puppet performance. My thesis asserts that it is possible to improvise with digital puppets, and I have devised my own solution in order to do this. I argue that the real-time, improvised manipulation of digital puppets offers creatively advantageous opportunities for spontaneity and expressivity. My research presents the technique of digital puppetry as an expansion of what I call the pro- puppetry thesis – the idea that puppets have dramatic advantages over human actors. It also contributes to the ontological discourses surrounding the Human-Machine Interface (HMI), trans-embodiment, the post-human, the illusion of life, and cybernetics. In addition, it explores how algorithms can be used in the arts, particularly in performance (see Kleber & Trojanowska 2019, p.101). It makes a timely contribution to the pool of knowledge, because I see digital puppets as zeitgeists – apt vehicles for human hopes and fears surrounding the digital and existential angst that is part of the fabric of 21st-century life. The professional digital puppetry practice undertaken and discussed in this study requires both general, transferable human-machine interacting skills, and the specific digital puppetry skills necessary for project phases such as rigging and manipulation. This practical approach prioritises the physical, as opposed to the psycho-physical. Informed by the theories and practice of human dramatic technique practitioners such as Decroux and Lecoq, it takes the basic building blocks of movement identifiable in the segmented anatomy and rotational articulations of the actor-as-puppet, rod, glove and string puppets, and configures them in their digital counterpart in order to accentuate the odd by means of atypical combinations, economy of motion, and asymmetry. My approach is underpinned by the idea that a puppet is uninhibited or influenced by its own ego, backstory, or emotions. Nevertheless, these properties are present in a channeled, and therefore changed, form that emanates from the Human in the Loop, namely, the digital puppeteer. In digital puppetry, the protagonist is a digital puppet operated by a human who is embedded at the core of the activity, and who simultaneously witnesses their own emotional responses as they are acted out in front of them. My work demonstrates that non-realistic, expressive approaches to movement performance derived from human physical theatre techniques (including theatre clowning), combined with the use of algorithm-assisted techniques of rigging and manipulation, mean that the puppets are not under the complete control of their puppeteers. Instead, their rogue nature and irrationality enables digital puppets to satirise and subvert notions surrounding computer-generated imagery and artificial general intelligence, while avoiding exile in the “uncanny valley” (Mori 1970). In this study the term puppet is used both literally and metaphorically. Deployed literally, puppet refers to a figure or object that is manipulated in real time in the presence of a puppeteer, in both tangible and intangible material formats. The term is used metaphorically in the sense that a puppet can be seen as an analogue of a human being that acts as a mirror, reflecting aspects of the human condition or predicament. My work expands upon an understanding of the quest for ‘life’ in physical puppets and the corresponding development of their ur-narrative – as described by Kohler & Jones (2009, p.346) – and extends this to include digital puppets. By combining the concept of this quest with Rokeby’s idea of technology as a “prosthetic of philosophy” (2019, p.107) and with the notion of art as a mirror, I explore how distinctive features present in digital puppetry practice can be used to express truths about being human. The work employs a practice-as-research methodology that provides moments for reflection during the creative process, and reflection on the creative outcomes: reflection-in-action and reflection-on-action. The thesis can be expressed in the form of the following formula: Extemporising with puppets + the ur-narrative of puppets + art as a mirror + technology as a mirror = a pro-digital-puppetry thesis on HMI, AGI + what it is to be human [1] The term animation used here refers not to the techniques of incremental, frame-by-frame posing of characters, but to the act of bringing them to life. [2] ‘Gloppid’ is also the name of a glove-puppet character invented for an ecologically-based travelling show performed during the 1980s (Childs 1988). [3] Goniometric refers to the measurement of the range of motion in a joint.