Cyber Subin
Evolving Cultural Heritage through
Collaborative Choreography with AI
A 100 year journey of Thai choreography from cultural root to integration with computer
About
This project introduces an approach for translating traditional dance knowledge into interactive computational models extending beyond static dance performance recordings. Specifically, this research presents the concept of “Human-AI co-dancing,” which involves integrating human dancers with virtual dance partners powered by models derived from dance principles. To demonstrate this concept, the project focuses on the choreographic principles deconstructed from the knowledge of traditional Thai dance. The principles are analyzed and translated into computational procedures that dynamically manipulate the movements of a virtual character by altering animation keyframes and the motions of individual joints in real-time.
We developed an interactive system that enables dancers to improvise alongside the virtual agent. The system incorporates voice control functionality, allowing the dancer, choreographer, and even the audience to participate in altering the choreography of the virtual agents by adjusting parameters that represent traditional Thai dance elements. Human-AI rehearsals yielded intriguing artistic results, with hybrid movement aesthetics emerging from the synergy and friction between humans and machines. The resulting dance production, “Cyber Subin,” demonstrates the potential of combining intangible cultural heritage, intelligent technology, and posthuman choreography to expand artistic expression and preserve traditional wisdom in a contemporary context.
Cyber Subin = Cybernetics + Subin
Cybernetics = a field of study that focuses on the principles of communication, control, and feedback in various systems, including biological, technological, and social systems.
Subin = Dream (in Thai)
Introduction
Dancing is a living form of cultural heritage that transmits choreographic knowledge between bodies across time. Recent motion capture technology has enabled the digitization of traditional dance movements to preserve this intangible heritage. However, digitization can only capture the visual form, not the tacit knowledge and improvisation. This research introduces “Human-AI co-dancing,” which combines human dancers with AI-powered virtual dance partners derived from dance principles. The researchers developed a system based on the “No. 60” principles of Thai choreographer Pichet Klunchun, which deconstruct the traditional Thai dance “Mae Bot Yai.” They analyzed the six choreographic principles and created computational procedures to manipulate a virtual character’s movements. Rehearsals and performances highlighted synergies between human and machine, expanding from traditional roots into posthuman choreography. This research bridges computation with traditional movement vocabularies to unite past and future, human and machine.
Culture
Thai traditional dance is rooted in the “Mae Bot Yai” (the “Greater Fundamentals”), a set of 59 interconnected poses that serve as the foundation for classical Thai dance and enable dancers to execute the complex choreography of “Khon”, a centuries-old Thai classical court masked dance. Khon features distinctive footwork, circular stepping patterns, fluid knee movements, and rhythmic stamping actions, along with elaborate hand and finger movements, such as intricate finger curls and stylized poses. Dancers employ techniques like body weight shifts, controlled torso movements, and the integration of arm and leg actions to create flowing, lyrical motions while maintaining firm stances. Smooth transitions between poses and an unbroken continuity of movement are essential to executing the Mae Bot Yai with grace and elegance, allowing dancers to express the beauty and aesthetic of Thai classical dance. In 1924, the introduction of camera technology allowed the knowledge of “Mae Bot Yai” to be recorded and formalized, preserving Thai dance techniques that are still used today.
Decode
Today, contemporary dance performances in Southeast Asia have been experiencing growing recognition and appreciation, as evidenced by increasing audience engagement. Choreographers across the region have gained greater visibility due to their diverse artistic forms of expression, which are informed by rich cultural backgrounds. A notable aspect of this contemporary dance scene is that many of its pioneers and practitioners began their creative journeys within the context of local traditions or folk dance practices. Pichet Klunchun, a renowned Thai contemporary dancer and choreographer, has gained recognition for his reinterpretation of traditional Thai dance. Among his notable works are “I Am a Demon” (2005), “Pichet Klunchun and Myself” with Jérôme Bel (2006), “Nijinsky Siam” (2010), “Black and White” (2011), and “Dancing with Death” (2015).
In 2017, Klunchun and his company launched Project No. 60, which focused on deconstructing the 59 fundamental poses and movements of Mae Bot Yai to speculate on an imaginary 60th pose. Klunchun created diagrams that explore the relationship between the forms of movement and the body in an attempt to deeply understand the knowledge embedded in Mae Bot Yai.
Choreography
In the process of deconstructing the 59 fundamental poses and movements of Mae Bot Yai, the foundation of traditional Thai dance, Pichet Klunchun synthesizes six choreographic principles through meticulous analysis: Energy, Circles & Curves, Axis Points, Synchronous Limbs, External Body Spaces, and Shifting Relations. His goal is to revitalize the the traditional art form by creating innovative combinations from the deconstructed traditional vocabulary.
Here are the principles derived from Mae Bot Yai:
- Energy – Exploring the dynamic quality and vitality of the movement and dance phrases. Looking at aspects like force, momentum, acceleration/deceleration.
- Circles & Curves – Examining the circular pathways, arcs, and curved lines that the body makes in space. Analyzing rounded versus linear patterns.
- Axis Points – Considering points on joints and segments of the body that serve as axes of rotation and balance. Identifying the key pivot points around which motion occurs.
- Synchronous Limbs – Analyzing the coordinated movement between different limbs, sides, or parts of the body. Looking at synchronous versus asynchronous body movements.
- External Body Spaces – Exploring the spatial patterns, geometry, and relationships between the body and environment. Observing the directions, planes, and dimensions moved through.
- Shifting Relations – Investigating how transitions between movements and poses direct audience attention and focus to different body parts and actions. Analyzing how progressions from one form to another create a seamless flow.
Mythology
As part of the research, the team drew inspiration from the mythological layer that creates a parallel to their work. The fundamentals of Mae Bot Yai are traditionally used to depict the Ramakien, a significant work of Thai literature based on the ancient Indian epic, the Ramayana. One passage in the Ramakien that has inspired this work portrays the demon king Thotsakan (known as Ravana in India) grappling with haunting visions in his dreams. Bibhek, Thotsakan’s brother and a shaman, interprets these dreams as omens of an impending apocalyptic battle that threatens to destroy the demonic realms. Enraged by Bibhek’s interpretation, Thotsakan banishes Bibhek from the kingdom, inadvertently pushing him to become an ally of Prince Rama, which ultimately leads to Thotsakan’s downfall.
The cautionary tale of heeding a premonition holds significant relevance in the contemporary era. The story of Bibhek, who foresaw the destruction of the demons, serves as a powerful metaphor for the potential threat AI poses to human artistic creativity and spirit. Instead of casting aside this complex, perilous vision, this project proposes an alternate path where humans could have a symbiotic relationship with technology, cultivating a novel form of contemporary dance. In this project, the artist’s dream presents a glimpse of how tradition can thrive in the contemporary world, where machines, humans, legends, and myths intermingle. Just as Bibhek’s unheeded warnings proved, valid, prophetic dreams of posthuman choreography begin materializing through peaceful machine kinships rather than destructive rejection.
Technology
The integration of technology in dance has its roots in Merce Cunningham‘s groundbreaking use of 3D computer graphics in 1989 to create choreography, as well as the Bauhaus Ballet‘s influence on applying geometric and mathematical concepts to the human body and movement. Over time, computational choreography has advanced alongside developments in various emerging technologies.
This research presents an approach to advancing Thai traditional dance through computational choreography. The model encapsulates the Mae Bot Yai fundamentals, allowing human dancers to co-create performances and develop new choreography with a virtual dancer that embodies the intelligence of traditional choreographic knowledge.
The development process, involving interdisciplinary collaboration between technologists, choreographers, and dancers, consists of four major stages:
Capturing Movement Data: The creation of a virtual character begins with capturing a human dancer’s movements through motion capture technology. This step provides the foundational movements for the virtual character, which the computational model can then modulate and modify to generate novel choreography.
Encoding Choreographic Intelligence: Building an AI model involves translating principles from traditional dance knowledge into a computational process that manipulates and transforms the captured movement data of the virtual character. This model equips the virtual character with the choreographic intelligence derived from traditional dance.
Interface Development & System Integration: Integrating the model and virtual character into an interactive interface that allows the dancer, choreographer, and audience to control various aspects of the system. This interface supports real-time, responsive, and improvisational co-creation through voice control, enabling users to dance alongside the virtual agent.
Rehearsing Human-AI Co-dancing: Collaborative performances and exercises utilizing this technology explore new choreography through human-AI interaction. These endeavors act as an experimental research platform for the future of dance.
Intelligence
We developed a computational model of “choreographic intelligence” based on traditional dances, which is capable of responding to and choreographing with human dancers in real-time. We acknowledge the importance of non-Western knowledge systems that are often overlooked in the colonial understanding of AI. This step integrates traditional dance principles and techniques into a model composed of computational procedures that serve as adjustable parameters to alter movement data in virtual characters.
Building this model requires a thorough analysis of the six choreographic principles of No. 60. The research team then creates corresponding generative computational procedures to manipulate the virtual character by dynamically adjusting animation keyframes and modifying specific joint movements using TypeScript. These generative procedures aim to replicate the impact of the choreographic principles on the virtual character, enabling the system to create new movement possibilities while preserving the core essence of traditional dance vocabulary.
Here are the algorithmic interpretations of the six choreographic elements:
Energy
This principle represents the dynamic range of motion in different parts of the body over time, which is important to the aesthetics of Mae Bot Yai, such as the unique knee movements and rhythmic stamping actions. To computationally model energy, the algorithm scales the timing of the different groups of the limb’s animation keyframes to increase or decrease the velocity of each individual part’s movements, resulting in a variation of movement speed across the bodies.
Circles & Curves
This principle represents the circular and curved movement trajectories with rounded rather than linear pathway transformations, which make the Mae Bot Yai dance fluid, graceful, and pleasing to watch. To computationally model this principle, we apply mathematical equations to the rotational quaternions of the limb’s animation keyframes, such as a Gaussian smoothing filter, derivatives, low-pass, and high-pass filters. This results in an increase or reduction of curvature in the movement.
Axis Points
This principle represents key pivot points and body segments, which serve as the reference points for hand movement. The hand points towards this reference point, creating elaborate hand and finger gestures in Mae Bot Yai. To computationally model this principle, the algorithm applies inverse kinematics and smooth linear interpolation to gradually interpolate the position and rotation of arm and leg limbs towards the core axis points. The original movement is altered to gravitate towards the core skeletal axes points.
Synchronous Limbs and Shifting Relations
These two principles represent opposing qualities in Mae Bot Yai, where synchronous limbs create synchronous bodily movement across the body, and shifting relations create an asynchronous movement that draws the audience’s attention to specific body parts. To model these principles, the algorithm applies timing offsets to the individual limb’s animation keyframe to de-synchronize the limb movements from the original movement. While the original movement is often mirrored and synchronized due to the nature of the dance, the resulting movement becomes more independent and uncoordinated.
External Body Spaces
This principle represents the negative geometric shapes outside the body that create beauty in Mae Bot Yai. In order to computationally model this principle, the algorithm detects transition signals between sequence positions to slow down the movement and influence the audience to see the negative space in the frozen pose. The algorithm identifies the movement sequences containing minimal rotational changes and then extends the timing of the animation keyframe for those sequences to pause the movement. The resulting movement highlights the external spaces around and within the dancer’s body.
Joint Rotation
Additionally, outside of the six principles derived from Mae Bot Yai, we have introduced additional parameters such as Joint Rotations. This parameter allows the algorithm to manipulate the virtual character’s joint angles and orientations.
In reality, a dancer’s joint rotation is restricted by their own physical capability. However, in the realm of the virtual, such restrictions are obviated. By experimenting with the joint rotations, we can create new, novel body configurations for the virtual character, enabling movements that would ordinarily be impossible in reality. This expansion into possibilities that transcend physical constraints deepens the dance vocabulary of the virtual character and further heightens the creative scope of the system.
Parameters
The developed model features a set of adjustable parameters that embody the principles of No. 60. These parameters can be modified by both performers and spectators, allowing for exploration of how a given principle affects a specific movement. When a parameter is set to 0%, the virtual character performs the original dance move without incorporating the principle. On the other hand, setting the parameter to 100% significantly alters the character’s movements, fully integrating the specified principle. The model allows for flexible use of these elements, either individually or in combination, and can be applied to the whole body or specific body parts. This versatility makes it a valuable tool for studying the dynamics of motion and the impact of various algorithmic procedures.
The responsive user interface is developed with the Vue.js framework. The virtual character is rendered and integrated into a 3D environment using the Three.js library. The interface supports voice recognition, allowing users to issue spoken commands to control the character’s movements. Choreographic elements are accessible via dropdown menus, where users can, for example, select “Energy” and then specify details such as which body parts to modify. Additionally, slider controls provide fine-tuned parameter adjustments, while timeline widgets enable the visualization of movement sequences. Human dancers can observe the character’s live animated responses, facilitating an improvisational, call-and-response interaction during the performance. The interactive interface is projected on a screen, allowing the dancer to interact with the virtual character in real-time. This real-time interactivity promotes creative experimentation between the human and virtual dancer, leading to the co-development of innovative choreography that merges tradition and technology.
Human-AI
Co-Dancing
To investigate the creative potential of Human-AI co-dancing, we organized a two-week workshop involving technologists, choreographers, and dancers. The workshop integrated rehearsals, reflection, and technological development to delve into computational choreography, merging tradition with technology. During these sessions, the human dancers and virtual characters engaged in improvisation, reacting to each other’s movements. The dancer channeled their traditional vocabulary while the virtual character exhibited new responses generated algorithmically from traditional choreographic knowledge.
Different interaction methods were explored to enable co-dancing with the virtual character in real-time. In the “Direct Control mode,” the dancer could activate computational procedures by voicing specific principles and providing immediate feedback during the dance. In the “Mediated Mode,” another dancer or choreographer adjusted the parameters in real-time, allowing for more intricate commands without hindering the primary dancer’s movements. The most intriguing results emerged from the impromptu “Audience Control mode,” where viewers changed the movement of the virtual character mid-performance, allowing for emergent collective choreographic interventions.
During co-creation exercises, the human dancers were challenged to intentionally resist, embrace, or conceptualize contrasting responses to the virtual agent’s generative dance offerings. Attempts to harmonize with machine-generated interpretations opened spaces of synergy, where hybrid vocabularies organically arose from fusions of traditional principles and synthetic outcomes. Alternately, foregrounding moments of rupture, friction, and opposition highlighted the technology’s non-human quality against the dancer’s physicality. This fluid spectrum between resonance and dissonance highlights the complex interplay between humans and AI. The emerging choreographic process points to the rich potential for developing future posthuman performances
After the workshop, the culmination of this research was the dance production “Cyber Subin,” co-created by Pichet Klunchun and Pat Pataranutaporn, which debuted at the National Theatre in Taiwan in March 2024.
Discussion
As the human dancer physically challenges, resists, follows, and improvises with the virtual dance partner’s computational interpretations of traditional dance vocabularies, intriguing frictions arise that highlight each intelligence’s distinct perspectives and capabilities. This dynamism suggests rich questions around autonomy, influence, control, and co-dependence as humans and virtual agents entangle through points of resonance and dissonance. Such interplay prompts vital discourses on the essence of the human spirit and freedom when choreographically coupled with non-human outputs operating in the same social and cultural space.
The influence of the machine on the dancer and the dancer’s influence on the machine, forming a cybernetic loop, prompts the inquiry into the future of human creativity. It provokes the question of how human-machine assemblage could be more than the sum of its parts. What role does technology play in re-animating wisdom from the past? How can integrating diverse epistemologies into computational systems counteract humanity’s potential homogenization and preserve experiential heterogeneity amidst rapid technological change?
Situating this research within the concept of “experimental preservation” offers an alternative approach to safeguarding cultural knowledge. Unlike traditional preservation methods that often present culture as a static and lifeless object, experimental preservation acknowledges culture as a living, evolving phenomenon that belongs to everyone. By democratizing cultural knowledge, such as traditional dance, through technology, this approach could foster a sense of ownership and connection to ancestral heritage for future generations. Ultimately, this paper presents an imaginative vision for the future of intangible cultural expressions through the fusion of dance, data, code, and culture.
In conclusion, the Human-AI co-dancing paradigm presents a novel approach to preserving and extending traditional dance knowledge through the integration of human dancers and virtual dance partners. By deconstructing the choreographic principles of Thai traditional dance and translating them into computational procedures, the developed interactive system allows for the real-time synthesis of virtual characters’ movements. The incorporation of voice control functionality further enhances the participatory nature of the system, enabling dancers, choreographers, and even audiences to contribute to the choreography of the virtual agents. This work highlights the importance of diverse cultural knowledge systems that are often overlooked in modern perspectives on AI. The developed system facilitates emergent choreographic dialogues between the past, present, and future of this art form. This is a 100 year journey of Thai choreography.
Team
Co-creator:
Pichet Klunchun
Pat Pataranutaporn
Choreographer/Director:
Pichet Klunchun
Dancers:
Padung Jumpan
Tas Chongchadklang
Chang Hong Chung
King Fai Tsang
Music Director and Composer:
Lamtharn Hantrakul
3D and Animation Creator:
Piyaporn Bhongse-tong
Creative Technologist:
Phoomparin Mano
Chayapatr Archiwaranguprok
Cyborg Scientist/Philosopher:
Pat Pataranutaporn
Lighting Designer:
Ray Tseng
Dramaturg:
How Ngean Lim
Producer:
Sojirat Singholka
Stage Manager:
Jirach Eaimsa-Ard