Co-creation with robots

Sougwen Chung, the artist whose work we have featured in this issue,explores the collaboration between humans and machines in her artisticprocesses and performances. This gives her a direct insight into the potential mutual learning that AI opens up for us.

evolve: How did you come to be interested in the intersection of art and technology?

Sougwen Chung: I came to be interested in art and technology from an early age – my mother was a programmer and my father a musician. I grew up with musical training in instruments and access to technology, and an interest in the early internet. It’s read that the experiences you have in early childhood shape the foundation of how you experience the world as an adult. In that vein, I’ve always found a way to express myself through these mediums as an alternative to verbal communication. I often believe there’s something inherently resonant about artistic practice that cannot be imbued any other way.
The intersection of art and technology has seen many shifts during my years of being a practitioner. It’s an ever-shifting nexus, and in its ambiguity it invites various kinds of interpretation. It can be editorial or critical. Maybe in my work it’s more relational than anything else.

»It led me to the realization that maybe part of the beauty of human and machine systems is their shared inherent fallibility.«

e: What was the impulse to start co-creating with AI technology and robots?

SC: Many of us here use technology in our day-to-day. And some of us rely on technology to do our jobs. For a while, I thought of machines and the technologies that drive them as perfect tools that could make my work more efficient and more productive.

But with the rise of automation across so many different industries, it led me to wonder: If machines are starting to be able to do the work traditionally done by humans, what will become of the human hand? How does our desire for perfection, precision and automation affect our ability to be creative?

Art and research are both about learning by doing. And impulse is the right word – a curiosity, a need to be satiated. I was curious about the robotic form, its ubiquity in culture and the robotic arm as a symbol. A symbol of displacing human labour, a machine that replicates human action. There’s a duality inherent in how we frame a robotic unit. I’m not alone in my fascination with the mechanical double, the robotic other.

When I started I wanted to explore the robotic form as a metaphor but also as a material, as a way to enhance my understanding of what it takes to build a “thinking machine”. Even after so many years, I’m still learning more about how the technical systems that have come to be so prominent in the world are built, maintained, and how that differs from the constructed idea of them in society.

This thread of interest and thinking coincided in 2015 with a popularization of the term AI driven by advances in deep learning. One of the stories that would come to define the landscape, editorialize the technology en-masse was the story of Lee Sedol and Alpha Go, a software program designed by DeepMind to play WeiChi. After Lee Sedol, the top WEiChi player in the world was defeated, he mentioned being stunned at “the beauty of a non-human move”.

What was interesting to me about that particular moment was not only the novel and unusual outputs by the computational system but the notion that through the interaction a master at their craft could experience a moment of appreciation and beauty beyond what was in their original capacity – a sense awareness of the notion of the non-human as a source of awe.

And I think that awe is at the center of the question I ask in my practice, and of myself — of where does AI end and we begin. I like to say that there's no such thing as artificial intelligence because there's no such thing as a single natural intelligence. All these systems are extensions of us in some way. It's one of those questions that’s like: what comes first, the artwork or the question? And I have to say, because I find systems, and building systems, and understanding them, really intellectually and creatively stimulating. That was my original intent with exploring this robotic gesture drawing system. From this "collaborative" performance on canvas with these marks, one of the questions that arose from it was who did what in this drawing? It's not clear. I think it was almost a criticism, but it's not clear how to distinguish the authorship of the mark making. I actually found that to be a more interesting question around creative extension, around what it means to collaborate with the tools we build.

All the drawing operations are extensions of me, and our perception of it being other is a deeply human one. When we can start to see that the AI system is actually us in another form, in another mode of temporality, then we're heading in the direction around a multiplicity of intelligences and approaches to intelligence that genuinely create something new, versus these notions of the singularity and novel AI, AGI—I think it's a little bit of a Silicon Valley pipe dream.

e: What did you learn when you started with D.O.U.G 1?

SC: In my work as an artist and researcher, I explore AI and robotics to develop new processes for human creativity. For many years now, I've made work alongside machines, data and emerging technologies. It's part of a lifelong fascination about the dynamics of individuals and systems and all the messiness that that entails. It's how I'm exploring questions about where AI ends and we begin and where I'm developing processes that investigate potential sensory mixes of the future. I think it's where philosophy and technology intersect.

Doing this work has taught me a few things. It's taught me how embracing imperfection can actually teach us something about ourselves. It's taught me that exploring art can actually help shape the technology that shapes us. And it's taught me that combining AI and robotics with traditional forms of creativity -- visual arts in my case -- can help us think a little bit more deeply about what is human and what is the machine. And it's led me to the realization that collaboration is the key to creating the space for both as we move forward.

It all started with a simple experiment with machines, called "Drawing Operations Unit: Generation 1." I call the machine "D.O.U.G." for short. Before I built D.O.U.G, I didn't know anything about building robots. I took some open-source robotic arm designs, I hacked together a system where the robot would match my gestures and follow them in real time. The premise was simple: I would lead, and it would follow. I would draw a line, and it would mimic my line.

I learned so much when I started D.O.U.G 1. We began with mimcry, which was significant because it was a simple positional mirroring at the heart of drawing. The decisions we made were between colour tracking and infrared, both reductive visual spectrums that are machine-readable vision processing complimentary to human vision.

Through MIMICRY, the first generation of Drawing Operations, the bugs, glitches, and inaccuracies of the robotic tracking became a point of gestural adaptation as the work was created on the page in real-time in front of an audience. The error state became centred as a visual style. For me, this translation of the embodied mark as a machine movement, however simple, became the first cornerstone in what has become 10 years of engaging in feedback loops and relational configurations.

»Where does AI end and we begin.«

e: Can you explain how the process of the different versions of D.O.U.G has been for you and what you learn about the collaboration of human wit AI technology?

SC: So back in 2015, there we were, drawing for the first time, in front of a small audience in New York City. The process was pretty sparse -- no lights, no sounds, nothing to hide behind. Just my palms sweating and the robot's new servos heating up. Clearly, we were not built for this. But something interesting happened, something I didn't anticipate.

D.O.U.G., in its primitive form, wasn't tracking my line perfectly. While in the simulation that happened onscreen it was pixel-perfect, in physical reality, it was a different story. It would slip and slide and punctuate and falter, and I would be forced to respond. There was nothing pristine about it. And yet, somehow, the mistakes made the work more interesting. The machine was interpreting my line but not perfectly. And I was forced to respond. We were adapting to each other in real time.

And seeing this taught me a few things. It showed me that our mistakes actually made the work more interesting. And I realized that through the imperfection of the machine, our imperfections became what was beautiful about the interaction. And I was excited, because it led me to the realization that maybe part of the beauty of human and machine systems is their shared inherent fallibility.

Since then, it has been a conceptually and technically faceted evolution of MIMICRY (D.O.U.G._1),  to MEMORY (D.O.U.G._2) to COLLECTIVITY (D.O.U.G._3) to SPECTRALITY (D.O.U.G._4) to ASSEMBLY (D.O.U.G._5). Throughout each generation the work framed art practice as an ongoing engagement with research, art-making, performance, and technology, but also a way of living with or becoming with the iterative practice of creativity and development. It has raised questions for me about the evolution of the human hand and centred a sociotechnical understanding of the work beyond speculation, that produces research inquiry. What new relational dynamics emerge in collaboration with ai systems and human-and-machine feedback loops?

If D.O.U.G._1 was the muscle, and D.O.U.G._2 was the brain, then I like to think of D.O.U.G._3 as the family. I knew I wanted to explore this idea of human-nonhuman collaboration at scale. So, for instance, I worked with my team to develop 20 custom robots that could work with me as a collective. They would work as a group, and together, we would collaborate with all of New York City.

I was really inspired by Stanford researcher Fei-Fei Li, who said, "if we want to teach machines how to think, we need to first teach them how to see." It made me think of my life in New York, and how I'd been all watched over by these surveillance cameras around the city. And I thought it would be really interesting if I could use them to teach my robots to see. So with this project, I thought about the gaze of the machine, and I began to think about vision as multidimensional, as views from somewhere. We collected video from publicly available camera feeds on the internet of people walking on the sidewalks, cars and taxis on the road, all kinds of urban movement. We trained a vision algorithm on those feeds based on a technique called "optical flow," to analyze the collective density, direction, dwell and velocity states of urban movement. Our system extracted those states from the feeds as positional data and became pads for my robotic units to draw on. Instead of a collaboration of one-to-one, we made a collaboration of many-to-many. By combining the vision of human and machine in the city, we reimagined what a landscape painting could be.

e: How do you experience the creative process of the cooperation with AI machines?

SC: Throughout all of my experiments with D.O.U.G., no two performances have ever been the same. And through collaboration, we create something that neither of us could have done alone: we explore the boundaries of our creativity, human and nonhuman working in parallel.

The practice of engaging in a mark-making space becomes gesturally relational, so the conscious mind, for me, must in a sense become quiet and disengaged for the reactive, responsive, and adaptive flow-state to become centred, and the driver. Over the past few years, I have been thinking of it as a way to hold the oscillations of fear and hope of technology in the same space. It becomes a form of sensory extension, and a research-in-action. It challenges what I consider possible in embodied feedback loops with computational systems, real-time data, and space.

»The practice of engaging in a mark-making space becomes gesturally relational, so the conscious mind, for me, must in a sense become quiet and disengaged for the reactive, responsive, and adaptive flow-state to become centred, and the driver.«

e: How do you come to the ideas for a certain work or performance?

SC: Sometimes the arrive from a curiosity, a particular tension, a poem, a song or a feeling. It’s quite free-form, I try not to control where ideas come from or have a specific pattern I’m enacting to create work.

e: Is there something particular that you want to communicate to people looking at your works?

SC: Alternatives, possibility, curiosity, agency.

e: Why do you think it is important to understand AI systems as Non-human collaborators?

SC: The practice of decentering the human subject is a practice of opening. Opening oneself to the idea of the non-human, other-than-human, a view beyond one's self.  A relational mode of being.

e: You say “the beauty of human and machine systems is their shared inherent fallibility”. Can you say what you mean by that and what the implications are for our relationship to AI systems?

SC: It’s easy to exaggerate the potential of new technology; to lead to a sense that technological advancement is inevitable, and infallible, becoming like an addiction. The same can be said for human beings, which leads to hubris and a fear of the unknown or anything that interrupts or challenges that paradigm. It is an alluring conceit but it is also brittle, to believe so wholly in human or technological exceptionalism. Through the work, I explore the notion that the reality is more interesting, that by exploring our own limitations and the limitations of technology, we can create hybrid, relational forms that better address the uncertainty of the time in which we are living. It’s these points of inflection that can become catalysts for creativity, and philosophical inquiry.

With Scilicet, my lab exploring human and interhuman collaboration, we're really interested in the feedback loop between individual, artificial and ecological systems. We're connecting human and machine output to biometrics and other kinds of environmental data. We're inviting anyone who's interested in the future of work, systems and interhuman collaboration to explore with us. We know it's not just technologists that have to do this work and that we all have a role to play.

We believe that by teaching machines how to do the work traditionally done by humans, we can explore and evolve our criteria of what's made possible by the human hand. And part of that journey is embracing the imperfections and recognizing the fallibility of both human and machine, in order to expand the potential of both.

This interview was first published in the German evolve Magazin no. 40.