At first, I did not see the value of building robotic painters. I first came across them in computer graphics research. For many years, many of us worked on algorithms for digital painting and drawing. Our goals as a community were to design algorithms that could automatically “paint” and “draw” in many artistic styles, thereby creating new artistic tools. We tried to model different kinds of existing artistic styles algorithmically, and we worked entirely in software. Some researchers in this community began incorporating robots inside the painting process. While the work was impressive, I didn’t see the point of the robot itself. Why not do the whole thing in simulation?

Now a new wave of artists is building robotic painters, where a robot arm paints or draws directly on paper or canvas. My first response to this work was again: why not do this in simulation? If you need a physical artifact at the end, then just print it out.

After all, every printer is a robot, and so any artwork you print out has been drawn by a robot. Conversely, a robot arm is basically just a difficult-to-use printer. While one can’t model how people paint without including the drawing itself in the loop— we humans cannot fully plan an artwork in our heads—any painting robot you can build could also be modeled in a computer simulation.

Putting human-like features on what is a basically a printer also advances the harmful myth of Artificial Intelligence, that the robot is genuinely intelligent and creative, rather than just another software system. We perceive life in the motion of a robot, just as a skilled puppeteer can bring a piece of cloth to life. The difference between the robot and the puppet is that the audience knows how a puppet works, and won’t be fooled into thinking the cloth itself is intelligent. Robot painters can be a powerful tool for misleading stories of “creative AI.”

The illusion of life is powerful even when the mechanisms are apparent

But the more I’ve learned about the current set of artists using robots, the more I’ve come to appreciate them.

For any robot painting project, I think we should ask the following question: Why not generate the image in software and then print it out? If there is no good answer to this question, then the robot is just a gimmick.

In current artworks, I see three possible answers to this question.

Physics affects style

The first works that changed my mind were by Patrick Tresset, who began building a system called Paul in 2011, and has been developing new versions since.

The system draws with pen on paper, controlled by a robot arm. In these drawings, I perceive gesture and style that come from the pen being controlled by a physical arm. In some ways the style looks hand-drawn, while other stylistic features seem non-human.

The drawings show a sense of gesture, an art term referring to the expressive use of brush strokes. When you see a brush stroke in many kinds of paintings, it’s easy to imagine the artist’s hand movements. Even when comparing messy handwriting or very neat and detailed hand-writing, you get a sense of the writer’s hand-movements, and, through them, perhaps some perception of their personality or mental state. These robot pen strokes also convey some sense of gesture and personality through the physical mechanism that produced them.

The physical mechanics of your arm, hand, and neuromuscular control system affect how you write or draw; your arm is not just a plotter. For example, motor control is noisy: the control signals from your brain never precisely get executed by your muscles, and, how your pen moves across paper likewise is affected by the random texture of the paper. In one motion controller project, we found that animation controller styles can respond to changes in motor noise and other sources of randomness.

Hence, robotic artworks likewise can be interesting when the robot is not just a plotter, but when the mechanical system itself affects the resulting style. Plotters can precisely follow any given path for a pen. When the mechanical drawing system is imperfect and follows its instructions imperfectly, a sense of gesture could emerge. Style may arise indirectly when certain brush paths are difficult or impossible to produce. While perhaps this mechanical system could be simulated, working with the real physical system is more direct.

I can’t tell to what extent Tresset’s robots are equivalent to plotters: is the artwork different each time even if run on the same inputs (with a fixed random seed)? Watching the video makes me think there is at least a little bit of variability arising from the physical system.

Daniel Berio, a graffiti artist-turned-roboticist, has done both: modeled in software how the mechanics of the hand affect drawing, and also built robots that create graffiti. Here, the robot seems to be more of a wall printer, also the output is not entirely predictable, e.g., the way the paint drips.

(I think I’ve also seen Joanne Hastie mention that the kinematic limitations of her robotic arm affects the range of curves that she uses. For example, there are certain kinds of curves the system can’t draw. This limitation affects the styles of her robot paintings.)

(Incidentally, both Tresset and Berio began developing their systems during PhD studies with Frederick Fol Leymarie at Goldsmiths.)

You get a different artifact

Joanne Hastie’s robot arm controls a physical paint brush. What you get in the end is an physical artwork with real paint on it. While I haven’t seen her works up close, I expect that look qualitatively different from anything that came out of an ordinary color printer. Having real paint on it makes it different kind of aesthetic artifact than a print-out, and real paint can be very pleasing to look at.

This physical output can also lend a sense of uniqueness. People care a lot about uniqueness when purchasing artworks; a single painting is seen as more valuable than a digital print. I suspect that, with enough effort, any robot paintings could be reproduced. We mainly have the artist’s promise that they won’t. And that’s good enough for me, and it has been good enough for a century of collectors of photography, printmaking, casted sculpture, and other kinds of mechanical reproduction. I’d rather own a limited-edition print of a digital work that I love than a unique painting that I merely like, though others may feel differently. But, given the choice, I’d rather have a physical painting rather than a print of the same painting.

It’s a different medium

When you create artwork with different media, you get different art. Using watercolor, one naturally gravitates towards different visual styles than with using oil paint. Likewise, I recently found that I gravitated to different visual styles with digital painting than I had with real-world painting. Likewise, I have no doubt that writing code to produce images with a robot is a different medium than writing code to produce images on a screen.

The effect of the medium is subtle and hard to appreciate. Since I haven’t tried building robot painters myself, it’s hard for me to imagine the specifics of how these media differ beyond what I’ve written above. For the most part, we as viewers can only guess.

However, the role of the robot is more visible in the work of Sougwen Chung, who “collaborates” with robotic arms on paintings and drawings:

In principle, it should be possible to do something like this on the screen. But the fact that she can use a real pencil or brush means she paints in a different way than if she used a tablet or mouse. And, of course, she will produces a unique physical artifact in the end. Her work also includes a performance aspect that wouldn’t be the same if she was just drawing on a screen.

The Embodiment Question

A big question in studies of intelligence, both human and artificial is: to what extent is embodiment necessary for true intelligence? That is, we as humans learn by interacting with the real world, and perhaps intelligence is only possible with some kind of physical embodiment as well.

One could ask a related question for art: how does the physical embodiment of the painting system affect the artwork? Obviously, embodiment is not necessary for computer art. But the nature of the work may be fundamentally different. Neither is necessarily better or worse, but the physical embodiment can its own unique artistic capabilities.

One reason to pursue these art projects and research projects is to explore this question. I suspect that some of the above artists and researchers began these projects without a clear idea of what they’d find. I certainly wouldn’t have thought any of the above thoughts without having seen their work. I do maintain some skepticism for robot painting. Nonetheless, it seems likely that future artworks in this space will come up with interesting new answers to this question.

Thanks to Tom White and Patrick Tresset for pointing out links in a previous online discussion.