Three Types of Art Technologies

The impact of “AI” upon art is complex and multifacted, with different effects on different communities, and different behavior in the short-term versus the long-term. We’re now inundated with glib utopian slogans (“AI will democratize creativity!”) and simplistic dystopian fearmongering (“AI will kill art!”), both of which have elements of truth while also being wildly misleading.

In a previous blog post, “When Machines Change Art”, I described how previous artistic technologies provide useful guidance for understanding the development and impacts of new technologies, with many historical examples.

But, different kinds of technologies have had different kinds of impact. Recorded music and photography are both relevant analogies to “AI” art, but “AI” art is much more like the latter than the former.

Now I propose a categorization of artistic technologies, in order to identify both how these kinds of technology differ and how they all ultimately end up changing art.

There are three categories in this list, plus a hybrid category.

1. Direct Control

Direct Control technologies replace an existing technology with similar kinds of controls to what an artist is used to, but removing some low-level drudgery, and adding more convenience. Of the categories here, these are often the least-threatening to traditional artists.

When I switched from physical painting to digital painting, it felt similar in many ways, with many of the same skills involved, but with digital capabilities like “undo,” and no physical materials or mess to deal with.

Yet, as soon as you switch to a different way to create, you create different things. An artist’s style still may come through clearly with these kinds of technologies, e.g., compare Hockney’s physical to his digital paintings. Nonetheless, using a new medium changes one’s style, especially if one learned on that medium. With my digital paintings, I am exploring many new styles that I didn’t or couldn’t with physical media, even though I think there are elements of my style that remain.

You can use a new medium to work in an old way, but this is not the trend; each media pulls you in its own way because of its affordances. I recall an old painting teacher saying that you could use watercolor like it’s oil paint, by using very little water, by why would you? “Respect the medium,” he said.

There are many more examples; it’s hard to think of a physical-world artform that hasn’t been digitized in some way. For example, the progression from hand-written novels, to typewriter, to word-processor; digital photography (replacing film), and digital video editing.

In the change of medium, it feels like “cheating” to be able to use “undo” and layers and all the other conveniences offered by digital painting. But I came to realize that it’s no more “cheating” than any other digital artistic tool, such as “undo” in a word processor, or digital video editing tools, which at times, traditionalists looked down upon. Even Michaelangelo thought that oil painting, which was much easier than traditional fresco, was “for amateurs.”

But I do think that the skill and effort involved plays a key role in how these media are valued, both in the artist’s valuing of their own skill, as well as others’. A physical oil painting may, even today, be valued much more than a digital one by the same artist, because of some combination of visible physical effort, permanance and scarcity, and/or traditional notions of value.

2. Distribution Mechanisms

Recorded music, broadcast television, video tapes, online video streaming: all of these technologies provided new ways to experience the same things as before. With music recording, you could hear music without being in the same room as the musician; with online video streaming, we could watch movies without having to go to a theatre or get a DVD. Movable type, which made it far easier to print books, transformed history, against the resistance of religious elites.

These technologies might not seem like they change anything but the way we get our media. But they ultimately have profound impacts on the art itself, and the way artists and companies make money.

Consider the way that music recording led to all sorts of experimental music that could only happen with recording (e.g., The Beatles, Musique concrète), to album-oriented rock, to hip-hop (based on turntables and faders), to sample-based electronic music, and so many other innovations.

But performance musicians campaigned against recorded music, since it took away from performance jobs. Likewise, corporations campaigned against newer distribution technologies like video-cassette tapes and file sharing, saying they would kill the television and music industries.

And these labor battles continue. As a friend of mine working in Hollywood put it, “each new technology seems to be a way for studios to find ways to not pay us a fair amount.” Broadcast television led to ways to underpay artists, leading to strikes and the residuals system, and then the studios used the streaming service model to avoid paying residuals, leading to the more-recent strikes. Each of these new distribution platforms provided studios ways to avoid paying artists fair compensation.

3. Decision-making

Oil painting and digital painting seem like fundamentally similar artistic activities.

Technologies that make pictures semi-automatically, like photography, generative art, and text-to-image, automate all of the low-level decisions, providing mainly high-level controls to the artist. Many of the skills involved in using these technologies are radically different from the traditional technologies, and learning curve in using them is much easier. To a traditionalist steeped in the old technology, sometimes the new technology doesn’t look like “art” at all.

Hence, these technologies, such as photography, generative art, and text-to-image offer the most apparently-radical shifts to making and understanding art, threatening the livelihoods of traditional work-for-hire artists, while making us rethink the roles of technical skill in the nature of art.

4. Hybrid

Although I’m presenting a categorization, there’s really a continuum, and most technologies will operate on more than one of these axes time.

3D computer animation, like Pixar’s Toy Story, provided both digitization of old tools, but also automation of many low-level decisions. Whereas digital ink-and-paint was simply a digitization of traditional hand-drawn animation, computer graphics (like Pixar’s Toy Story) both required similar animation mastery as in hand-drawn animation, but very different techinical skills (moving control curves rather than drawing pictures). In some ways, computer animation authors details—much of it is like cinematography—and, in other ways, it is directly controlled by animators the fine-tune every aspect of the motion and shape. So it straddles the mechanization and decision-making categories.

Where does “AI” fit in?

“Artificial Intelligence” is a blanket term that means many different things; a better word would be “software systems,” since almost every kind of software system has been called at “AI” at some point. These days “AI” mostly means data-fitting algorithms with various wrapper code around it.

Different “AI” systems could each fit into different parts of this categorization. “AI” can be used to augment digital painting with painted texture synthesis; or, like text-to-image, it can be used to automate many high-level decisions. And, insofar as LLMs memorize information, they could be viewed as different ways to transfer and retrieve information, replacing databases and search engines (and there there are some obvious dangers in using them this way).

All technologies change art

All of these technologies change art: in Marshall McLuhan’s gnomic words, “the medium is the message.” Even oil paint and digital painting have different histories, different styles, even different meanings. While you could theoretically paint the same picture with each, one almost never does. Likewise, photographs, painting, digital painting, and text-to-image might all produce pictures than can be represented as pixels, but these pixels all mean different things. And it can take years, or decades, to fully understand what all those impacts are. However, history provides a guide to understanding what they might be, how the hype is misleading, and how we might consider the societal impacts.