Sometimes when I sketch quickly, I feel like I’m scribbling some random squiggles, and, yet, recognizable shapes of my subjects emerge:

This experience feels a bit like magic when it works out, like pure fluid intution. And, to me, the drawings are visually interesting because many of the strokes do not correspond to visually-meaningful features, the way more conventional line drawings do.

This kind of scribbling also shows up in more-recognizable drawings. A lot of drawing a scene is about filling in plausible, sensible details rather than precisely drawing what’s actually there.

Here are a few drawings of dogs; key outlines and features are visible, but a lot of the detail and texture is just scribbling:

Sometimes I like to think of my sketching as “continually scribbling until it doesn’t look wrong.”

Statistical recognizability

To me, these kinds of drawings are suggestive of a sort of “statistical visual recognition,” by which I mean “recognition from statistics of a single image.” These drawings do not necessarily have natural visual features like contours, nor do they seem to fit with the usual demonstrations of Gestalt principles or the visual illusions (e.g., Kanizsa triangle) that illustrate perceptual inference. It’s not just scale space; e.g., downsampling the picture doesn’t make the object more recognizable, as like in Impressionism, hybrid images, or Arcimboldo paintings.

Instead, they seem more statistical, and, several abilities of human vision have been described in terms of the statistics of an image, including texture recognition, peripheral vision, and rapid scene recognition (“gist”). With some image classification deep networks have shown similar textural recognition properties, such as the adversarial examples in the “intriguing properties” paper and a paper that showed that these networks relied heavily on texture recognition, ignoring shape and other 3D information.

As a model for what’s going on here, the work of artist Tom White offers something very suggestive. For example, here’s a picture of an electric fan that Tom created in 2018:

To create this print, he created an algorithm that optimizes the placement of strokes and blobs, maximizing the confidence with which a state-of-the-art image classification algorithm (as of 2018) classified the image as “electric fan.”.

Here are a few more images from that series: each image is optimized to look like a different object class:

You can see more of his work here, including abstract pictures optimized to activate NSFW classifiers. Those examples look abstract to us, but to NSFW filters (as of 2018), they look absolutely smutty. I’m not including them here, just to be safe, because I don’t want to this post to be tagged as explicit content.

There are other examples, like Matty Mariansky’s Evolutionary Faces. I think this also relates to other ideas in different kinds of ambiguous imagery, such as Rob Pepperell’s notion of indeterminate imagery, which can be produced by machine learning models too.

The path to abstraction

Abstract art seems foreign and obscure to a lot of people. How do you appreciate it?

For me, one gateway to appreciating abstract art has been on a path from realism—what I would now describe in statistical terms. One of my favorite artists is Richard Diebenkorn, and you can see something like a progression or continuum from realism to abstraction in his work:

The last one is from Diebenkorn’s Ocean Park series, painted when he lived near the beach in Southern California. In the Ocean Park paintings, you can see the general statistics of beach landscapes. There are small shapes near the horizon and large shapes near the bottom, just as there would be in a perspective drawing or photograph with the camera near the horizon.

Many historical Modern artists, and their precursors, who began realistic, but then stripped away some notion in their processes. For example, Piet Mondrian painted realistic trees, then increasingly abstract trees, ultimately evolving toward the purely abstract style for which he was most famous. This took him over a decade.

I think there is a strong case to be made that the aesthetics of abstraction are tied to the statistics of real experience, including old observations that image statistics can predict aesthetics, and Jackson Pollock paintings share key statistical properties with real-world imagery.

I see my own scribblings as a move toward abstraction. In the few situations when I have experimented with abstract drawing, I’m using the skills of realistic drawing (and scribbling), but removing recognizability.

One way this manifests is when sketching on my phone, with my finger, which is naturally quite clumsy:

Another way is when I try to draw scenes out the window of an airplane (as mentioned in that previous post): the landscape is continually changing, so it’s impossible to capture a “snapshot:” any drawing is just pulling visual elements from what I see and mostly just creating some abstraction. Here are drawings from two plane flights:

Directions

Many of these ideas have been explored in the vision science literature in some way, but I think there’s more to do here. Specifically, the idea of image recognition as being based on statistical properties, together with the role of statistical descriptions of both realistic (but scribbly) and the hypothesis that abstract art “works” in relation to familiar, real-world scene statistics.