Multitouch Art, Sand to Silicon

My Dad sent me this video today. Apparently it's been doing the rounds since 2009 but I'd not seen it. The video is from the TV show Ukraine's Got talent and contains eight and a half minutes of astounding 'sand animation' by Kseniya Simonova.

Take a little break and watch the performance of 'The Great Patriotic War' here. Link to large size video, which I recommend. 8:30 long.

There are three exceptional things about this video. One — it's great art — enthralling performance, emotional themes, beautiful imagery. Secondly, the performance itself is technically amazing, yet, apparently the artist has only been doing this for one year. Finally, all of this is being achieved with some plain sand on a flat (backlit) surface. The tools for art don't get much simpler.

And yet…this is exactly the type of real-time, subtle, organic, sensual and fast art I always imagine computers could be capable of. Unlike many swooshy multitouch demos, this is not art for art's sake, instead the animation covers very human topics; one of every four people in the region died in WWII's Eastern Front. And she's using every last creative aspect of sand, from brushing, to finger and palm painting, throwing sand and scraping with the edge of her palm.

Two Hands are Better Than One

So this is how great it can be with some sand. How about some silicon? Matt Gemmell wrote a great piece on iPad application design I enjoyed. On the topic of the iPad's large, multitouch area, he writes…

The important point is that there are other, more obvious ways to accomplish these things; the two-handed input features are conveniences and power-user features. They’re useful and time-saving and possibly discoverable, but they’re not the only way to accomplish those tasks. We’re only just beginning to come to terms with the possibilities of dual-handed input; essential functionality shouldn’t require it yet.

You can see in the video that Kseniya rarely uses two hands. My stopwatch recorded only 1:15 minutes of two-handed use in the eight-and-a-half minute performance. That is, she only uses two hands simultaneously in this performance — 15% of the time. When she does, it's to do something quickly like clear an area. She also seems to use two hands when she's wants to draw symetrically, like the hair at 3:43.

 The matter is not that simple though. Many times she switches hands in the performance because she wants to draw on the far left (she appears right-handed) or because she wants a particular shape, or needs to approach from a particular side. 

Sometimes she switches for speed, and artistic effect; alternating left and right throws.

 

Just the Tip(s) of the Iceberg

I love this video because of the richness in the interaction. It's an encyclopaedia of gestures, from a single finger-painting, to multi-finger dabbing, parallel lines with thumbs and middle-inger. French-curve arcs with a palm, broading erasing strokes with the whole hand and intricate air-brush effects with sand released from above. I agree with Matt: we are at the beginning of this whole wonderful adventure. I'm going to keep Kseniya performance in mind as something to strive for. This is a great interface. 

Comments

[Alisey Lebedev]

And she's beautiful. I live in Ukraine but I haven't seen this video before.

[Adam Behringer]

Great post Keith! It is an amazing performance and insightful observations on your part.

[Alexandr]

I think she is not using both hands to avoid blocking the image from the camera.