I’ve been thinking about visual languages and the morphology of symbols (as opposed to words) for a while. I had the opportunity to start putting some of these ideas into code at a really excellent openframeworks workshop this week, run by Joel Gethin Lewis and Arturo Castro.
Here’s what it does:
Makes the point nicely that symbols and spaces can intertwine.
Using opencv blob detection, the regularity, direction and area of the shapes map to envelope modulation, resonance and pitch. The drawing is then sequenced into a melody using the minimum spanning tree (from the boost library) of the shape centroids, where distance maps to inter-onset interval.
It also has a mode for projecting the red circles and highlights back on the drawing surface which worked well.
This is only the second thing I’ve made with openframeworks, and while I don’t really get on with the codeblocks editor recommended for linux, I’m impressed with how accessible it makes opencv and all that.
Update: sourcecode
This is quite charming.
But do they function as symbols? or geometric drawings?
I love the idea of drawing to make sound.
Seeing your experiment has got me thinking about a load of possibilities. Nice work.
Hey Graham, symbols are geometric drawings aren’t they? :) But it’s not doing any grapheme detection or anything like that, the computer finds meaning in the shapes.
Yep Jamie I think this approach has a lot of promise, which means it’s probably all been done before, probably in the 50s, documented on a lot of fanfold paper in the attics of forgotten geniuses. Got anything in particular in mind?
Ha, lots of great graphical score stuff in the 1950s, like Earl Brown’s classic “December 1952″:
http://www.chelseaspace.org/archive/december1952-pr.html
I love the casual way you manipulate the drawings Alex, the way a gesture’s meaning can change and you can connect previously disconnected shapes, distort the drawing, etc. Having the red balls bouncing around really opens up possibilities as well. You’ve probably seen this oldy but goody:
http://balldroppings.com
[...] also be good to see what Sarah Angliss has been doing lately with robotics, Alex’s acid sketching and the other good stuff being [...]
rad, dude. a nice concept. what happens with really dense drawings?
Thanks Evan.. In theory you should get intricate sequences with lots of warbling 303 glides, although as soon as two blobs a connected with a line they are treated as a single blob. In practice there are bugs and the more blobs there are, the more get missed out of the sequence for some reason…
Must return to this sometime soon
[...] TurTan inspired a system by Alex McLean and introduced here, with the working title of Acid Sketching. In Acid Sketching, a sound is specified simply by drawing a shape, where morphological measurements are mapped to parameters of an acid bassline synthesiser. The area of a shape is mapped to pitch, its regularity (perimeter length vs area) mapped to envelope modulation, and relative angle of central axis mapped to resonance. Several such shapes are drawn in an arrangement, where a minimum spanning tree of their centroids is taken as a polyphonic sequence, where distance equals relative time. Feedback may be projected back on to the drawing surface, so shapes flash red as they are triggered. A static figure would not make this clearer, however illustrative video is available online at http://yaxu.org/acid-sketching/. [...]