We consider the artist-programmer, who creates work through its description as source code. The artist-programmer grandstands computer language, giving unique vantage over human-computer interaction in a creative context. We focus on the human in this relationship, noting that humans use an amalgam of language and gesture to express themselves. Accordingly we expose the deep relationship between computer languages and continuous expression, examining how these realms may support one another, and how the artist-programmer may fully engage with both.

Our argument takes us up through layers of representation, starting with symbols, then words, language and notation, to consider the role that these representations may play in human creativity. We form a cross-disciplinary perspective from psychology, computer science, linguistics, human-computer interaction, computational creativity, music technology and the arts.

We develop and demonstrate the potential of this view to inform arts practice, through the practical introduction of software prototypes, artworks, programming languages and improvised performances. In particular, we introduce works which demonstrate the role of perception in symbolic semantics, embed the representation of time in programming language, include visuospatial arrangement in syntax, and embed the activity of programming in the improvisation and experience of art.

Feedback is very welcome!

BibTeX record:

@phdthesis{McLean2011,
    title = {{Artist-Programmers} and Programming Languages for the Arts},
    author = {McLean, Alex},
    month = {October},
    year = {2011},
    school = {Department of Computing, Goldsmiths, University of London}
}

RIS record:

TY  - THES
ID  - McLean2011
TI  - Artist-Programmers and Programming Languages for the Arts
PB  - Department of Computing, Goldsmiths, University of London
AU  - McLean, Alex
PY  - 2011/10/01

I started in the usual way with an outline slide, going through bullet points one by one outlining the structure of my talk.  Importantly, I tried to talk continuously while the slide was up.

On the next slide was a picture of a boy throwing a stone into the sea, I talked about it for a while, making the point that it was easy to perceive the image while listening to my voice.  The audience hopefully found they could attend simultaneously to the visual scene and my linguistic speech.

I then skipped back to the previous slide and pointed out that the outline slide actually had little to do with what I had been saying.  Here’s the contents of that first slide:

• A live coding talk towards the end of the conference
• Some strange programming languages were shown
• He made a point about cognition that I didn’t quite get
• The demo didn’t work out too well
• I was a bit tired but he seemed to be trying to say something about syntax

This got some laughs.  There were quite a lot of people in the room, and the slide had been up for a while, but as far as I could gather no-one had managed to read any of it.  My contention was that they couldn’t read it while listening to my voice, it’s too difficult to attend to two streams of language at once.  I didn’t really know what would happen, but from talking to audience members afterwards it seems at least some people got a sense that something was wrong, but couldn’t work out what it was until I told them.

This was a nice practical demonstration of Dual Coding theory, and lead into my argument for greater integration between visual and linguistic elements of computer languages.  However there’s probably a point in there about the design of presentation slides.  If you want people to listen to what you’re saying, put short prompts on your slides, but not real sentences, because the audience won’t be able read them while listening to your voice.

Secondly I’m working with Access Space in Sheffield to make a headphone festival on 16th July 2011. Preliminary info is over at http://lurk.org/placard/, if you’re interested in taking part do let us know.

Abstract:
In the context of the live coding of music and computational creativity, literature examining perceptual relationships between text, speech and instrumental sounds are surveyed, including the use of vocable words in music. A system for improvising polymetric rhythms with vocable words is introduced, together with a working prototype for producing rhythmic continuations within the system. This is shown to be a promising direction for both text based music improvisation and research into creative agents.