Alex McLean

I’m very happy to be putting together the evening programme for knowledge futures on October 16th at Goldsmiths. Excitingly, my superheroes Leafcutter John, Finn Peters and Yee-King and Spacedog UK (Sarah Angliss + co) have agreed to play. I’m going to play too as part of slub.

Details are coming together over here. Buy your ticket early!

Computational Creativity seminar

I attended the start of the Dagstuhl seminar on computational creativity last week, although sadly had to leave after the first day due to family illness (which very sadly is still ongoing).  Anyway here is the position statement I prepared for the seminar, which attempts to answer some fundamental questions around creativity.  I regret I had to leave before getting feedback on these thoughts from the group at Dagstuhl, so any comments here are very much appreciated!

What does creativity produce?

A concept is `a mental representation of a class of things’ (Murphy 2002, p.5), and concepts are the primary output of a creative process. In other words, creativity is the process in which a creative agent recognises a new kind of thing, or modifies their understanding of a kind of thing, changing their view of the world in some valuable way. The visible output of a creative process may be a single thing, but it is the novelty and value of the concept behind the thing that shows creativity. The creative outcome is in the mind, not in a physical object.

Where are concepts represented?

A conventional view is that conceptual representation, and indeed cognition in general is functionally separated from perception. Theories of embodied cognition however take the view that concepts are inherently perceptual; that concepts arise from recurrent states in sensory-motor systems, which in turn form the building blocks of higher level abstract thought. If we creatively generate new concepts, then we are literally altering our perception of the world and of ourselves.

How are concepts represented?

How a concept is represented in human cognition is an open question, for example one view is that a concept is represented by a single best example or prototype, and another being that a concept is represented by a large number of memories, or exemplars. Theories of embodied cognition such as perceptual symbol systems proposed by Barsalou (2009) and conceptual spaces proposed by Gärdenfors (2000) take the prototype view. For example according to Barsalou (2009) concepts are based on incomplete, distorted and often vague summaries of prior perceptual states. Barsalou attributes the unpopularity of embodied cognition to the lack of understanding of this fragmentary and partly subconscious nature of perception. Gärdenfors (2000) also takes a prototype view, but in addition proposes that concepts are inherently geometric, where conceptual properties are convex regions within the quality dimensions of conceptual domains. He goes on to base a system of cognitive semantics on this geometric view, grounded in spatial metaphor as mappings between geometric domains.

What is creative search?

Creativity is described by Boden (1990) and formalised by Wiggins (2006b,a) as a search in a space. Three sets of rules are employed in this search; rules defining traversal of the space, evaluation of the concepts found in the space, and the space itself. However, a creative search is more than a reactive process of traversal and evaluation. Creativity also requires introspection, self-modification and for boundaries to be broken. In other words, the rule sets described above need to be examined and challenged by the agent following them. In the terms of Gärdenfors (2000), the search space is a concept, and the search is for concept instances.[1] For example in a creative search for music within a genre, the genre would be the concept and a piece of music conforming to a genre would be a concept instance.

Artists often speak of self-imposed constraints as providing creatively fertile ground. In terms of a creative search such constraints form the boundary of a space. It is possible for a search to traverse beyond that boundary, thus finding invalid concepts. If invalid yet (according to evaluation rules) valued concepts are found, then the space should be enlarged to include the concept. An invalid concept which is not valued indicates that our traversal strategy is flawed and should be modified to avoid such concepts in the future. A single traversal operation may result in both valid and invalid concepts being found, indicating both the traversal rules and the definition of the space should be modified. Returning to our musical example, we can think of a creative piece of music that has altered the boundaries of a music genre, or defined a whole new genre. Indeed music which does not break boundaries to any degree could be considered uncreative.

It is important to recognise that changes in conceptual structures first happen in an individual, which in the case of music would be the composer or improviser. Another individual’s conceptual structures may be modified to accord with a composer’s new concept by listening to the new concept instance, although success is only likely if the individual already shares the music cultural norms of the composer.

Embodied creative search

Wiggins (2006b,a) formalises creative search in order to provide a comparative framework, and so is agnostic to views of representation. However by taking the view of embodied cognition summarised here, we may define embodied creative search, where sensory-motor faculties are used to navigate a geometric space, in direct metaphor to a search through a physical space. In this view, creative computation requires concepts to be represented in a manner at least sympathetic with the way humans perceive, act and introspect. More detail on this position in the context of musical creativity is given by Forth et al. (2008). Further, an approach to symbolic description of musical sounds informed by human perception termed vocable synthesis is provided by McLean and Wiggins (2009). Both papers are available for download on the Dagstuhl seminar website alongside this position statement.

Footnotes

[1] The terms used by Gärdenfors (2000) diverge from those used by Wiggins (2006b,a). Wiggins uses the term conceptual space in the place of Gärdenfors’ concept, and concept in the place of concept instance. The meaning is however the same, particularly when the recursive heirarchy of Wiggins’ theory is taken into account.

Bibliography

Barsalou, L. W. (2009).
Simulation, situated conceptualization, and prediction.
Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1521):1281-1289.

Boden, M. (1990).
The Creative Mind.
Abacus.

Forth, J., McLean, A., and Wiggins, G. (2008).
Musical creativity on the conceptual level.
In IJWCC 2008.

Gärdenfors, P. (2000).
Conceptual Spaces: The Geometry of Thought.
The MIT Press.

McLean, A. and Wiggins, G. (2009).
Words, movement and timbre.
In Proceedings of NIME.

Murphy, G. L. (2002).
The Big Book of Concepts (Bradford Books).
The MIT Press.

Wiggins, G. A. (2006a).
A preliminary framework for description, analysis and comparison of creative systems.
Journal of Knowledge Based Systems.

Wiggins, G. A. (2006b).
Searching for computational creativity.
New Generation Computing, 24(3):209-222.

I’ve had a paper accepted for the 25th CHArt conference on the subject of live coding, here’s the synopsis.

Programmers do their work by writing and modifying software in text form.  That is, a piece of software is a structure made from words. These structures are generally too big to comprehend in their entirety, so programmers instead focus on small detail and overall plans; looking up for parts to combine and simplify and looking down for places to build.  But this is not architecture, these structures are more like machines than static buildings.  A programmer’s work is set in motion by a program interpreter, with information flowing in and around processing units before being directed outward in response.

Usually a programmer will build their work off-line, stepping back to start it up, watch it work and decide upon the next edit.  Live coding programmers however work on their software while it runs, much like modifying a running machine.  Because the software is built from words, this is done by editing it as text, adding new routines or changing the nature of an existing one.  Such a change affects the output immediately, allowing fast creative feedback.

Where a written novel exists to describe human activity, written software exists to simulate it.  Therefore the live coder can take the role of an artist, constructing simulators in order to generate patterns of movement, either as music, video animation or both.  This can be done in front of a live audience, so that the process of building software becomes the process of improvising music or video in performance art.

In this way programmers are finally taking to the stage, introspecting and encoding their musical thoughts before an audience.  A tradition of live coding has quickly formed where computer screens are projected, making the programmer’s reactions to their work visible.  Questions of authorship disappear; the performance is live, the programmer improvising through the medium of written language.

Live coders allow us to look upon the interplay between language and performance art with renewed interest.  In this paper the current activity of live coders will be reviewed, to try to understand where this relatively new practice is taking us.

I’ve been trying to make music with Haskell for a little while now.  One thing about Haskell is that its strict typing forces you to think a lot about how you represent things.

I wanted to support two things in my musical pattern representation, firstly polyphony (more than one sound being triggered at the same time) and secondly recursive structure. In the west we often think of rhythm in quite `flat’ terms – a 2d pattern like this:

Of course that’s a pretty useful techno interface, allows for polyphony, and could be represented in haskell using a straightforward list of lists.  However this 2D structure is only really suitable for making music with fixed timed signatures, most notably 4/4.

Time: you're doing it completely wrong

Enter Bernard Bel (pictured right).  His BP2 software represents music in a manner inspired by Indian classical music.  His 2001 paper Rationalising musical time has all the details and is a great read, formalising a quite beautiful way of thinking about time. As part of that it defines an elegant pattern syntax that allows patterns to be embedded inside patterns of differing time signatures.

Here’s my first attempt at representing analogous structures in Haskell.  Straightforwardly, a musical event is either a sound (represented by some string) or some silence:

data Event = Sound String
           | Silence

A musical structure is either an event, a cycle of structures, or a polymetry of structures:

data Structure = Atom Event
               | Cycle [Structure]
               | Polymetry [Structure]

A cycle allows us to sequence structures one after another, and a polymetry allows us to represent several structures occuring over the same period of time.  If the structures within a Polymetry have different durations, then they are ’stretched’ until they are the same length (equal to the lowest common multiple of their lengths). I’ve written something in parsec that parses Bel’s notation into the Structure datatype.

The problem with the above representation is that it gets unwieldy with complex rhythmic structures.  The whole point of generative music is being able to make very long patterns out of much shorter rulesets.  However with this representation, to find the 10,000th event you’d have to calculate the 9,999 events preceeding it.  This becomes a particular problem when modifying these structures as part of a live coded musical improvisation.

So I instead came up with this representation:

data Pattern a = Pattern {at :: Int -> [a], period :: Int}
type SoundPattern = Pattern String

Instead of explicitly representing a recursive structure, I’m now hiding everything inside a function mapping from integers to a list.  You pass the function the current position in the pattern (e.g. the current ‘beat’, and get back the list of events at that position.  For certain operations you still need to know the period (length) of the repeating pattern, so that’s in the datatype constructor too.

What kind of operations can I do to these patterns?  Here’s one that simply shifts events backwards in time:

(<~) :: Pattern a -> Int -> Pattern a
(<~) p steps = Pattern (\n -> at p (n + steps)) (period p)

And in the other direction:

(~>) :: Pattern a -> Int -> Pattern a
(~>) p n = (<~) p (-n)

Here’s a higher order one that applies an operation conditional on the time value (while preserving the period of the original):

when :: (Int -> Bool) -> (Pattern a -> Pattern a) -> Pattern a -> Pattern a
when pred f p = Pattern l (period p)
where l n = at (if (pred n) then (f p) else p)  n

Making this possible …

when (\n -> n `mod` 8 > 3) (<~ 2) p

… which shifts the pattern back in time two steps, but only affects every other group of 4 steps.

This is probably fairly humdrum stuff, but quite fascinating to me as someone relatively new to both haskell and functional programming. Being able to pile functions on top of functions in this way lets me compose complex patterns out of simple parts, and I only ever have to deal with the simple parts in memory, not the complex patterns. Great! Thanks to the parameterisation of the Pattern datatype, I can also use patterns for different aspects of a rhythm, my recent screencasts [1,2] showing concurrent patterning of samples, vowel formant filtering, sample playback speed and a comb filter.

I’ve also been looking at making the Pattern a Monad (my first ever monad instance, woo!):

instance Monad Pattern where
    return x = loop $ [x]
    (Pattern l len) >>= f = Pattern l' (lcm len len')
        where l' n = at (combine $ map f (l n)) n
              (Pattern _ len') = f undefined

Where ‘combine’ is another function not shown that which merges patterns together. The problem I’m having with this is that functions of type a -> Pattern a for use with >>= can’t do that much, not being able to consider any pattern context, and not being able to return a different period depending on the input (otherwise that ‘undefined’ value would bite). This is about the most useful thing I can come up with, which ‘masks’ events every other time measure:

mask :: (a -> Pattern a)
 mask x = Pattern (\n -> if n `mod` 2 == 0
                         then [x]
                         else []
                   ) 2

Maybe I should be thinking of what I could do with more general functions of type (a -> Pattern b) instead. I guess I’m just getting a better handle on what monads are really for…

Monads aside I guess my Pattern representation would really become unstuck with patterns that have a lot of interdependencies. For example if every event depended on the event preceding it then you’d have to recalculate the whole pattern all the time. Hmm.

Well that was quite an extended haskell wibble. If by some strange chance you’ve read this far I’d love to hear your thoughts and especially advice!

Another screencast:

Trying a lower bpm than normal. Also I’m starting to add more higher order rhythm functions to increase live coding potential… “(~> n)” shifts the pattern one beat to the right, (<~ n) to the left, and "when" allows such ops to be applied selectively depending on function over the beat number. I feel a monad coming on.

Someone on twitter wondered whether the last one was faked, I’ll take that as a compliment! As Kassen pointed out, that one was mainly just a sequencer. A polyrhythmic sequencer, with sequences for parameter values as well as samples but a sequencer nonetheless. My aim is to try to stay as simple as possible but I’m currently looking for ways of transforming sequences through live coding. This is a step towards that.

Thanks for all the source code requests, I will release the code eventually and announce it here, but might take a while… Sorry.

I might do an opendork showing this stuff at dorkbotlondon tonight. Might not be time though as we’ve got a lot of classy presentations to get though.

If you are in the UK and have a vote, please use it. The BNP are racists, who want all “non-indigenous” (i.e., non-white) people to leave the country. If there is a low voter turn out, they have a good chance of getting their first MEP, which means much more money for their cause and stronger links with other fascist parties in europe. That would be a disaster.

Before the current BNP leader, Nick Griffin, took charge, the BNP were openly racist. He has been working hard to clear up their image and the language they use, but that is all that has changed. He remains a racist, holocaust-denying homophobe who believes that no black or asian person could ever be described as british. Griffin considers the BNP to be “a strong, disciplined organisation with the ability to back up its slogan ‘Defend Rights for Whites’ with well-directed boots and fists. When the crunch comes power is the product of force and will, not of rational debate.”

See wikipedia articles on the BNP and Nick Griffin, and the hope not hate website including this leaflet for more info.

But most of all — please vote, for any party but the BNP! If you’re unsure who to vote for, euprofiler.eu does a good job of matching your opinions to party manifestos.