My new representation also allows me to treat musical structure as both a discrete pattern and a continuous signal, which I’m very happy about, but haven’t explored the depths of yet..

Anyway with a few tweaks and effects it’ll be ready for the algorave in London this weekend.

data Pattern a = Pattern {at :: Int -> [a], period :: Int}

data Pattern a = Atom {event :: a}
                 | Arc {pattern :: Pattern a,
                        onset :: Double,
                        duration :: Maybe Double
                       }
                 | Cycle {patterns :: [Pattern a]}
                 | Signal {at :: Double -> Pattern a}

I’ve got rid of periods, now patterns always have a relative period of 1. However they can be scaled down by being enclosed in an Arc pattern, and given a floating point duration and time phase offset (which in music parlance is called an onset), which should be less than 1.  A Cycle pattern consists of a number of Arcs, which may overlap in time.

The end result is a nice representation of cyclic patterns within patterns, with floating point time so that events don’t have to occur within the fixed time grids of the acid techno I’ve been making.

It is also still possible to represent a pattern as a function, which is what a Signal is in the above.

The Functor definition is straightforward:

instance Functor Pattern where
    fmap f p@(Atom {event = a}) = p {event = f a}
    fmap f p@(Arc {pattern = p'}) = p {pattern = fmap f p'}
    fmap f p@(Cycle {patterns = ps}) = p {patterns = fmap (fmap f) ps}
    fmap f p@(Signal _) = p {at = (fmap f) . (at p)}

The Applicative functor definition isn’t so bad either:

instance Applicative Pattern where
    pure = Atom

    Atom f <*> xs = f <$> xs
    fs <*> (Atom x) = fmap (\f -> f x) fs

    (Cycle fs) <*> xs = Cycle $ map (<*> xs) fs
    fs <*> (Cycle xs) = Cycle $ map (fs <*>) xs

    fs@(Arc {onset = o}) <*> s@(Signal {}) = fs <*> (at s o)
    fs@(Arc {}) <*> xs@(Arc {}) | isIn fs xs = fs {pattern = (pattern fs) <*> (pattern xs)}
                                | otherwise = Cycle []

    fs@(Signal {}) <*> xs = Signal $ (<*> xs) . (at fs)
    fs <*> xs@(Signal {}) = Signal $ (fs <*>) . (at xs)

Here’s how to turn a list into a pattern:

class Patternable p where
    toPattern :: p a -> Pattern a

instance Patternable [] where
    toPattern xs = Cycle ps
      where
        ps = map (\x -> Arc {pattern = Atom $ xs !! x,
                             onset = (fromIntegral x) / (fromIntegral $ length xs),
                             duration = Nothing
                            }
                 ) [0 .. (length xs) - 1]

And here’s how to make a Signal pattern of a sinewave:

-- sinewave from -1 to 1
sinewave :: Pattern Double
sinewave = Signal {at = f}

    where f x = Arc {pattern = Atom $ (sin . (pi * 2 *)) x,
                     onset = mod' x 1,
                     duration = Nothing
                    }

-- sinewave from 0 to 1
sinewave1 :: Pattern Double
sinewave1 = fmap ((/ 2) . (+ 1)) sinewave

Finally, here’s how to multiply a Cycle of discrete events by a Signal, thanks to our Applicative definition:

(*) <$> toPattern [1 .. 16] <*> sinewave

Well this may all be rather trivial, but somehow I find this really exciting, that continuous functions can be multipled by (potentially) complex discrete, hierarchical patterns with such tersity.  Furthermore that time can be manipulated outside of fixed grids.  I’ve been putting off making sounds from this, to try not to prejudice possibilities, but am really looking forward to experimenting with it live in performance.

It’s probably not of much use to anyone else at the moment, but the code is over here.
 

We consider the artist-programmer, who creates work through its description as source code. The artist-programmer grandstands computer language, giving unique vantage over human-computer interaction in a creative context. We focus on the human in this relationship, noting that humans use an amalgam of language and gesture to express themselves. Accordingly we expose the deep relationship between computer languages and continuous expression, examining how these realms may support one another, and how the artist-programmer may fully engage with both.

Our argument takes us up through layers of representation, starting with symbols, then words, language and notation, to consider the role that these representations may play in human creativity. We form a cross-disciplinary perspective from psychology, computer science, linguistics, human-computer interaction, computational creativity, music technology and the arts.

We develop and demonstrate the potential of this view to inform arts practice, through the practical introduction of software prototypes, artworks, programming languages and improvised performances. In particular, we introduce works which demonstrate the role of perception in symbolic semantics, embed the representation of time in programming language, include visuospatial arrangement in syntax, and embed the activity of programming in the improvisation and experience of art.

Feedback is very welcome!

BibTeX record:

@phdthesis{McLean2011,
    title = {{Artist-Programmers} and Programming Languages for the Arts},
    author = {McLean, Alex},
    month = {October},
    year = {2011},
    school = {Department of Computing, Goldsmiths, University of London}
}

RIS record:

TY  - THES
ID  - McLean2011
TI  - Artist-Programmers and Programming Languages for the Arts
PB  - Department of Computing, Goldsmiths, University of London
AU  - McLean, Alex
PY  - 2011/10/01

Experimenting with webcam overlay. Video recorded using gstreamer, source for screencaster here (screensave.c).

UPDATE, here’s another from a different angle to appease douglas.

Once the participants had got the hang of things on headphones, we all switched to speakers and the seven of us played acid techno for an hour or so together, in perfect time sync thanks to netclock. Here’s a mobile phone snippet:

The sound quality doesn’t capture it there, but for me things got really interesting musically, and it was fun walking around the room panning between the seven players…

I won’t be touching this until after the workshop on Saturday.

I’ve also made the source for the visual interface available here under the GPLv3 free license. To get it actually working as above you’d also need to install my tidal library, Jamie Forth’s network sync, my sampler, the nekobee synth, and somehow get it all working together. In short, it’s a bit tricky, I’ll be working on packaging soonish though.

I have had a bit of time to make Text, a visual language I mentioned earlier, a bit more stable, here’s a test run:

A bit of a struggle, partly due to the small screen area I gave myself for the grab, but also due to some UI design issues I need to sort out before my workshop at Access Space in Sheffield next week, on the 5th February. Access Space is a really nice free media lab, but will turn nasty unless I free the workshop software, so expect a release soon.

In case someone is interested, here’s the linux commandline I use to record a screencast with audio from jackd:


gst-launch-0.10 avimux name=mux \
! filesink location=cast.avi \
ximagesrc name=videosource use-damage=false endx=640 endy=480 \
! video/x-raw-rgb,framerate=10/1 \
! videorate \
! ffmpegcolorspace \
! videoscale method=1 \
! video/x-raw-yuv,width=640,height=480,framerate=10/1 \
! queue \
! mux. \
jackaudiosrc connect=0 name=audiosource \
! audio/x-raw-float,rate=44100,channels=2,depth=16 \
! audioconvert \
! queue \
! mux.


It’s basically Haskell but with syntax based on proximity in 2D space, rather than adjacency.  Type compatible things connect automatically, made possible though Haskell’s strong types and currying.  I implemented the interface in C, using clutter, and ended up implementing a lot of Haskell’s type system.  Whenever something changes it compiles the graph into Haskell code, which gets piped to ghci.  The different colours are the different types.  Stripes are curried function parameters.  Lots more to do, but I think this could be a really useful system for live performance.

hackpact23

Bit of an error with the screencast, see if you can spot the problem. Pretty happy with the sound though. (will take a while to appear due to vimeo’s encoding queue)

I’m musically humbled by my son who has taught himself how to play the guitar and sing the blues. He’s two today (24th), happy birthday Harvey.

hackpactoops

Well I made a blueberry birthday cake for young Harvey on the 24th, which is a hacklet at a push, photo when I find the transfer cable. No creations at all on the 25th or 26th, I was away for the weekend and thankfully didn’t get a moment with my laptop. I did record a session last night thought that I’ll upload at some point…

The hackpact has been really good for making me get bored with my software and develop it further. I need some longer term development time now though to play out some of my frustrations I’ve been feeling over the last month. In particular using a command line interface is feeling like a big limitation, I need to express relationships over more than one line. Maybe I can adapt the yi haskell editor for my needs.

Next month I think I’ll switch to making a fixed recording every week. I’ve never really made fixed recordings so should be interesting.

Now I’ve fallen off the hackpact wagon I’m not sure if I’ll be able to totally get back on, particularly as I need to finish my PhD upgrade report by the end of this month. We’ll see..

hackpact15

Another screencast with some good moments and also a couple of bugs found… I realised I forgot to switch off cpu scaling, so there might be more jumps in the recording.

hackpact16

Preparing for the haskell users group talk, so I tried to make some ultra quick demos of features of my pattern language. Failed really, I kept getting distracted at making the patterns sound better rather than demonstrate how they work, so I will likely just do a live demo instead.

The results are on youtube — I’m doing the presentation in google docs, and youtube is the only way of getting videos in there. In fact this is the only way I found of making a presentation with videos in under linux. Video support in openoffice sucks and didn’t work anyway, the html based s5 gets all slow and glitchy with videos in, the video library used in the python based bruce wouldn’t play any video I found, etc, etc…

hackpact17

Got through the haskell users group talk in one piece, happy with it actually. I managed to do some short sub-minute demos for it today which I think made it easier for people to follow, and I think count as today’s hackpact. Here’s the final presentation in full:

hackpact18

Uploaded my current Haskell stuff. Not very useful at all without instructions for how to install and use it which I’ll get to soon in a proper release, but some might be interested to read through the Pattern module at least.

Big shout out to Kassen, while he is behind on documenting his hackpact stuff he assures that he’s still doing daily hacks in the background. Hope to hear more from his side of our acidpact soon.

I’m pretty tired, going to try and make a new screencast tonight nonetheless.. Ah, here you are. Got rid of need for explicitly calling parser by using the overloaded string literals extension, thanks for the tip Ganesh.

hackpact19

Today’s screencast is here, although it’s not all that. No update tomorrow, we’re off to do some live coding at Plymouth Planetarium, documentation will hopefully surface eventually.

hackpact20

Well the planetarium experiment went well, hopefully will turn into a tour. Some documentation to appear sometime soon.

hackpact21

Too tired to make music, so worked on a poster design. Ended up too gloomy, to rework.

hackpact22
[[blue, blue [lightblue lightskyblue lightblue, lightblue lightskyblue lightblue ~] blue, blue], blue lightblue ~, blue]