Composing music withclojure.specWojtek Franke@apogsasisClojure/conj 2016(s/def ::music ...)

Welcome to "Composing music with clojure.spec"

Agendafun with clojure.specgenerate some tunesreflect on generative music

We'll have some fun with clojure.spec.We'll model a genre of music using.We'll make music.Finally I'll reflect on generative music and the direction where it's heading.

Not on the agendahow to use clojure.spec in productionbest practices

you can find this in talks elsewhere.

$> whoami

this is a computer's rendition of me - I especially like the chalice neck

$> whoamicomposing and performing music as ENAY since ~2007software testinginterested in chaos

composing and performing electronic music since around 2007used procedural real-time 3d visuals - used to be called demoscene back thenbut also a software tester by trade - I like to break stuff and use it in novel ways.I'm approaching this more from the perspective of the musician on a vision quest

Describing musical dataHypothesis:

The primary paradigmof computer music is

"Music is Data"

You might be wondering how does clojure.spec apply generating music? We'll let me state a hypothesis. ... If music is data we should be able to use whatever tools we know for manipulating data to.

Toolsclojure.specleipzigovertone

I will be using spec.Chris Ford's library leipzig which you may be familiar with if you've seen his great talks. There's a lot of Chris Ford fans here in the audience.and finally overtone for the actual audio (up till 1.9.0-alpha8)this is all pretty standard stuff for Clojure and music

Picking the domain

As you can probably imaging I did not build a universal music making machine.Instead I just focus on one particular type.So what kind of music will I be showing?

I resisted the temptation to do something completely crazy - as it's not Paris and it's not 1924. Instead I decided to pick a genre that I think is fairly simple and "canonical" but also close to my heart.

And it's Jungle. [PLAY SONG]A good way to explain it is that it's a flavour of drum and bass with a twist. Let me give you the context of what we'll be creating.

Modeling musicaudio

symbolic representation

When speaking about computer music we can speak of really 2 kinds of data - the audio whether it's a .wav file or a buffer of number which is ultimately converted from Digital -> Analogue and ultimately air vibration. The other is the symbolic representation of music - sheet music, MIDI files, guitar tabs. This data is a blueprint for musical performance.

Modeling musicaudio

symbolic representation

To an extent we can translate between these domains. We want to synthesise audio from a symbolic representation but also we may want to analyse or fingerprint audio to go the other way round. To create jungle we will rely on both sources of data - jungle is heavily sample-based making it an interesting choice for autonomous composition.

Hardware samplers

The birth of jungle and also drum and bass and hip-hop is linked to the emergence of digital samplers. Their availability, lower cost and ease of use compared to earlier tape techniques meant the democratisation of musical production. Tons of people were sampling vinyl records, whatever they could get their hand on and using samplers to loop, slice, stretch, pitch shift, reverse bits of audio.

What's in a track?

So what's the bare minimum that I think constitutes a classical jungle track?

What's in a track?sampled drumsbassvocal samples

What's in a track?sampled drumsbassvocal samples

Let's start with the drums as that's the core of the jungle sound.

Breakbeatsbreakbeat |brekbit|noun

a sample of a syncopated drum beat, usually repeated to form a rhythm used as a basis for dance music, hip hop, etc.

The breakbeat is literally a break in the song when the pitched instruments stop playing and the drummer plays solo. These most often come from funk, soul or gospel tracks. Syncopated means that the accent is placed on the "weak" parts of the beat.

What's in a track?drums aka breakbeat or break

Here's the most iconic breakbeat of them all.Even if you're not into jungle you've probably heard it in everything from NWA's "Straight Outta Compton" to TV commercials, Futurama theme song.

It's called the 'Amen break'. Heck it even has its own Chardonnay- that's how big of a deal this is. Notice this pattern at the top is a transcription of the drum pattern for the crash cymbal, ride, snare and bass-drum. Let's listen to it again. Maybe if you have good eyesight you can follow along

The Amen Breaka drum solo (breakbeat) from "Amen Brother" a 1969 track by The Winstonsone of the most sampled songs in historyrevered for its timbre and rhythm

Notationmusic scenes develop their own vernacularnotation can allow easier, more intuitive communication

OK, so we know that we want to use sampled breakbeat - how do we represent them symbolically? If you look at the wine bottle you'll have the instruments separated out. But this is not really what we're working with. It's really a single layer, the breakbeat is the instrument. I wanted to create a notation for using breakbeats that feels natural. When thinking about a beat I hear in my head something close to the sounds themselves. Since my first language is Polish

Polish Jungle Notation

It seems only appropriate to call it the Polish Jungle notation

Polish Jungle Notation

At it's simplest we can think of this as a set of single-syllable onomatopaoic sounds which correspond to the dominant percussion sound happening at the time.But this is's enough information to get us going. As in most genres of electronic music, the key is repetition. And since

Polish Jungle Notation

We define simple rules for how many times a specific sound can be repeat and build a collection we can use.

Spec doesn't force you to use it in

Composing breaksThe different renditions of the amen break and other breakbeats are composable. We can mix and match parts of different breaks.

There is a interesting property - that these rules can be applied not just to the amen break but to other breaks as well.

Preparing samplesCharacteristics of the samples:

equal in length in the same tempo volume is normalised

To do this we need to prepare our sample base.

Picking breakbeats

I load up a bunch of them as audio buffers. Define a fuction which plays back a specific slice of the buffer. And pick some different renditions of the amen break to begin with. Here's how they sound like:

Slicing

Here we play a single kick drum sound.

Slicing

We can map the syllables to selected parts of the break.

Wait for it...

Now we can generate a sequence from the spec earlier. We flatten the repetitions, pick 4 bars of 8 sounds. Rhythmise just takes a collection of amen sounds and arranges them in time using absolute values.

Playing some breaks (finally!)

Finally we can play our breaks together. Repeating the entire sequence 2 times, playing at a brisk 172 beats per minute.

Composing breaks

I mentioned that we can mix and matche different breakbeat sounds - here I'm doing so at runtime. This is a pattern I commonly use - to choose.

Effectsexpanding the model with effect triggers (reverse, pitch shift, filters)

In the future we could expand our model to perform even wilder variations. With per-note effect triggers. There are really a lot of variations we can do to replicate the kinds of edits that human musicians do.

What's in a track?drums (breaks)bassvocal samples

Now that we've set up the scene for slicing breakbeats let's talk a bit about the bass.

Bass

Now these sequences are rather rudimentary. But bass in jungle plays a rather simple role.

What's in a track?bass

Typically we'll see a simple sine oscillator used for generating the bass like in this example.

Bass

note |nt|noun

a single tone of definite pitch made by a musical instrument or the human voice

Bass is obviously a pitched instrument, meaning that the height of the sound varies to create a melody. When speaking about melodies at least in the West we thisk of them as sequences of notes.

Bass

Here I'm representing pitch in the usual fashion - as an integer operating withn two octaves and the duration is represented by the familiar rational number notation used in sheet music.

Bass sequence

But playing pitch as random integers is like having a monkey sit at the piano - they may get it right sometimes but we'll likely be annoyed with our primate companion first. Instead we can compose leipzigs scale functions to transform our random integers into random integers on a scale of our choosing.

Bass sequence

It's not great but at least I don't hate it which is good.

What's in a track?drums (break)bassvocal samples

Alright let's add our final spice to the mix - the vocal samples. And specifically, we'll be using ragga samples.

What's in a track?ragga |ra|noun [ mass noun ]

a style of dance music originating in Jamaica and derived from reggae, in which a DJ improvises lyrics over a sampled or electronic backing track.

Now not ragga as that of India.

What's in a track?ragga samples

These sound something like this.

Ragga samples

Nothing fancy here, just loading a set of these samples and specifying their timing.

Ragga samples

Let's hear all of them together.

Building songs

So now that we know how to build a short sequence. What is stopping us from building an entire song?

We can specify an arrangement using the familiar "intro, verse, breakdown, reprise, outro" pattern and even generate the artist, song names.

Specify which instruments should be present

Even generate some artist names

Building discographiestelescoping nature of musical composition

But why stop there if we can make whole discographies. The idivual edits -> macro level of whole libraries of music.If I want to perform a 2-3 hour set of music - if I have a compositional tool to which I can delegate some of the work I can focus on other things.

Building discographies

But why stop there if we can make whole discographies of music.

I feel like I could..

Take on the world! So you see where this is going.

Is there any practical reason why you'd want to do this? Would I even want to listen to all that music? The examples you've heard today are "OK" at best. It's a start.

Sharing specs

There's some other interesting properties of specs that play to our advantage. We want to build a web app where people can listen to tunes and rate them. Providing feedback back to the generating process? We could do that!

Share specs

Generate documentation with live example data? sure we can do that.

Share specs

Generate documentation with live example data? sure we can do that.

What workedthe end result is decentI now have a new jam buddyit's scalable!single data specification I can share between backend and frontend

The end resuslt is decent enough to use these breakbeats. As a musical perfectionist this is a big gain -

What didn't workclojure.spec generators are unaware of the contextmaking custom synthesisers

Generation of elements independently of each other meant that there was really no chance for the generator to build context-aware sequences. I tried to generate parameters to overtone synths but this posed a similar problem - one parameter changes - say the resonance of a filter and this affects the volume, but the volume is also a generated parameter then I can have pretty unpleasant results.

Remember this slide?Hypothesis:

The primary paradigmof computer music is

"Music is Data"

OK, this is the start of the rant part.

Remember this slide?Hypothesis:

The primary paradigmof computer music is

"Music is Data"X

The next generationof computer-generated musicneeds some understandingof human cognition

Perception, acoustics, mood, context

DesignAffordanceAppropriation

There is always a tension between Design and Appropriation but as as William Gibson wrote - The Street finds its own use for things.

Agency in computer-human systemsis hybridised

There are certain shorthands for thinking about such. Oh it's all generated - there is no human aspect to it. It's not autonomous at all -it's all controlled by the human creator. I don't think either of these propositions is true. Rather - agency is hybridised.

What happens when 2 artists with their computers jam music and video together.

A lot of things are hidden or blackboxed in technical systems.

Coal Fired Computers (300,000,000 Computers 318,000 Black Lungs)

Harwood, Yokokoji and Denmars

This project opens up the black box of computing. That we may feel that the technological goods like laptops or phones are "clean" but the cost of tier manufacture and operation is often elsewhere.

What's next?self healing live-performances

self healing live performance - you played out of key? let me generate a conforming note and play it instead. Stuart Hallowey said yesterday that spec is preventing him from doing stupid things. Why not apply that to musical performance? There is a quote attributed to Syd Barrett the late frontman of Pink Floyd - it's not important to know which notes to play, just which notes NOT to play.

What's next?self healing live-performancesonline generative jungle radio

The second thing which want to explore is to expose a live stream of generated music. I'm a big fan of jungletrain.net with is a human-run radio station.

What's next?

The second thing which want to explore is to expose a live stream of generated music. I'm a big fan of jungletrain.net with is a human-run radio station.

What's next?self healing live-performancesonline generative jungle radioexplore the cognitive aspect

I want to explore the cognitive aspect - if we want our code to make us more creative then it needs to know more about the idiosyncracies of how you think.

ResourcesCarin Meier - Genetic programming with spechttps://github.com/gigasquid/genetic-programming-specChris Ford - African Polyphony & Polyrhythmhttps://www.youtube.com/watch?v=EK4qctJOMaUCreativeAI - On the Democratisation & Escalation of Creativityhttps://medium.com/@ArtificialExperience/creativeai-9d4b2346faf3#.k8ubhhauk

Using spec themselves as data.Modeling a type of music but he also acknowledges the cultural bias in removing the musical data from its performance context.

ResourcesGITHUB PLACEHOLDER HERE

Find me@apogsasis

apogsasis.org

Spotify, iTunes: ENAY

Fin

Otherwise we might find ourselves living in a world with an overabundance of mediocre music. And on that optimistic note, let me play you out with a little tune: