Dexy for Scientific Computing

Ana Nelson

dexy.it

July 14, 2011

For all source code check out: http://bit.ly/ruN6yx

This document’s source: http://bit.ly/n82NGO

Writing About Code

• Software Documentation

• Computational Workflows

• Dependency Management• File Provenance

• Reproducible Research

• Total Automation

Writing About Code

• Software Documentation

• Computational Workflows

• Dependency Management• File Provenance

• Reproducible Research

• Total Automation

Writing About Code

• Software Documentation

• Computational Workflows

• Dependency Management• File Provenance

• Reproducible Research

• Total Automation

The Big Idea

No Dead Code

• Any code you show comes from a live, runnable file.

• Any images or output you show comes from running live code.

Tool for the job

Dexy

• Open Source (mostly MIT, some AGPL)

• Written in Python

• Command Line, Text Based

• * Agnostic

• My Day Job and my Mission in Life

Demo

• Simulation in Python (GarlicSim)

• Data Analysis in R

Demo

from garlicsim_lib.simpacks import prisoner

import csv

import garlicsim

import json

NUMBER_OF_PLAYERS = 100

NUMBER_OF_STEPS = 500

Demo

csv_filename = " dexy--sim-output.csv "

csv_file = open(csv_filename, " w " )

data_writer = csv.writer(csv_file)

data_writer.writerow([ " step " , " agent " , " points " , " strategy " ])

Demo

state = prisoner.State.create_messy_root(NUMBER_OF_PLAYERS)

i = -1

Demo

def collect_data():

for j, agent in enumerate(state.players):

strategy = agent.__class__.__name__

data_writer.writerow([i, j, agent.points, strategy])

Demo

collect_data()

for i in range(NUMBER_OF_STEPS):

state = garlicsim.simulate(state)

collect_data()

Demo

csv_file.close()

json_filename = " dexy--sim-params.json "

json_file = open(json_filename, " w " )

json.dump({

’ number_of_players ’ : NUMBER_OF_PLAYERS,

’ number_of_steps ’ : NUMBER_OF_STEPS

}, json_file)

json_file.close()

Demo

> library(rjson)

Demo

> data = read.csv( " 095ac033-eda6-4a5d-ac65-f1ddddfee1cb.csv" , sep= " ," )

> strategy.counts <- table(list(strategy=data$strategy, step=data$step))

> last.period = subset(data, data$step==max(data$step))

Demo

> pdf(file= " 9b3b9647-917e-4dc7-9ec3-e844005fac75.pdf" , width=10, height=6)

> barplot(

+ strategy.counts,

+ legend.text = TRUE,

+ border=NA,

+ args.legend = pairlist(bty= ’ n’ ),

+ col=terrain.colors(3)

+ )

> dev.off()

null device

1

Demo

−1 20 41 62 83 106 132 158 184 210 236 262 288 314 340 366 392 418 444 470 496

TitForTatDevilAngel

020

4060

8010

0

Demo

> pdf(file= " cce2816d-01ac-432e-ab94-593e694c2071.pdf" , width=10, height=6)

> hist(last.period$points, main= " " ,

+ ylab= " Number of Agents" ,

+ xlab= " Final Score" ,

+ col= " purple"

+ )

> dev.off()

null device

1

Demo

Final Score

Num

ber

of A

gent

s

−150 −100 −50 0 50 100 150 200

05

1015

2025

30

Demo

> sim.results <- list( max_points_in_last_period = max(last.period$points))

> sim.results

$max_points_in_last_period

[1] 155

>

> json.filename = " 1d45a798-0e29-4cc8-afde-5c9927dcfe35.json"

> write(toJSON(sim.results), json.filename)

The maximum number of points in the last period was 155.