BIOTURKTHE SMART PROGRAMMABLE WETLAB

Klavins/LaMarca – ISTC-PC retreat 4/13

Klavins Lab

Biochemical Circuit Design / Genetic Engineering / Synthetic Biology

What we want to do: Design and implement new genetic programs.

What we spend most of our time doing: Transferring, heating, cooling, filtering, and spinning colorless liquids. Refining and optimizing biochemical protocols.

DO NOT DISTRIBUTE

Where is the Knowledge?• Postdoc’s brains

• Tribal Knowledge

• Superstitions

• Best Known Methods

• Electronic notebooks continually eschewed

Starbucks

Barista

NIH Postdoc Starbucks

Manager

CIF Fellows

Postdoc

Google

Programmer

Assistant

Professor

Position

20

40

60

80

100

Salary in $1,000s

DO NOT DISTRIBUTE

Irreproducible ResultsA study in 2012 by researchers at Amgen showed that only 6 of 50 supposedly seminal results in published cancer research were actually reproducible.

Is this what the $30B NIH budget pays for?

From “Selective cell death mediated by small conditional RNAs”, PNAS, 2010. Now being retracted by the authors who could not reproduce their own results (after trying for two years).

DO NOT DISTRIBUTE

Industry Has a Problem TooDO NOT DISTRIBUTE

Formalize How-to KnowledgeDO NOT DISTRIBUTE

BIOTURK ALPHA

SEMI AUTOMATED

HUMAN IN THE LOOP

WETLAB

DO

NO

T C

RO

SSS

A

BST

RA

CTI

ON

BA

RR

IER

DO

NO

T C

RO

SS

AB

STR

AC

TIO

N B

AR

RIE

R D

O N

OT

CR

OSSDO NOT DISTRIBUTE

BIOTURK ALPHA

DO

NO

T C

RO

SSS

A

BST

RA

CTI

ON

BA

RR

IER

DO

NO

T C

RO

SS

AB

STR

AC

TIO

N B

AR

RIE

R D

O N

OT

CR

OSSDO NOT DISTRIBUTE

Outcomes

• New bar for reproducibility

– “Fractal” notebooks

remember every detail!

– Diffs on protocols

• Efficiency

• Teaching/training

• Preservation / codification of knowledge

DO NOT DISTRIBUTE

-10-

New Capstone for 2013: The Smart Wet Lab10

Similar in many ways to ‘smart kitchen’:

Fixed space, smaller than a room

Tasks performed primarily at a surface

Users execute complex multi-step tasks

Large number of objects, materials and user actions

Can be viewed as a single user or multi-user problem

Ripe for machine assistance

-11-

Smart Lab v1: Method Capture, Index and Replay

11

Our version 1 smart wet lab will capture: Depth camera video of the lab bench from 2+ angles

Possible audio annotation from the experimenter

Presence and usage people, tools, containers and other materials

Data from instrumented lab tools (e.g.: Pipette that transmits its usage)

We will develop a replay tool to review captured data View the audio/video data with synchronized metadata

Metadata collected from tags, sensors and camera-based recognition

Allows rich indexing and annotation: “Queue up Lisa’s experiment from yesterday” “Show me when the promethean bromide was

being used”

-12-

Smart Lab v2: Offline Experimental Analysis

12

To object models needed for v1, add

The order of steps to be taken and any important timing information

A database of the basic lab activities (opening, closing, pouring, shaking, interacting with all the tool and all the machines, etc.)

A database of “proper practices” DOs and DON’Ts for web labs.

e.g.: Do not invert the pipette or let the tip touch the bench

Using these inputs, analyze captured data and extend metadata

Look for errors in experiment execution (verify materials, actions and ordering)

Identify instances in which lab ‘best practice’ has not been followed

Incorporate analysis into the offline experiment viewer:

e.g: “Show me misuses of the pipette on Tuesday.”

-13-

Smart Lab v3: Real-time tutoring and Diagnosis

13

Make sense and perception real-time and interactive:

The user may announce the intended experiment

The system will follow along, in real time offering feedback

For senior experimenters: notifications of errors

For junior scientists: may make more of a tutorial form

Feedback could use both audio and video cues

-14-

Smart Wet Lab v1

ISTC for Pervasive Computing

14

Connected lab equipment (scales,

pipettes, autoclaves, etc)

Lab Bench

Structured lightDepth camera

Microphones

Linux PC

Windows PC

Rfid Reader Bioturk serverWeb server + Bioturk scripts,

protocols and usage data

SmartWetLab serverObject and activity models

+ control server +

user trace data

Touchscreen TabletShows Bioturk protocols

OpenNI2

<RFID SDK?>

bluetooth

UHF RFIDantennas

Perception SDK

USB or analogHttp (control)NFS (data)

http (control)

http

Actions: - Edit bioturk protocol - View bioturk results - Visualize lab activity data - Playback experiments

http

NFS

http or serial likely

(Alternative is a single joint server)

-15-

ISTC Ingredients15

RGBD recognition of lab equipment and actions

Sensors/tags that can identify and monitor usage of lab equipment

Accurate object localization/tracking via vision or sensors

Multimodal recognition algorithms (vision + sensor tag data)

Algs to match objects and action recognition to an experiment definition

Grounding speech in the larger experiment to allow capture of parametric variations

Most of what we’re developing in the ISTC can be used: RGBD+egocentriccamera algs, WISP, IMS, GMTK, grounded speech, etc…