Traffic Lights, Targets, & Tolstoy:Introduction to Tolstoy Targets

Dennis Sweitzer

www.dennis-sweitzer.com

denswei@gmail.com

(Enterprise dashboard with 4 dredging projects)

Outline

• Principles:

– Targets, Tolstoy & Traffic Lights

• Conventions

– The Waterline

– Symbols

• Practical Examples

– Project Results, Statuses

– Massively Parallel Outcomes

– Reporting

PRINCIPLES

Testing Drug Izitwerkin: Traffic Light

The predefined objectives

of the study are:# 1 ……, #2…. #3….. #4….. #5…..

For X$ within Y months

Cut to the chase: we know

what are the objectives.

Did we meet them?

We need to make plans

for lunch…..

Testing Drug Izitwerkin: Green Light

Great, we can quit

early

& celebrate

over lunch

Met all predefined objectives

for efficacy, safety, etc.

And also budget, timelines,….

Testing Drug Izitwerkin: Red Light

Clearly failed on predefined

objectives for efficacy, safety,

budget, timeline, etc.

Which failed? ….If it’s critical, we cancel the rest.

We’ll commiserate over lunch….

Testing Drug Izitwerkin: Yellow Light

Mostly met objectives, but….

Neither clear success

nor clear failure

But what? …. Should we have had more patients?

A little over budget? A little late? Some bad luck? Unusual circumstances?

What can we salvage?What do we have to redo?

Get lunch delivered, it’s going to be a long day…..

Targets: Testing Drug IzitwerkinMultiple Objectives, Multiple Shots:

Timelines Budget

Adverse Events

Abnormal Lab Changes

Pain Relief

Drug Stability

Speed of Relief

Ready…..

Targets: Aiming for Multiple Objectives

Timelines Budget

Adverse Events

Abnormal Lab Changes

Pain Relief

Drug Stability

Speed of Relief

FIRE!!!Aim…..

Targets: Traffic light color code

Timelines Budget

Adverse Events

Abnormal Lab Changes

Pain Relief

Drug Stability

Speed of Relief

Targets: 1 Radial Axis per Objective

Timelines Budget

Adverse Events

Abnormal Lab Changes

Pain Relief

Drug Stability

Speed of Relief

Targets: Simplify

Timelines Budget

Adverse Events

Abnormal Lab Changes

Pain Relief

Drug Stability

Speed of Relief

Target: Minimize Clutter

Tolstoy?

⇒Visual, rapid, high level understanding without having to read & interpret

⇒ Click on each target to drill down for details

Happy families

are all alike;

Every failed project

fails

in its own way.

--Not Tolstoy

Every unhappy family

is unhappy in its own way. --Leo Tolstoy

(Anna Karenina)

Successful projects

are all alike;

CONVENTIONS

Convention: The waterline

Above the waterline:

• Known unknowns

• Things we know we don’t know

• Eg, efficacy

• Things to get us off the ground(…MUST have some Green…)

Below the waterline:

• Unknown unknowns

• Things we don’t know we don’t know

• Eg, safety•Things to sink us(…MUST NOT have any Red…)

Grouping Attributes by Direction

Easy to see

general

areas of

success

and failure!

Adding Confidence/Certainty RangesElicited opinions,

Statistical Calculations

• Best Case

/Worst Case

• Hi/Med/Low

Uncertainty

• Best likely

/ Worst Likely

• Optimistic

/ Pessimistic

• Estimates

+ 95% Confidence Intervals

Piling on the Symbols

Circles

and pluses

and X’es,

oh my!

Perhaps:

●⟶Splats are point estimatesO ⟶ Past Estimates

+ ⟶ Optimal (nice to have)?

X ⟶ The competition?

Extreme Example (24 axes x 6 values)

• Complex but Interpretable

• Additional Symbols take some effort

• Splats aren’t bad

• Use sparingly

● Point estimates

― RangeO Past Estimates

+ Optimal

X The competition

PRACTICAL EXAMPLES

Comparing Projects in the ABC program

At a glance, can see

successes & failures!

--AndWhere!

Study ABC-OhNoStudy ABC-GoGo

Study ABC-GoSlo Study ABC-NoNo

Larger project with some issues –apparently including damaged

equipment

Big Picture: Multiple Project Dashboard

(Randomly generated dredging examples)

Small project with many

problems, but on schedule and in

budget

Larger project with a couple of possible

problems, but overall doing well

Metrics for Dredging Projects& Summary over all Projects

Bad weather, so a little behind

schedule, but in budget

Bars indicate range of metric over all projects

Mass Screening of Evolving EnzymesCriteria:

At each step:• Pick best candidates for each criteria• Recombine those to generate new candidates• Repeat until optimal

Alkalinity, Acidity, Yield, Salinity, Metal Tolerance, Durability, Km (Michaelson’s constant)

T.Targets provide comprehensive & visualfeedback on process

Gen

erat

ion

X

Ge

ne

rati

on

1

Ex. Transfusion RisksWhole plot ≣ One Patient

Each Radial Axis ≣ Blood type Group

Each Dot≣ Patient Risk

Green Dot Low Risk⇒ Normal procedures

Red Dot High Risk⇒ Clear & identified risk⇒ Special Procedures

Yellow Dot Uncertain RiskModerate Risks

⇒ Caution⇒ Further testing?

No Dot No Information¡No confusion with Low Risk !

Transfusion Risk of Multiple Patients

Risks for 16 patients⟶ Each Legible3 Redundancies:

• Color ≣ Traffic-Light coding

{Red, Yellow, Green}

• Location ≣ TargetsRed on Rim, next to label⟶ Easier to identify

• Size ≣ Proportional to Risk

The Tolstoy principal: "All happy families are alike; Every unhappy family is unhappy in it's own way.” (Anna Karenina)

Low Risk patients look alikeHigh Risk patients are distinct!⟶What way & How much

Predicted Risks vs Outcomes

• Unclear connections between Risks & Outcomes

• Add feedback on outcome• Some risks may have a

stronger connection with bad outcomes

Circle the target(Same conventions)

Good Outcome ≣ Thin GreenMixed outcome ≣ YellowBad outcome ≣ Thick RedUnknown outcome ≣ No Circle

Example: ―――⟶One Bad Outcome Patient with 2 risk factors

Mostly Good Outcomes Even with Risk Factors

Assessment Scores

• For 12 patients

• Factor scores from 3 assessments

• Mania, Depression, Schizophrenia

• 3 Total Scores+ 11 subscales

Ex. Gallup Well-Being Index

Pennsylvania

PA City Rankings as Tolstoy Targets

For more information, see my website:www.dennis-sweitzer.com

My linked-In profile:http://www.linkedin.com/in/dennissweitzer

Or email me:denswei@gmail.com