A Web-Based Teaching Tool for Psychotherapy Research and Training

Developed by

Jakob Valen and Leigh McCullough

Making Psychotherapy BetterStep By Step

There have been daunting challenges in

Psychotherapy Process Research

This involves the following….

•Archives of thousands of VIDEOTAPES from Randomized Controlled Clinical Trials of Psychotherapy.

•Development of many PROCESS MEASURES to evaluate therapy

•Training of hundreds of RESEARCH RATERS

• 2,000 VIDEOTAPED SESSIONS PER TRIAL

• 2 HOURS PER SESSION TO EVALUATE

= 4,000 HOURS PER TRIAL or10 YEARS PER PERSONTo rate one clinical trial With just one instrument

CHALLENGES IN TRAININGPROCESS RESEARCHERS

But we need to rate each session with a number of well-established

PROCESS MEASURES from a wide variety of domains of psychotherapy

•For Example:

•Patient-Therapist Interaction:(Alliance, Attunement, Ruptures)

•Therapist Interventions: (Clarification, Support, Cognitive Strategies)

•Patient Responses: (Maladaptive cognitions, Affect, Defense)

In short, it would take

a small army of research

raters a few decades to

complete all the ratings

that are needed

Meanwhile….

The Traditional Method of Training is Labor Intensive and Time Consuming

Ongoing small group teaching by the originator of the instrument can take months and yield only a few reliable raters.

See Note Below

So what are we to do?

Leigh McCullough, a Visiting Professor in Trondheim, Norway, was explaining this

problem when, a doctoral student, named

Jakob Valen, had an idea…..

STDPPsychodynamic

CBTCognitive

DBTBehavioral

There is such an archive being built in Norway:The Trondheim Psychotherapy Research Program

Videotape Archives Library(Tapes from at least 3 RCT’s will be studied)

There is also a large number of graduate students in clinical psychology

At the University in Trondheim, Norway (NTNU)

Why not Build a Web-Based System for Process Research

Slide 38

So that many raters could be trained to evaluate the thousands of videotapes that have been

collected from the many clinical trials of psychotherapy

So Jakob Valen

began to build the program

www.jakobsladder.com

And Leigh McCullough helped him develop it

JakobsLadder: Internet Training

Students could go to their computer,log on to jakobsladder.com and select an instrument that they would be interestedin learning.

And now, so can you….

The Introductory Page of Jakobsladder

FIRST, REVIEW THE LIST OF INSTRUMENTS THAT ARE AVAILABLE;

Each one assesses important aspect of psychotherapy

See Note Below

THEN, SELECT AN INSTRUMENTTO EVALUATE THERAPY PROCESS

AAS - THE AFFECT ATTUNEMENT SCALEATOS – ACHIEVEMENT OF THERAPEUTIC

OBJECTIVES SCALE (COMMON FACTORS)DMRS – DEFENSE MECHANISM RATING SCALE

For example, we will first select the DMRS

When you select the DMRS ( The Defense Mechanisms Rating Scale), you would go to this page….

The menu to the left allows you to select the DMRS Manual

or to select segments to be Rated

See Note Below

See Note Below

When you click on one of the ‘Ratings’ buttonsyou will receive transcripts of sessions -

as you see above.

EXAMPLE FROM THE TEXTSee if you can find a defense

mechanism in this selection….

As you read the transcript and note a defensemechansim, click on the menu to the right, where you see “Defence Mechanism #1”.…

The Defense Drop-Down Menuguides the process……

When you select the Defense Mechanism buttonYou will be asked to do three things:1. Select a defense from the drop down list2. Briefly state your reason for the selection3. Cut and paste the example from the text that illustrates the defensive behavior

See Note Below

Use the cursor to select a

defense, in this case ‘Reaction

Formation’

The Defense drop-down menu will giveyou a list of Defensesto choose from

The selected defense will appear in the defense box, and the rater will be able to type in the reason for the selection

The rater proceeds through the transcripts until all defenses have

been identified.

This process provides important information on the psychological

functioning of the patient for both research and clinical training.

Let’s take another example…Maybe you want to learn about common factors

in psychotherapy such as insight, motivation, or experiencing of feeling…

Go to the Instrument List

and select the ATOS Scale

This is the Home Page for the ATOS Scale

See Note Below

Use the Menu to select the ATOSManual or Practice Ratings

See Note Below

With the ATOS Scale…. You can choose to rate a

Transcript Or you can rate from

Videotapes.

First we’ll show an example of a

Transcript.

This is what the transcript page looks like….

See Note Below

Close up of the ATOS Transcript Rating Page

Or you canPRACTICE RATING

A VIDEOTAPED SEGMENT

It is much more enjoyable and educational to

watch and discuss a videowith another rater

See Note Below

See Note Below

This is the page that is shown when videotapes are rated.

See note below

Some examples of data the ATOS rater enters…

45 60 15

See note below

The rater then can compare their rating to the Gold Standard ratings…as wellas see an explanation of why the group of experts gave the rating they did.

60

Another example of feedback provided by the

ATOS program.

When the rater has completed rating approximately 20 segments on a specific

instrument, the rater may receive a “grade” or a proficiency rating in the

form of a RELIABILITY STATISTIC

.43

RESULTS OF RATINGS

See Note Below

JakobsLadder: ICC Rating

GRADING SYSTEM

Above .7 = GOOD

.4 -.7 = FAIR

Below .4 = POOR

Sometimes the rater will receive this message; “You’re not quite reliable,

but keep practicing and you’ll improve!”

So , JUST KEEP PRACTICING…

AND PRACTICING…

.75

And before too long, you’ll be able to say,

“I’M RELIABLE!”

BENTE

JOHANSEN

has met

criteria

for Reliability

on the

DMRS Scale

At .75

When the rater becomes reliable on the rating instrument, a Reliability Certificate can be printed out

The ICC reliability rating can then be given to the Principal Investigator of a study to verifiy tat the rater is able to begin research ratings.

But a reliability score also informs clinicians-in-training that they satisfactorily understand the clinical concepts that are being evaluated; such as the types of defense

mechanisms a patient uses or the degree of affect experienced in the session.

EXCEL FILE

Once research ratings have been entered, they are storedon an Excel file to be easily accessed and analyzed by the

Investigators in the study.

See Note Below

Also available is a program for a

Videotape Grid for each study. Like a card catalog of a library, each session is

identified by a box and the signs in the box indicate whether it is available, checked out, rated by one or more process instruments and by how many raters.

See note below for more info.

Thus, JakobsLadder is not only useful for:

TRAINING PROCESS RESEARCHERS

But is also an excellent tool for

TEACHING PSYCHOTHERAPY

AIRLINE PILOT STANDARDS

Just as pilots have to log hundreds of hours in the ‘test cockpit’ before being given the responsibility of flying a plane

and transporting live passengers, hopefully we will develop ‘simulated therapy sessions’ that will adequately prepare therapist trainees before they are given the

responsibility of working with suffering patients.

Someday we hope there will be archives of videotapes available for therapist trainees to review and evaluate

hundreds of hours of pre-selected segments of therapy demonstrated by senior therapists,

so that they will have ample exposure to

how therapy might optimally be conducted.

How does Freud fit into all this?

Freud envisioned a “Scientific Psychology”

What might Freud think of JakobsLadder?

?

A web based tool for teaching psychotherapy

JAKOBSLADDERDeveloped by

Jakob Valen and Leigh McCullough

Making Psychotherapy BetterStep By Step

The End

Hmm...

OTHER INFORMATION ABOUT HOW TO LEARN

MORE ABOUTTHE ATOS RATING

SYSTEM

www.affectphobia.com To obtain ATOS manual for coding

psychodynamic processes

A treatment manualthat describes the

interventions on which the ATOS Scale is based

Great Resources for Psychotherapy Process Research! But how do we train all these graduate students

to rate thousands of tapes?