2012 03 19 webinar

© International Institute of Business Analysis

Cover this area with a picture related to your presentation. It can be humorous.

Make sure you look at the Notes Pages for more information about how to use the template.

IIBA® Pharma/Biotech SIG Monthly Webinar

What about ...? What I wished I had asked the regulatory and testing people.

March 19, 2012

Griffin Jones, Congruent

Compliance

Agenda


• Welcome & Announcements

• Presentation

“What about ...? What I wished I had asked the regulatory and testing people.”

• Q&A

Welcome


• Today’s IIBA Pharma/Biotech SIG Staff

Host

Michael Calluori (Sanofi), VP of the Community

Question Moderator

Carol Scalice (Pfizer), President

Webinar Administrators

Matt St. Louis (Pfizer), VP of Marketing

Jim Blay (Covance), VP of Technology

Join the Community


• Starting April 2012

Closed webinars (not public)

Must be an IIBA® member

• Join the Community

http://community.iiba.org/pharmabiotechsig

No additional cost

http://community.iiba.org/pharmabiotechsig

Volunteer


• Help Needed – Open Positions

VP of Event Planning

VP of Communications

Tell us what you think…


• We want to know what you think about our webinars

• We want to know you would like in the future

• Look for a survey from us in the near future, we promise it will be very brief and we really want to know what you think!

Who to Contact


Please contact one of the board members listed below:

Michael Calluori: VP, [email protected]

Mike Horn: VP, [email protected]

Leah Estberg: VP, [email protected]

Other Board Members


Carol Scalice: [email protected]

Matthew St. Louis: VP, Marketing [email protected]

Jim Blay: VP, [email protected]

Next Webinar


• Pre-Competitive Collaborations: The Pistoia Alliance

John Wise, Executive Director, Pistoia Alliance

Monday, April 30th, 11a-12p EST

Register Today

https://www2.gotomeeting.com/register/227390282

• Call for Speakers

Send us your topic ideas and submissions today!

Today’s Presentation


• What about ...? What I wished I had asked the regulatory and testing people. Speaker: Griffin Jones

Owner & Principal Consultant, Congruent Compliance

• Q&A “Raise Hand” Chat Window

11

What about ...? What I wished I had asked the regulatory and testing people.

Requirements Analysis (Chapter 6 in the BABOK® Guide) describes the business analysts’ role in the verification and validation of requirements. During

that task, business analysts’ are in direct contact with the stakeholder roles of tester and regulator.

Why are those roles special in an FDA regulated context, and how can a business analyst more

effectively interact with them?

3/19/2012IIBA Pharma/Biotech SIG

Outline three current topics relevant to the tester and regulatory roles

Opportunity to ask your questions Leave with a better understanding and

empathy for the concerns and issues of testing FDA regulated products and services

Goal for this Webinar

3/19/2012 12IIBA Pharma/Biotech SIG

Introduction◦ SW Testing and Regulatory Compliance◦ Medical devices and Clinical Research Organization

Three current topics in Testing & Regulatory◦ How to do Perfect Testing

Jim Bullock’s simple summary◦ The FDA’s initial thinking on Agile practices

Summary of AAMI Agile TIR SW Committee’s“Guidance on the use of AGILE practices in the development of medical device software”

◦ Why testing experts disagree Status on the “Schools of Testing” conversation

Audience Questions

Introduction and Overview


James Bullock’s “The Big Book of Perfect Testing”◦ http://www.qasig.org/past_meetings.html

Review James’s Five Principles Follow up with Jerry Weinberg’s

◦ Perfect software: And Other Illusions about Testing Read it yourself, then share it with others

How to Do Perfect Testing


1 - Testing is an informational function. ◦ You are paying to learn something.

Any time your bury or misinterpret information you get from testing you are corrupting what you are paying for.

2 - Testing produces information reliably grounded in the observed behavior of a system. ◦ Observation is what separates testing from analysis,

induction, deduction and speculation. Any time you argue yourself out of an observation because what

you see "just doesn't make sense" or create circumstances that corrupt the observing, you are defeating the purpose of testing.

One of the most powerful observations that can come from testing is noticing what is not being, or cannot be reliably observed. Bury that information at your peril.

3/19/2012IIBA Pharma/Biotech SIG 15

The Big Book of How to Do Perfect Testing“If you would succeed, know why you do what you do.”

James Bullock © 2004, 2006

3 - Information is valuable if you act on it. ◦ Otherwise it is entertainment. So, each bit of information

produced by testing needs a customer; a willing, enthusiastic customer eager to use the information they hope to get from testing to do a better job. If you are a tester without customers, your job description is

inoperative. If you are a manager whose testers don't have eager

customers, your testing is irrelevant. If you work at creating artifacts, especially software, and

your testers aren't your best friends why are you so sure you are right?




4 - The techniques of testing are endless - literally. ◦ The correct technique is the one that best gets you the information

you need to take a better next step than you would otherwise. If you are stuck on a technique rather than the results, you have

descended into ritual. If you or your testers are unwilling to discover, embrace or if need

be invent whatever technique it takes to find the information you need, you aren't doing testing, you have descended into administration.

◦ So what do you really want to know about your system, and how do you find that out?

5 - Act always according to these four principles and you will do perfect testing, every time. ◦ The rest is details.




Is there one principle that resonates with you?

1. Testing is an informational function. You are paying to learn something.

2. Testing produces information reliably grounded in the observed behavior of a system.

3. Information is valuable if you act on it. 4. The techniques of testing are endless - literally.

Take a better next step. 5. Act always according to these four principles.


Questions? Comments?

Search for the current draft of AAMI Agile TIR SW Committee’s◦ “Guidance on the use of AGILE practices in the

development of medical device software” http://www.aami.org/committeecentral/Committee/

CommitteeDocuments.cfm?ComID=0SW0000 (Note that access to the doc is currently restricted)

Expect that this will be applied across all SW◦ regulated by FDA & other non-US regulatory bodies

For AGILE to be established in the medical device software industry, guidance is needed to adapt it to fit that unique context.

The FDA’s initial thinking on the use of Agile practices


Agile adds value and can be adapted to this context

The values of AGILE add to the QMS values Demonstrate how your incremental SW dev.

lifecycle satisfies regulatory requirements Agile needs strong change management

systems Agile needs a functioning QMS to live inside

Five Recommendations


This is an important document!Read the detailed recommendations

as soon as you can.

Has anyone else had access to the document?

Who else has started implementing Agile practices? What works well? Challenges?



Schools of Software Testing◦ Share some of Bret Pettichord’s slide deck◦ http://www.prismnet.com/~wazmo/papers/

four_schools.pdf Debate: Are Testing “Schools” a good idea?

– Doug Hoffman & James Bach◦ http://www.associationforsoftwaretesting.org/2012/03/08/debate-

are-testing-schools-a-good-idea-doug-hoffman-james-bach/

Why testing experts disagree


Recent separation in the Context-Driven school◦ Cem Kaner

http://context-driven-testing.com/?page_id=9◦ James Bach

http://www.satisfice.com/blog/archives/724◦ Michael Bolton

http://www.developsense.com/blog/2012/03/i-might-be-wrong/

◦ Scott Barber http://scott-barber.blogspot.com/2012/02/is-testing-dead-

dunno-but-context.html◦ Matthew Heusser

http://xndev.blogspot.com/2012/03/lets-call-calling-off-off.html

Why testing experts disagree


I find the school model to be useful to explain my behavior, or as a line of inquiry to understand someone else’s behavior.

I find “Standard” and “Quality” schools are common in Pharma/Biotech industries

Think about what school you might belong to, and why. Perhaps you are a blend? (Griffin shares his school)



What questions do you have?

Questions from the Audience


Pre-Questions


Safe and Effective Risk-based Under Control Accountability Data driven decisions

The real intent of FDA regulations


Is it really an FDA or regulator requirement that users have to do scripted system testing?◦ No. Why people think/say that is interesting.

Been trying to push back because what they have been doing as UAT is really just users doing functional system testing. ◦ All that tells you is whether the systems fulfills the

requirements ◦ It doesn’t tell you if the user actually accepts and

can use the solution delivered.◦ Agree - See Jim Bullock’s five principles

User acceptance testing practices


are they documented? where do we find them? how should we document or refer to them in

our requirements?◦ How do you currently handle the regulatory

requirements for21CFR Part 11 Electronic Records; Electronic Signatures?

Eliciting requirements specific to regulations


How to “cover yourself” in case you are audited?◦ YOU are the expert◦ Explain why what you are doing is sufficient◦ Too many requirements or too much detail

is as bad as too little detail

Best practices to document requirements


How those changes are documented◦ Follow your processes◦ If your processes are broken/unsustainable –

expose the pain to help get the processes fixed.

Dealing with change control in a validated solution


What is different in the Pharma/Biotech world related to verification & testing?

What is the same?

Verification and Testing in Pharma/Biotech


Blending regulatory compliance with Agile practices to create sustainable validation compliance

Validation solutions that are practical and tailored to their business model context

Alternative options to difficult software validation problems◦ E.g., Quintiles white paper: “The Art of

Compliance” http://www.quintiles.com/information-library/white-papers/sustainable-

regulatory-compliance/

How to do theses?


The regulatory is not your business partner

The regulatory has police powers See my other presentation

◦ http://www.associationforsoftwaretesting.org/?dl_name=What_do_auditors_expect_from_testers_-_Griffin_Jones.pdf

Regulatory risks to business


I have another presentation that may be helpful for FDA regulated project teams members◦ “What do auditors expect from testers?

The audit survival heuristics of an FDA regulated exploratory testing team”

http://www.associationforsoftwaretesting.org/?dl_name=What_do_auditors_expect_from_testers_-_Griffin_Jones.pdf

Audit Survival Heuristics


Griffin Jones

Congruent [email protected]

Thank you for attending


Thank you for attending!


• Next Webinar “Pre-Competitive Collaborations: The Pistoia Alliance” – John

Wise, Executive Director, Pistoia Alliance Monday, April 30th, 11a-12p EST Register: https://www2.gotomeeting.com/register/227390282

• Contact the President: [email protected] Topic ideas and speaker submissions Volunteering – Communications, Event Planning Joining the Community

• Look for our survey in the near future!

https://www2.gotomeeting.com/register/227390282

2012 03 19 webinar

Technology

Transcript of 2012 03 19 webinar