SINGLE-USE SYSTEMS CHANGE-NOTIFICATION SCORECARD … · 2019. 12. 15. · BioPhorum Operations...

1

CONNECT COLLABORATE

ACCELERATE TM

SINGLE-USE SYSTEMS CHANGE-NOTIFICATION

SCORECARD SURVEY

Scorecard survey 2©BioPhorum Operations Group Ltd

Contents

1.0 Executive summary........................................................................................................................................... 6

2.0 Introduction ........................................................................................................................................................ 7

2.1 Purpose ..............................................................................................................................................................................................7

3.0 Questions asked ................................................................................................................................................ 8

3.1 Respondentprofiling ......................................................................................................................................................................8

3.2 QuantityofSUSchangenotifications .......................................................................................................................................8

3.3 QualityofSUSchangenotifications ..........................................................................................................................................8

3.3.1 Categorization and timing ....................................................................................................................................................... 9

3.3.2 Change description effectiveness ........................................................................................................................................ 9

3.3.3 Data package quality ................................................................................................................................................................. 9

3.3.4 Workflow/Communicationeffectiveness .......................................................................................................................10

3.3.5 Overall satisfaction ..................................................................................................................................................................10

3.3.6 Ranking .........................................................................................................................................................................................10

4.0 Development of the survey .......................................................................................................................... 11

4.1 Guidanceoncompletingthesurvey ....................................................................................................................................... 11

5.0 Results ................................................................................................................................................................ 13

5.1 Respondentprofiles .................................................................................................................................................................... 13

5.1.1 Respondent company type ....................................................................................................................................................13

5.1.2 Respondent roles ......................................................................................................................................................................14

5.2 QuantityofSUSchangeshandled ........................................................................................................................................... 15

5.2.1 Mean number of SUS changes handled ............................................................................................................................15

5.2.2 Variability in number of SUS changes handled ..............................................................................................................16

5.3 QualityofSUSchangenotifications ....................................................................................................................................... 17

5.3.1 Outlier ..........................................................................................................................................................................................17

5.3.2 Mean rating for each aspect ................................................................................................................................................18

5.3.3 Mean scores for each aspect surveyed in different subgroups of the supply chain .........................................19

5.3.4 Inclusion of outlier data .........................................................................................................................................................20

5.3.5 Variabilitywithinrespondentgroups ...............................................................................................................................21

5.3.6 Ranking of perceived challenge and perceived impact ...............................................................................................23


List of figures

Figure1:16Companiesrespondedtothesurvey,eightoftheseself-identifiedasdrugsponsors,sixassuppliersofSUSassemblies andtwoassuppliersofSUScomponents................................................................................................................................................................................................................................................................. 13

Figure2:Cross-functionalteamswereusedbythreedrugsponsors.‘Other’functionsidentifiedbysuppliersincludedregulatorysupport andmarketing/productmanagementroles. ........................................................................................................................................................................................................................................................... 14

Figure3: Mean number of SUS changes handled by different respondent groups. A clear trend is seen, withanincreaseinthenumberofchangenotificationsissued/receivedasproductsmovefromcomponentmanufacture,throughassemblytouse. ......................................... 15

Figure4:Variabilityinthenumberofchangesrelatedtosingle-usesystemsreviewedbydrugsponsorsorissuedbysuppliersoverthelastyear. .............................................. 16

Figure5:Meanindustryscoreandtargetforeachaspectofchange-notificationpracticessurveyed ......................................................................................................................................... 18

Figure6:Comparisonofmeanscoresbetweenrespondentgroupsforeachaspectsurveyed ....................................................................................................................................................... 19

Figure7:Comparisonofmeanscoresbetweenrespondentgroupsincludingoutlier ......................................................................................................................................................................... 20

Figure8:Low,meanandhighscoresforresponsesreceivedfromdrugsponsors ................................................................................................................................................................................ 21

Figure9:Low,meanandhighscoresforresponsesreceivedfromsuppliersofSUSassemblies ..................................................................................................................................................... 22

Figure10:Low,meanandhighscoresforresponsesreceivedfromsuppliersofSUScomponents .............................................................................................................................................. 23

Figure11:Averageperceptionofchallengeofimprovingeachareaofthechange-notificationprocess ................................................................................................................................... 24

Figure12:Averageperceptionofimpactofimprovingeachareaofthechange-notificationprocess ......................................................................................................................................... 25

6.0 Discussion.......................................................................................................................................................... 26

6.1 Numberofchangeshandled ....................................................................................................................................................26

6.2 Estimationofqualitativeaspectsofsingle-usesystemchangenotifications ............................................................26

6.3 Categorizationandtiming ......................................................................................................................................................... 27

6.4 Changedescriptioneffectiveness ........................................................................................................................................... 27

6.5 Datapackagequality .................................................................................................................................................................. 27

6.6 Workflowandcommunicationeffectiveness ..................................................................................................................... 27

6.7 Rankingofthechallengeandimpactposedbyincreasingscoringineachaspect

ofthechange-notificationprocess ........................................................................................................................................ 28

7.0 Conclusion ......................................................................................................................................................... 29

Appendix ....................................................................................................................................................................... 30

References .................................................................................................................................................................... 31

List of tables

Table1: Industry mean score for each aspect and associated outcome from the scorecard model .............................................................................................................................................. 26


Authors

BioPhorum Sam Denby


About BioPhorum

TheBioPhorumOperationsGroup’s(BioPhorum’s)missionistocreateenvironmentswheretheglobalbiopharmaceutical industry can collaborate and accelerateitsrateofprogress,forthebenefitofall.Since its inception in 2004, BioPhorum has become theopenandtrustedenvironmentwhereseniorleaders of the biopharmaceutical industry come together to openly share and discuss the emerging trends and challenges facing their industry. Growingfromanend-usergroupin2008,BioPhorumnowcomprises53manufacturers and suppliers deploying their top 2,800 leaders and subject matterexpertstoworkinsevenfocusedPhorums,articulatingtheindustry’stechnologyroadmap,definingthesupplypartnerpracticesofthefuture,anddevelopingandadoptingbestpracticesindrugsubstance,fillfinish,processdevelopment and manufacturing IT. In each of these Phorums, BioPhorum facilitators bring leaders together to create future visions, mobilize teams ofexpertsontheopportunities,createpartnershipsthatenablechangeandprovide the quickest route to implementation, so that the industry shares, learns and builds the best solutions together.


1.0

Executive summaryThissurveyisakeytoolforcontinuousimprovementintheongoingmissionoftheBioPhorumdisposablesprogramtoachieveatleast90%right-first-timesupplierchangenotificationsforsingle-usesystems(SUS)usedinthebioprocessingindustry.

Itisanticipatedthatachievingthisgoalwillprovidesignificantcostandtimesavings,improve

qualityandprovideconfidencetomorerapidlyadoptsingle-usesystems,andtheflexibilitythey

provide as a strategic tool for biopharmaceutical manufacturers.

Thisisthefirsttimethissurveyhasbeenrunanditthereforeestablishesabaselineforthe

performanceofchangenotificationsforsingle-usesystemswithinthebioprocessindustry.

Resultsofthesurveyhighlightthatatpresenttheindustryisoperatingat50to75%right-first-

timesupplierchangenotificationsinfourkeyaspectsmeasuredinthesurvey(categorization

andtiming,changedescriptioneffectiveness,datapackagequalityandworkflow/

communicationeffectiveness).Thetargetamongbiomanufacturersandsupplypartnersisto

increase this to greater than 90%.

Thesurveyresultsrevealthatthereappearstobelimitedalignmentofwhatconstitutes

‘good’inrespectofdatapackages,whichpresentsakeychallenge.TheBioPhorumsupplier

changenotificationworkstreamisseekingtoaddressthisbyproducingcontenttostandardize

datapackagerecommendations.Ultimately,theteambelievesthataddressingthiswilldrive

significantimprovementinworkfloweffectivenessforchangenotification.

Movingforward,theintentionistousethistooltodemonstratethebenefitsthatalignmentof

change-notificationpracticesbringsandtheindustry’sprogressinthiscontinuousimprovement

initiative.Thenextstepwillbetorunthissurveywithabroaderaudience,andyouareencouraged

to invite your customers, your suppliers and their suppliers to participate in this survey.


2.0

IntroductionSince2015,theBioPhorum/BPSASingle-UseSystemsChangeNotificationteamhavebeenworkingtodevelopastandardized,industry-wideframeworkforthenotificationofchangestosingle-usesystemsusedinthebioprocessingindustry.In2015theteamidentifiedthechallengesposedbycurrentwaysofworkingandtheneedforanindustry-widesolution(WhiteandOtt,2015).Theteamdevelopedaproposedmethodologyandpublishedanindustryproposalforchange-notificationpracticesforsingle-usebiomanufacturingsystems(Carteretal.,2017a;Carteretal.,2017b).Asignificantnumberofmembercompaniesexpressedinterestinimplementingthesepracticesandderivingbenefitfromthem.

Followingpublicationofthepaperin2017,itwasclear

thattherewerechallengesinthechange-notification

processes for single-use biomanufacturing systems. The

industryproposalforbestpracticewaswrittentoimprove

change-notificationprocessesacrosstheindustry.The

teamevaluatedthesituationandacknowledgedthatwhile

therewasacollectiveunderstandingthattheprocess

couldbeimproved– andareasforimprovementhadbeen

identified– therewasnomechanismbywhichtosurvey

andmeasurethestateofchangenotificationsforsingle-

use biomanufacturing systems. Furthermore, the team

identifiedtheneedtomonitorwhetherthewaysofworking

outlinedinthe2017bestpracticeproposalwereeffective

inimprovingthestatusofchangenotificationsforthese

products. To this end, the team agreed to periodically run a

scorecardsurveytoestablishhoweffectivetheproposed

best practices are in bringing about improvements to

change-notificationprocessesintheindustry.

2.1 PurposeThe purpose of the survey is to understand the aggregated

perception of the industry from drug sponsors through the

supply chain. Currently, the survey is designed to capture

input from component manufacturers and single-use

systemintegrators,althoughitcouldbeextendedthrough

the supply chain.

Each time the survey is run, participating companies are

requested to complete the survey by providing one collated

response from their company. The purpose of the survey

hosted by BioPhorum is not for companies to complete the

scorecard survey separately for each supplier and customer

interaction.Companiesarewelcometousethescorecard

tomonitororimprovespecificrelationshipsaspartoftheir

quarterlybusinessrevieworotherprocessbutshould

not report the measurement of a single relationship to

BioPhorum.

A central tenet of the survey design is that it should be

easy to complete and should not provide an unnecessary

administrative burden. During the design of the scorecard

survey,itbecameapparentthatveryfewcompanieshave

readilyavailablehardmetricsaroundhowmuchrework

isnecessaryduringachangenotification,howeffectively

changes are categorized or the completeness of data

packagesprovided.Furthermore,whilemostcompanieswere

abletoidentifyhowmanychangestheyhadhandledintotal

duringagiventimeperiodandwhatpercentageofthose

wereattributabletosingle-usesystems,mostcompanies

werenotabletoprovideharddataabouthowlongithad

takenthemtohandlechanges,howmanyassociateshad

beeninvolvedorwhetherthisvariedby‘severityofchange’.

Thismaybesomethingtoreviewasbestpracticesbecome

embedded through the industry and there is increasing

standardizationonhowchangesshouldbecategorized.

Forthemoment,whatismostimportantisunderstandingthe

directionoftrendsrelatingtochangenotificationforsingle-

usesystemsandwhethertheproposedpracticesarehaving

thedesiredeffect.Tothatextent,theteamagreedthatasking

each company for consolidated feedback from the associates

handlingthechange-notificationprocessshouldprovide

a good measure of the current status of single-use change

notificationswithintheindustry.

Thisisthefirsttimethesurveyhasbeenrunandsothe

purpose is to establish a baseline of the current status of

changenotificationamongthecompaniesparticipatingwith

theworkstream.


3.0

Questions asked

3.1RespondentprofilingSomestandardquestionswereaskedtoenablefollowupandclarification(nameofrespondent,respondent’sorganization).Thisinformationwillnotbepublishedandalldatashownisaggregatedandanonymized.

Forthisroundofthesurvey,respondentswereprofiled

byrole(qualityassurance(QA),supplierquality,

regulatory affairs, process development, technical

services/manufacturingsciencesandtechnology(MSAT),

procurement,orother(specify)).Whereacross-functional

teamresponded,itwaspossibletoselectmultiple

roles.Respondentswerefurtherprofiledbyhowtheir

organization’sbusinessselfidentified(drugsponsor,

contractmanufacturingorganisation(CMO),supplierof

SUS assemblies, supplier of SUS components, supplier of

SUSrawmaterials).

3.2QuantityofSUSchangenotificationsRespondentswereaskedhowmanychangenotifications

theyhadreceived(drugsponsors)andissued(supply

partners)forsingle-usesystemsinthepreviousyear.They

werealsoaskedtoindicatewhetherthetrendinthese

numberswasincreasingordecreasing.

3.3QualityofSUSchangenotificationsQuestionswereaskedtoidentifyrespondentscurrentand

desiredexperienceofchangenotification.Ineachsection,

questionswereaskedinpairs–howwouldyourateyour

currentexperienceandwhatratingdoyouneedtoachieve?

Inallcases,companieswereaskedtoprovideascore

between0and10fortheirexperienceinrelationtoeach

questionwith0beingextremelynegativeand10being

extremelypositive.AframeworkisprovidedinPerforming

thissurveyhasbeenaveryusefulexercise,providing

some great insights into current areas of convergence

anddivergenceofcompanies’experienceswithchange-

notificationprocessthroughoutthebiopharmaceutical

supplychain.Importantly,sincethisisthefirsttime

the survey has been run it has established a baseline in

termsofcurrentperformanceofthechange-notification

processes employed by respondent companies. It has

highlightedsignificantdifferencesinthenumberofchange

notificationsdrugsponsorsarehandlingforsingle-use

systems. Establishing the current status of these practices

is important as the intention of the team is not to increase

thenumberofchangenotificationsreceived,butrather

toidentifyandfocusonthechangenotificationslikelyto

have the most impact.

The survey has also been very useful in uncovering a

‘revealedproblem’.End-usersconsiderdatapackagesto

behighlyimpactfultothechange-notificationprocess.A

gooddatapackage,deliveredwiththechangenotification,

providestheleanestworkflowandminimizestheneed

tofollowupwithadditionalquestionstothesupplier.

Since the drug sponsors are the ultimate customers in

thissupplychain,theywilltypicallydrivereworkand

requests for additional information from the supply

chain.Figure8showsthatthereisalackofagreement

betweendrugsponsorcompaniesonthequalityof

thedatapackagestheycurrentlyreceive.Reflecting

onthis,theteamacknowledgedthatatthetimeofthe

survey,littleworkhadbeencarriedouttodefinewhat

constitutes‘good’withrespecttosingle-usesystem

change-notificationdatapackages.Withoutalignment

betweendrugsponsorsitisverydifficultforsupplypartners

tomeetexpectationsasthesemaydifferbetweendrug

sponsor companies. The impact of this on timelines may be

exacerbatedwhenrequestsforadditionaldataneedtobe

passed on from tier 1 suppliers through the supply chain.

Team agreed to a subgroup composed of both drug sponsors

andsuppliersworkingtodevelopguidanceondatapackage

recommendations.


Movingforward,thesurveyhasallowedcompanies

participating in the BioPhorum initiative to measure the

currentstatusofkeyaspectsofthechange-notification

process that their best practice is intended to address. This

formsabaselineagainstwhichcontinuousimprovement

efforts can be measured. A key revealed problem has been

identifiedpromptingthegrouptodevelopcontentto

addressthis.Theintentionisnowtorunthissurveywitha

broader audience and continue to measure the impact that

ourbest-practiceproposalishavingonchange-notification

single-usesystemswithinthebioprocessingindustry.As

morecompaniesimplementthispractice,whenthissurvey

isrepeateditwillbepossibletomeasurewhetherthe

goalsofthecollaboration(>90%right-first-timechange

notifications)arebeingmetorwhetherthereisaneedto

correctthecoursebyadaptingwaysofworkingorrevising

published tools and guidance.

3.3.1Categorizationandtiming

The questions in this section are designed to understand

whetherthecomplexity/impactofthechangehasbeen

wellunderstoodandeffectivelyincorporatedintothe

changenotification.Inthecaseofdrugsponsorsthiswill

be directly based on the changes they have received. For

supplypartnerstheaimistounderstandhoweffectively

they believe they are categorizing changes, typically based

onfeedback(orlackoffeedback)fromtheircustomerbase.

Thetwoquestionsaskedinthissectionare:

How would you rate your experience of Categorization and Timing? (0–5)

What rating do you need to achieve for Categorization and Timing ? (0–5)

3.3.2Changedescriptioneffectiveness

The questions in this section are targeted at understanding

whetherthechangehasbeendescribedeffectivelyenough

toallowanend-usertomakearight-first-timeimpact

assessmentforthechange,withoutneedingtorevertto

the supplier for additional information.


How would you rate your experience of change description effectiveness (and anticipated end-user-impact assessment)? (0–5)

What rating do you need to achieve for change description effectiveness (and anticipated end-user-impact assessment)? (0–5)

3.3.3Datapackagequality

This section addresses the availability and quality of data

packagessupportingachangenotification.Questionsin

thissectionaredesignedtounderstandwhetherdata

packagesareprovidedwithoutrequestandwhetherit

ispossibletousetheavailabledatapackageswithout

requesting additional information.


How would you rate your experience in relation to data package quality? (0–5)

What rating do you need to achieve in relation to data package quality? (0–5)


3.3.4Workflow/Communicationeffectiveness

This section addresses the effectiveness of both the

overallworkflowandcommunicationaboutthechange.

Changenotificationsandpre-notificationsshouldbe

providedinlinewiththeguidancesetoutintheindustry

proposalforchange-notificationpracticesassociatedwith

single-usebiomanufacturingsystems(Carteretal.,2017b;

Carteretal.,2017a).Wheretheworkflowiseffective,

anyreworkshouldbeminimal,allowingnotificationand

implementation of changes in a timely manner and in line

withtheguidanceforeachlevelofchange.


How would you rate workflow/communication effectiveness? (0–5)

What rating do you need to achieve in relation to workflow/communication effectiveness? (0–5)

3.3.5Overallsatisfaction

Thisquestionaimstounderstandhoworganizations’

perceptionofthechange-notificationprocessforsingle-

usesystemsisevolvingovertime.Respondentswerenot

askedwhattheirdesiredsatisfactionlevelwouldbe.

How would you rate your overall satisfaction with the single-use change-notification process? (0–5)

3.3.6Ranking

Questionsinthissectionaredesignedtoidentifythemost

valuableaspectsofchangenotificationtodrugsponsors.

Impactful areas that are relatively easy to achieve

representquickwins.

Respondentswereaskedtorankthefouraspects

(categorizationandtiming,changedescription

effectiveness,datapackagequality,workflow

effectiveness)from1–4,asfollows:

Please rank categorization and timing, change description effectiveness, data package quality and workflow effectiveness from most (4) to least (1) challenging.

Please rank categorization and timing, change description effectiveness, data package quality and workflow effectiveness from most (4) to least (1) impactful.


4.0

Development of the surveyThesurveywasdevelopedbythechange-notificationworkstream.Theintentionistousethissurveyoverthecomingyearstoassesshoweffectivelythebestpracticeprinciplesarebeingembeddedandsustainedthroughouttheindustryandhowpositivetheimpactonchangenotificationsforsingle-usebiomanufacturingsystemshasbeen.

4.1GuidanceoncompletingthesurveyIdeallytheteamwantedtousekeyperformanceindicators

(KPIs)basedonharddatacollectedbyparticipating

companies.Duetothecomplexnatureofthisco-

implementation,itwasnotpossibletoagreehard,data-

basedkeyperformanceindicatorsthatwerereadily

available.Allcompanieswithintheworkstreamhave

rigorous and effective change management processes and

have a sense that they are frequently needing to request

additionaldata,thatreworkisoftenrequired,orthatthereis

opportunity to improve the process. Many companies handle

changes through cross-functional teams and do not typically

recordhowmanyfulltimeequivalent(FTE)hoursare

associatedwithhandlingagivenchange,howcompletethe

datapackagewas,howlongthechangeshouldhavetaken

to implement and the time it actually took to implement.

If companies have any of this data, there is no agreement

onhowitshouldbemeasuredsotryingtocomparehard

datafromdifferentcompanieswouldbechallenging.As

companies adopt these practices, development of hard,

data-drivenKPIsmaybeaconsiderationforthefuture.

WhilehardKPIsmaynotbeavailable,practitionersof

changenotificationsandleadersofthesefunctionswithin

companiesdohaveasenseofhowtheirsuppliersand

customers are performing in respect to each of the aspects

covered in this survey. By asking each organization to

provide an aggregated score, the organization has the

opportunity to survey a representative group of associates

from different functions. By surveying a cross functional

teamdifferingviewsandexperiencesrelatedtoparticular

changenotificationsarebalancedout.Tosupportreaching

aunifiedscoreforthecompanyandtoprovidesomelevelof

moderationbetweencompanyresponses(preventingoverly

optimisticorpessimisticviewsfrombiasingoutputs),the

scorecardframeworkwasdeveloped(seePerformingthis

surveyhasbeenaveryusefulexercise,providingsomegreat

insights into current areas of convergence and divergence

ofcompanies’experienceswithchange-notificationprocess

throughout the biopharmaceutical supply chain. Importantly,

sincethisisthefirsttimethesurveyhasbeenrunithas

established a baseline in terms of current performance of

thechange-notificationprocessesemployedbyrespondent

companies.Ithashighlightedsignificantdifferencesinthe

numberofchangenotificationsdrugsponsorsarehandling

for single-use systems. Establishing the current status of

these practices is important as the intention of the team is

nottoincreasethenumberofchangenotificationsreceived,

butrathertoidentifyandfocusonthechangenotifications

likely to have the most impact.


The survey has also been very useful in uncovering a

‘revealedproblem’.End-usersconsiderdatapackagesto

behighlyimpactfultothechange-notificationprocess.A

gooddatapackage,deliveredwiththechangenotification,

providestheleanestworkflowandminimizestheneed

tofollowupwithadditionalquestionstothesupplier.

Since the drug sponsors are the ultimate customers in this

supplychain,theywilltypicallydrivereworkandrequests

for additional information from the supply chain. Figure

8showsthatthereisalackofagreementbetweendrug

sponsor companies on the quality of the data packages they

currentlyreceive.Reflectingonthis,theteamacknowledged

thatatthetimeofthesurvey,littleworkhadbeencarried

outtodefinewhatconstitutes‘good’withrespecttosingle-

usesystemchange-notificationdatapackages.Without

alignmentbetweendrugsponsorsitisverydifficultfor

supplypartnerstomeetexpectationsasthesemaydiffer

betweendrugsponsorcompanies.Theimpactofthison

timelinesmaybeexacerbatedwhenrequestsforadditional

data need to be passed on from tier 1 suppliers through the

supply chain. Team agreed to a subgroup composed of both

drugsponsorsandsuppliersworkingtodevelopguidanceon

data package recommendations.

Movingforward,thesurveyhasallowedcompanies

participating in the BioPhorum initiative to measure the

currentstatusofkeyaspectsofthechange-notification

process that their best practice is intended to address. This

formsabaselineagainstwhichcontinuousimprovement

efforts can be measured. A key revealed problem has been

identifiedpromptingthegrouptodevelopcontentto

addressthis.Theintentionisnowtorunthissurveywitha

broader audience and continue to measure the impact that

ourbest-practiceproposalishavingonchange-notification

single-usesystemswithinthebioprocessingindustry.As

morecompaniesimplementthispractice,whenthissurvey

isrepeateditwillbepossibletomeasurewhetherthe

goalsofthecollaboration(>90%right-first-timechange

notifications)arebeingmetorwhetherthereisaneedto

correctthecoursebyadaptingwaysofworkingorrevising

published tools and guidance.

Appendix1

Itisimportanttohighlightthatthisscorecardframework

is intended to guide thinking and support a discussion

toreachaconsensusaroundwhatscoreanorganization

provides in response to the questions. It is not intended

that companies collect data to justify that they are meeting

specificpercentageslaidout.Thereisalsosomeroomfor

interpretation.Forexample,whereasupplierissues10

notificationsinayearto50customersandreceivesone

report for nine of those changes that the categorization of

thosechangeswasnotcorrectand30reportsforoneof

thosechangesthatthecategorizationwasnotcorrectthis

couldreasonablybeinterpretedasascoreof7–8,withthe

supplier considering that they are categorizing changes

correctly 75–90 % of the time.


5.0

Results

5.1RespondentprofilesThereareanumberofwaysinwhichrespondentscanbegrouped.Forthisfirstsurveyitwasconsideredthatnocompanieshaveyetimplementedthepractices.

Figure1:16Companiesrespondedtothesurvey,eightoftheseself-identifiedasdrugsponsors,sixassuppliersofSUSassembliesandtwoas

suppliers of SUS components

Figure 1: 16 Companies responded to the survey, eight of these self-identified as drug sponsors, six as suppliers of SUS assemblies and two as suppliers of SUS components

0

1

2

3

4

5

6

7

8

9

Drug Sponsor CMO Supplier of SUSAssemblies

Supplier of SUSComponents

Supplier of SUSRaw Materials

Nu

mb

er o

f res

po

nd

ents

Respondent company profile

Survey Mar 18

5.1.1Respondentcompanytype

Atotalof16responseswerecollected,includingtwo

responsesfromonedrugsponsorandtworesponsesfrom

one SUS assembly manufacturer. One SUS assembler

responsewasconsideredanoutlierandwillbediscounted

fromanalysisanddiscussedseparately(seesection5.3.1).

NoresponseswerecollectedfromcompaniesidentifyingprimarilyasCMOsorsuppliersofSUSrawmaterials.Therewere

onlytworesponsesfromcompaniesidentifyingassuppliersofSUScomponentsandconsequentlycautionmustbetakenin

drawingconclusions.ThechartinFigure1(orasimilarchart)willbeusedaspartofadashboardtomonitoruptakeofthe

proposedchange-notificationpractices.


5.1.2Respondentroles

In general, respondents tended to be from quality roles

(QA,supplierquality).Responsesfromtechnicalservices

andcross-functionalteamswerehigheramongdrug

sponsors than other groups.

Figure2:Cross-functionalteamswereusedbythreedrugsponsors.‘Other’functionsidentifiedbysuppliersincludedregulatorysupportand

marketing/productmanagementroles.

Figure 2: Cross-functional teams were used by three drug sponsors. ‘Other’ functions identified by suppliers included regulatory support and marketing/product management roles.

0

1

2

3

4

5

6

Drug Sponsor Supplier of SUS Assemblies Supplier of SUS Components

Nu

mb

er o

f res

po

nd

ents

Supplier quality Process development

Regulatory affairs

QA

Technical Services Procurement Other Cross functional team

Figure2showsthatprocurementandregulatoryfunctionsare

underrepresented from all respondent companies. Process

developmentfunctionswerenotsurveyedatall,whichmay

reflecttheriseofMSATfunctionsastechnicalcustodiansof

chemistryandmanufacturingcontrol(CMC)activities.

Severalrespondentsineachrespondentgroupidentified

as‘Other’:

Drugsponsor–onerespondentidentifiedas‘globaltechnical

operations’andwasincludedundertechnicalservices.

SupplierofSUSassemblies–onerespondentidentified

as‘regulatorysupport’,tworespondentsidentifiedas

‘marketingoperations’.

SupplierofSUScomponents–onerespondentidentifiedas

‘productmanagement’.

This probably represents a cross-section of individuals

withinacompanywhoareengagedwithchangenotification

for single-use systems and demonstrates that a number of

stakeholderswithdiverseviewsarelikelytobeengaged

withnotifyingandreviewingchanges.


5.2QuantityofSUSchangeshandled

5.2.1MeannumberofSUSchangeshandled

Respondentswereaskedtoindicatehowmanychange

notificationsrelatingtosingle-usesystemstheyhandleon

an annual basis. The simplest snapshot of this is the mean

numberofSUSchangeshandled(reviewedornotified)by

each respondent group.

Figure3:MeannumberofSUSchangeshandledbydifferentrespondentgroups.Acleartrendisseen,withanincreaseinthenumberof

changenotificationsissued/receivedasproductsmovefromcomponentmanufacture,throughassemblytouse.

Figure 3: Mean number of SUS changes handled by different respondent groups. A clear trend is seen, with an increase in the number of change notifications issued/received as products move from component manufacture, through assembly to use.

Drug Sponsor Supplier of SUSAssemblies


Mean Changes Mar 18 182 66 4

0

20

40

60

80

100

120

140

160

180

200

Mea

n n

um

ber

of S

US

chan

ges

han

dle

d

Comparison of mean number of changes handled bydifferent respondent groups

Figure3showshowthenumberofchange

notificationsforsingle-usesystemsappearsto

increaseorpropagatethroughthesupplychainwith

significantlygreaternumbersofchangesbeinghandled

by drug sponsors than any other group surveyed.

Toanextentthismaybeexpected,sincethereisan

approximatelythree-folddifferencebetweenthe

number of changes handled by drug sponsors versus

suppliers of SUS assemblies. If a drug sponsor is using

multiplesourcing(twoorthreevendors)thenthisratio

wouldmakesense.ThehighlevelviewshowninFigure

3willbereportedaspartofthedashboardonthe

BioPhorumwebsite.

Furtherbreakdownofthesenumbersisprovidedin

section5.2.2andwillbemonitoredinthefullreport

provided for each iteration of the survey.


5.2.2Variabilityinnumberof SUSchangeshandled

Onestrikingaspectofthesurveywasthedifferencein

thenumberofchangesreviewedorissuedbycompanies.

There are several factors that should be taken into

consideration. For drug sponsors, respondents from global

functionsarelikelytoseemorechangesthanthosewith

morelocalresponsibility.Largercompanieswithabroader

product portfolio are likely to have more changes to

review.Similarly,forsuppliers,thosewithlargerormore

complexportfoliosarelikelytoissuemorechangesthan

thosewithsmallerorlesscomplexportfolios.

Figure4:Variabilityinthenumberofchangesrelatedtosingle-usesystemsreviewedbydrugsponsorsorissuedbysuppliersoverthelastyear.

Figure 4: Variability in the number of changes related to single-use systems reviewed by drug sponsors or issued by suppliers over the last year.

Drug Sponsor

Lowest 50

Mean 182

Highest 426

Supplier of SUSAssemblies

5

66

150


3

4

5

0

50

100

150

200

250

300

350

400

450

Nu

mb

er o

f SU

S ch

ange

sm

anag

ed in

last

yea

r

Variation in number of SUS changes reviewed orissued (March 2018)


Despite these factors, it appears that there is an eight-fold

differencebetweenthelowestandthehighestnumber

ofchangesreviewedbydrugsponsorsanda30-fold

differenceinthenumberofchanges(lowesttohighest)

initiated by suppliers of single-use assemblies. This is likely,

atleastinpart,tobecausedbydifferencesinwhatisand

isnotnotified/reviewedbetweencompanies.Publication

of a standardized decision tree to support categorization

ofchangesshouldgosomewaytowardsharmonizingthe

approach taken by different companies.

A very small number of component suppliers responding

to this survey and it may be that the average number of

changesissuebycomponentsuppliersissomewhathigher.

Based on the numbers reported in the survey it does seem

that a very small number of changes at the component

manufacturer level propagates to a very large number at

the drug sponsor level. This is something the team may

wishtorevisit,alongwiththenotionofacommonchange

identifiertoallowdrugsponsorstoknow/understand

whenmultiplechangespresentedfromSUSassemblers

havethesamerootatthecomponentorrawmateriallevel.

5.3QualityofSUSchangenotifications

5.3.1Outlier

One response from suppliers of SUS assemblies appeared

tobeanoutlierwithscoressignificantlylowerthanthose

provided by their peers. The objective of the survey

wastounderstandhowsuppliersscoredthemselveson

thenotificationstheyprovidetotheircustomers.Upon

discussion, it emerged that the outlier had provided

scoresbasedonthenotificationstheyreceivefromtheir

suppliersandconsequently,theoutlier’sscoreswere

removedfromanalysiswiththeSUSassemblysuppliers.In

someinstances,theoutlierresultsareshownseparately

as the scoring does reveal potential challenges that may

need to be addressed for these best practices to become

embedded and sustainable throughout the value chain.

Notably,workingwithtier1ortier2suppliersmaynotbe

sufficienttoimprovetheoverallperformance.


Figure5:Meanindustryscoreandtargetforeachaspectofchange-notificationpracticessurveyed

Figure 5: Mean industry score and target for each aspect of change-notification practices surveyed

Categorization& timing

Changedescription

effectivenessData packages

Workflow &communication

Overall score

Mar 18 3.4 3.5 3.2 3.2 3.5

Target 4.5 4.5 4.5 4.4

0

1

2

3

4

5

Change notification scorecard surveyscores: all respondents

Mar 18 Target

5.3.2Meanratingforeachaspect

Figure5showsthemeanscoreacrossallrespondentsfor

each aspect surveyed. Generally, there is a reasonable

levelofsatisfactionwiththecurrentstatus,althoughwith

a desire to improve further in all areas.

The team has focused most closely on categorization

and change description effectiveness and this is perhaps

reflectedintheresults.Analysisofvariabilitybetween

andwithinsubgroupsisshowninsubsequentsectionsand

reveals areas of focus for the team.


5.3.3Meanscoresforeachaspectsurveyedindifferentsubgroupsofthesupplychain

Although the mean scores for each aspect of change

notificationsurveyedarerelativelyhigh,thereissome

differenceinscoringbetweendrugsponsorsandthe

supply base. Generally, drug sponsors score the current

state less positively than supply partners. In part, this may

be related to drug sponsors being asked to score based

onwhattheyreceiveandsupplypartnersbeingaskedto

scorebasedonwhattheyprovide.

Figure6:Comparisonofmeanscoresbetweenrespondentgroupsforeachaspectsurveyed

Figure 6: Comparison of mean scores between respondent groups for each aspect surveyed

3.4 3.1 2.6 2.9 3.1

3.3 4.0 4.0 2.6 3.9

Drug sponsor

Supplier of SUS

Assemblies Supplierof SUS components

4.0 4.0 3.5 3.0 4.5

0

1

2

3

4

5

Mea

n s

core

Comparison of mean scores between drug sponsorand supply partner subgroups


Changedescription



effectiveness

Overallsatisfaction


What is particularly notable is a difference in scoring

betweendrugsponsorsandsupplypartnersinrelation

todatapackagequality.Thisdiscrepancywilldrive

challengesinrelationtoworkfloweffectiveness.Whatis

clearisthatinvestigatingthediscrepancyinexpectations

fordatapackageswillbeimportantinimprovingthe

generalperformanceofchangenotificationsforsingle-

use systems. Perhaps reassuringly, this is an area the

teamhadpreviouslyidentifiedbutnotyetfocusedon.

Conversely, areas such as categorisation and timing,

whichhavereceivedsignificantattention,scoremuch

more highly and much more consistently. While these

are still benchmarking data, it is an early indication that

improvementshavebeenmadeinareasonwhichtheteam

has already focused.

5.3.4Inclusionofoutlierdata

Inclusionoftheoutliermentionedpreviously,whereone

supply partner responded based on the quality of change

notificationstheyhavereceivedratherthanissued,

provides some valuable insights.

When scoring their supply base, this supplier of SUS

assemblies indicated very poor performance across all

aspects. This is based on only one data point and may not

beatruereflectionofthesituationseenbyotherSUS

assemblysuppliers.However,itprovidesanindicationthat

drugsponsorswillnotbeabletoimprovetheperformance

ofthechange-notificationsystemontheirown,anditis

likely to be necessary to engage as broad a cross-section

of the SUS supply chain as possible. It highlights the

importanceofcontinuingeffortstoengagelower-tier

suppliersinthiswayofworkingandinadoptingthetools

andpracticesdevelopedbytheworkstream.

Figure7:Comparisonofmeanscoresbetweenrespondentgroupsincludingoutlier

Figure 7: Comparison of mean scores between respondent groups including outlier

3.4 3.1 2.6 2.9 3.1

3.3 4.0 4.0 2.6 3.9

4.0 4.0 3.5 3.0 4.5

Drug sponsor

Supplier of SUS assemblies

Supplier of SUS

Components outlier 2.0 2.0 3.0 3.0 3.0

0

1

2

3

4

5

Mea

n s

core

Inclusion of outlier


Changedescription



effectiveness

Overallsatisfaction


5.3.5Variabilitywithinrespondentgroups

Aswellasdifferencesinresponsesbetweengroups,

significantdifferenceswereobservedinexperience

withingroups.Thiswasmostobviousinthedrug

sponsor grouping.

Figure8showsvariabilitywithinthedrugsponsorgroup.

Although current scoring is likely to vary due to differing

experiencesandexpectations,thereisatleastathree-

pointdifferencebetweenthelowestandhighestscorefor

eachaspectwithaseven-pointdifferencefordatapackage

qualityandworkflow/communicationeffectiveness.

Based on comments received data package quality is,

inmanycases,linkedtoworkflowandcommunication

effectivenesswithseveralrespondentscommentingthat

datapackagesmustberequestedmultipletimesand/or

are incomplete.

Itisclearthatworkremainstoharmonizefurtherdrug

sponsors’expectations,particularlyinrelationtodata

packagequality.Thoseareaswhichhavehadmorefocus

(e.g.categorizationandtiming)havemorealignmentin

termsofscoring.Itwillbeimportantfordrugsponsor

companiestomaintainengagementwiththeprocess

ofsustainingandembeddingthesechangenotification

practices.Ifexpectationsarealignedanddefinedbetween

drug sponsor companies, the likelihood of the supply chain

beingabletomeettheexpectationsincreases.

Figure8:Low,meanandhighscoresforresponsesreceivedfromdrugsponsors

Figure 8: Low, mean and high scores for responses received from drug sponsors

Figure 9: Low, mean and high scores for responses received from suppliers of SUS assemblies

0

1

2

3

4

5

Drug sponsors: current status range

Low Mean High

0

1

2

3

4

5

Suppliers of SUS assemblies: current status range


Changedescription

effectivenessData packages quality


effectiveness

Overall satisfaction


Changedescription



effectiveness


Low Mean High


Figure9:Low,meanandhighscoresforresponsesreceivedfromsuppliersofSUSassemblies

Figure 8: Low, mean and high scores for responses received from drug sponsors

Figure 9: Low, mean and high scores for responses received from suppliers of SUS assemblies

0

1

2

3

4

5

Drug sponsors: current status range

Low Mean High

0

1

2

3

4

5

Suppliers of SUS assemblies: current status range


Changedescription



effectiveness



Changedescription



effectiveness


Low Mean High

Figure9showsalignedscoringbetweensuppliersofSUS

assembliesonthequalityofthenotificationstheysupply.

Thisestablishesagoodbenchmark,butitwillbeimportant

to continue monitoring this to ensure there is adequate

feedback from drug sponsors to supply partners so they

cancontinueimprovingandaligningexpectationswith

whatisprovided.


Figure10:Low,meanandhighscoresforresponsesreceivedfromsuppliersofSUScomponents

Figure 10: Low, mean and high scores for responses received from suppliers of SUS components

0

1

2

3

4

5

Low Mean High

Suppliers of SUS components: current status range


Changedescription



effectiveness


Figure10showsgoodalignmentbetweensuppliersofSUS

components,however,itmustberememberedthatthisis

basedonsmallsamplesize(n=2).

5.3.6Rankingofperceivedchallengeandperceivedimpact

Inthefinalaspect,surveyrespondentswereaskedto

rankhowchallengingandimpactfultheybelievedeach

aspectofthechange-notificationprocesstobe.When

interpreting data, it is important to remember that

the number respondents is relatively small, especially

forsuppliersofSUScomponents.Figure11shows

howchallengingcompaniesperceivedittobetomake

improvements in each area.


Figure11:Averageperceptionofchallengeofimprovingeachareaofthechange-notificationprocess

Figure 11: Average perception of challenge of improving each area of the change-notification process

Figure 12: Average perception of impact of improving each area of the change-notification process

0.0

1.0

2.0

3.0

4.0

Val

ue

Perceived challenge to improve

Drug Sponsor

Difficulty

Supplier of SUS assemblies Supplier of SUS Components

0.0

1.0

2.0

3.0

4.0

Val

ue

Perceived impact of improving

Categorization & timing Change

descriptioneffectiveness

Data packages qualityWorkflow &

communicationeffectiveness

Difficulty





Drug Sponsor Supplier of SUS assemblies Supplier of SUS Components


Figure12:Averageperceptionofimpactofimprovingeachareaofthechange-notificationprocess

Figure 11: Average perception of challenge of improving each area of the change-notification process

Figure 12: Average perception of impact of improving each area of the change-notification process

0.0

1.0

2.0

3.0

4.0

Val

ue

Perceived challenge to improve

Drug Sponsor

Difficulty

Supplier of SUS assemblies Supplier of SUS Components

0.0

1.0

2.0

3.0

4.0

Val

ue

Perceived impact of improving





Difficulty





Drug Sponsor Supplier of SUS assemblies Supplier of SUS Components

Figure12showsresponsestothequestion‘Howimpactful

do you believe improvements to each aspect of change-

notificationpracticewouldbe?’Effectively,theaimof

thesequestionswastoidentifyareaswhichmightbeboth

easy and impactful to address.


6.0

DiscussionThesurveyprovedtobeveryusefulinrevealingdifferencesinhowchangenotificationsarehandledbetweencompaniesthroughoutthebiopharmaceuticalsupplychain.

6.1NumberofchangeshandledThere are clear differences in the number of changes

handledbydrugsponsors.Thismayreflectthedifferent

sizes of the organizations surveyed, or their current level

ofinvestmentinsingle-usetechnologies,howeverall

respondentsweremajorbiopharmaceuticalcompanies.

Similarly, there are differences in the number of changes

reported by single-use equipment manufacturers. Size of

company probably plays a role in the numbers reported

by supply partners, but there is still a three-fold difference

betweenthelargersuppliers(Figure4).

It is possible that the timing of the survey accounts for these

differences, but it seems unlikely that a three-fold difference

inthenumberofchangesreportedwouldbecausedby

externaleventsalone.Suchdifferencesinthenumbers

of changes handled indicate a need to harmonize on the

appropriate level of change reporting to avoid over- or

under-notificationofchangestosingle-usesystems.

6.2Estimationofqualitativeaspectsofsingle-usesystemchangenotificationsTheteamwasverykeentounderstandwhatwasworkingwell

andwhichareasofferedmostopportunityforimprovement

and asked companies to rate different aspects of the change-

notificationprocess,astheircompanywasexperiencingit

overall across the industry.

Figure5showstheglobalresponses,whichshowthesystemis

performingmoderatelywellacrosstheindustry,withatypical

scoreof3.Referringtothescorecardframeworkinappendix

1(asrespondentswereaskedto),thistranslatestochange

categorizationandtiming,changedescriptionandworkflow

andcommunicationeffectivenessbeingcorrect/effective

around50–75%ofthetime,asshowninTable1.Meanscores

for data package quality indicate that across the industry data

packagesareprovidedwithminorgaps.Thisglobalviewdoes

nothowevertellthewholestorywithend-userstypicallyless

satisfiedinmostareasthansupplypartners.Perhapsmost

notably, the average score for data package quality declines

from3.2to2.6whenonlydrugsponsorsareconsidered.As

thefinalcustomerinthischain,thisiscriticallyimportant,

especiallywhenthescorecard-modeloutcomeisconsidered

whichatscoresof2indicatesthattherearemajorgapsindata

packages that frequently impact implementation.

Table1: Industry mean score for each aspect and associated outcome from the scorecard model

Industrymeanscore Level3:maturing

Categorization and timing 3.4 50–75% of changes are categorized correctly according to the industry standard.

Change description

effectiveness

3.5 AchangedescriptionisprovidedthatwillallowtheEUtocompletetheRFTIA50–75%ofthetime.

Theremaybeaminornumberoffollow-uprequiredtoobtainproperdescriptions.

Data package quality 3.2 Datapackageisprovidedwithminorgapsthatareeasilyaddressedwithoutimpactingon

timelines for implementation.

Workflowand

communication

effectiveness

3.2 Notification(andwhenapplicablepre-notification)areprovidedwithsufficienttimetoproperlyimplement

50–75% of the time.

Implementation timelines are acceptable 50% of the time or more.

SPOCs are used and there is opportunity for timely and effective bidirectional feedback 50–75% of the time.


Inthesubsequentanalysiswewilllookateachaspectfor

whichaqualitativemeasurewasrequestedanddiscuss

differencesbetweendrugsponsor,supplierofSUS

assemblies,andsupplierofSUScomponentgroups.Wewill

alsolookatdifferenceswithinthedrugsponsorandsupplier

ofSUSassemblies.Wewillnotdothisforcomponent

manufacturersasthenumberofrespondentswaslow(n=2).

6.3CategorizationandtimingCategorization and timing received an average score above

3 from drug sponsors, suppliers of SUS assemblies, and

SUScomponentmanufacturers(Figure8,Figure9,Figure

10).Scoringwasalsorelativelyconsistentwithingroups.

Thisisoneoftheareasinwhichtheteamhasfocused

most, so it is good to see good consistency of scoring in

relation to categorization. It is surprising that the overall

averagescoreisslightlylowerthanforchangedescription

effectiveness,butthismayreflectthelackofprevious

guidance/systemsfortheclassificationofchanges.

6.4ChangedescriptioneffectivenessMeanscoresforchangedescriptioneffectivenesswere

alsoreasonablyconsistentbetweendifferenttiersofthe

supplychain,withaveragescoresabove3inallcases.

TherewasgoodagreementwithinthesupplierofSUS

assemblies group on the current position, but slightly more

variabilitywithinthedrugsponsorgroup.Thisprobably

reflectsthefactthatthedrugsponsorgroupwereasked

torespondonthebasisofthechangestheyreceivewhile

thesuppliersofSUSassemblieswererespondingonthe

basis of the changes they issue. Effectively, it is almost

certainly easier for the authoring party to understand the

description of a change than it is for the receiving party

as the change issuer is likely to have more background

information about the process and the change itself. There

is an opportunity to improve change communication by

diggingintohoweffectivethechangedescriptionisin

theabsenceofcontextabouttheprocessordetailthat

the supply partner holds about the change. At present,

this aspect is relatively aligned, but it may need to be

considered in future as an improvement target.

6.5DatapackagequalityDatapackagequalitywasidentifiedasthemajorarea

of focus for the team through 2018. As highlighted

previously,thiswas,inpart,duetothemeanscoreofthe

drug sponsor group indicating that typically there are

major gaps in the data packages received leading to delays

in implementation and the need to request additional

information(Figure6).Theteamhadidentifiedhigh-

quality data packages as an area that needed focus and

wouldultimatelydrivebenefitsinworkfloweffectiveness.

Whatwasinterestinghowever,wasthediscrepancyin

scoringwithintheend-usergroup(Figure8).Thisled

theteamtoaskthequestion,havewe(asdrugsponsors)

definedwhat‘good’or‘high-quality’lookslikewithregard

todatapackages?Inturn,thishasledtoasignificanteffort

betweensupplypartnersandend-userstodefinewhat

elements should typically be considered for inclusion

inadatapackageforeachtypeofchangeidentified

previouslybytheworkstreaminthe2018paperA guide

to the classification of changes to single-use biomanufacturing

systemsthecategorizationpaper.Itisexpectedthat

outputsfromthisworkwillbemadeavailableinQ2

2019. It is encouraging that this is an area the team had

previouslyidentifiedassignificant,butforwhichasolution

hadnotyetbeendeveloped.Thesignificantlylowerscores

(particularlyamongdrugsponsors)comparedtochange

categorisationdoindicatethatinareaswheretheteam

hasfocusedsignificantenergy(changecategorisation)it

has been possible to improve the overall performance of

theindustry.Oncepublisheditwillbeexcitingtoseewhat

impactdefining‘good’datapackageshasonthismetric.

6.6WorkflowandcommunicationeffectivenessPerceptionofworkflowandcommunicationeffectiveness

acrosstheindustryisrelativelyconsistentbetween

allgroups,withanaveragescorecloseto3inall

cases.Thisisreflectiveofthefactthatroughly50%

of the time communication is considered effective,

withcorrecttimelinesattributedtobothnotification

and implementation of the change. The ambition of

theindustryistomovetowardsachievingeffective

communication 90% of the time. There is some

discrepancyinscoringwithinthesupplierofSUS

assembliesgroup(Figure9)andevenmoresowithinthe

drugsponsorgroup(Figure10).Thediscrepancywithin

the drug sponsor group appears to be of roughly the

same magnitude as for data packages, and it is possible

thatthisdifferenceinviewofwhatconstitutesahigh-

qualitydatapackagemayalsobeimpactingworkflow

effectiveness. Current thinking is that improving data


packagequalitywillmakeasubstantialcontributionto

improvingworkfloweffectiveness.Twootheraspectsof

workfloweffectivenessthatneedtobeconsideredare

the use of a single point of contact and particularly the

feedbackmechanismbywhichsuppliersarenotifiedof

acceptance of the change. It is important to remember that

ineffectiveworkflowscanadverselyimpactsuppliersof

SUSequipmentassignificantlyorpossiblyevenmoreso

thandrugsponsors.AsdemandforSUSequipmentgrows,

capacity and supply chain planning becomes important. If

drugsponsorsdon’tclosetheloopandindicateacceptance

of a change it can pose challenges to SUS suppliers, in

extremecasesmeaningthatbringinganewfacilityonline

maybedelayed.Inturnthismayputpressureonexisting

facilities, potentially creating backorder situations and

ultimately posing drug sponsors challenges in supplying

therapeutics to market. Effectively these are challenges

tobusinesscontinuityplanningposedbybringingnew

products into a regulated environment, This may be one

reasonwhySUSsuppliersonaveragescoreworkflow

effectiveness less highly than drug sponsors.

6.7Rankingofthechallengeandimpactposedbyincreasingscoringineachaspectofthechange-notificationprocessFirstRespondentswerefirstaskedtorankhow

challenging they perceived improving each aspect of

thechange-notificationprocesstobe(Figure11).The

firstobservationisthatevenwithinrespondentgroups

thereislimitedagreementastowhichaspectitwould

bemostchallengingtoimprove.Ifallscoreswere

distributedequallyeachaspectwouldberankedat2.5.

More deviation is seen from this starting point among the

suppliers of SUS components, primarily driven by the small

samplesize(n=2).Thereissomeindicationthatingeneral,

improving data package quality and categorisation and

timing are seen as slightly more challenging aspects.

Change description effectiveness is generally seen as less

challenging to improve by drug sponsors and suppliers

of SUS assemblies. Wheras, although the sample size is

small, there is strong agreement among suppliers of SUS

components that the change description effectiveness

aspect is particularly challenging. It is essential to collect

datafromalargersamplesizebeforedrawinganyfirm

conclusions, but there are some early indications that

different challenges may be faced by different tiers of the

supply chain. It underlines the criticality of continuing to

build understanding and rapport through the supply chain

sothatsolutionsimplementedbytier2supplierslower

in the supply chain, ultimately address the challenges

faced by the drug sponsors. Conversely, drug sponsors

and SUS assemblers have a role to play in understanding

thechallengesfacedbylowertiersofthesupplychain,

enteringintodialoguewiththem,anddevelopingsolutions

thatworkforalltiersofthesupplychain.

Consideringhowimpactfulimprovingeachaspectof

thechange-notificationprocessmightbe(Figure12)

some trends begin to emerge. Generally, SUS assembly

manufacturers consider all aspects to be equally impactful.

This may be indicative of the different challenges faced

bydifferentrespondentsanditwillbeinterestingto

seehowthisevolves,asfurthertoolsandguidanceare

published and implemented. Interestingly, there is an

inversetrendinhowdifferenttiersofthesupplychain

rank particular aspects. Data package quality is ranked as

mostimpactfulbydrugsponsorswiththeimpactranking

decreasing through the supply chain. Conversely change

description effectiveness is seen as least impactful by

the drug sponsors and the impact ranking rises through

the supply chain. This may indicate that drug sponsors

rely heavily on critical analysis of the data provided to

support a change and less on the description of the change

provided by the supplier. This data-driven approach taken

bydrugsponsorsiswhollyconsistentwithoperationina

regulated environment. Change description effectiveness

mayhoweverhaveakeyroletoplayinsettingthecontext

of the change or in providing better understanding of the

technicalaspectsofsuppliermanufacturingprocesseswith

whichdrugsponsorsmaynotbefamiliar.Thismaybewhy

SUS component manufacturers have scored this highly.


7.0

ConclusionPerformingthissurveyhasbeenaveryusefulexercise,providingsomegreatinsightsintocurrentareasofconvergenceanddivergenceofcompanies’experienceswithchange-notificationprocessthroughoutthebiopharmaceuticalsupplychain.Importantly,sincethisisthefirsttimethesurveyhasbeenrunithasestablishedabaselineintermsofcurrentperformanceofthechange-notificationprocessesemployedbyrespondentcompanies.Ithashighlightedsignificantdifferencesinthenumberofchangenotificationsdrugsponsorsarehandlingforsingle-usesystems.Establishingthecurrentstatusofthesepracticesisimportantastheintentionoftheteamisnottoincreasethenumberofchangenotificationsreceived,butrathertoidentifyandfocusonthechangenotificationslikelytohavethemostimpact.

Thesurveyhasalsobeenveryusefulinuncoveringa‘revealedproblem’.End-usersconsiderdatapackages

tobehighlyimpactfultothechange-notificationprocess.Agooddatapackage,deliveredwiththechange

notification,providestheleanestworkflowandminimizestheneedtofollowupwithadditionalquestions

tothesupplier.Sincethedrugsponsorsaretheultimatecustomersinthissupplychain,theywilltypically

drivereworkandrequestsforadditionalinformationfromthesupplychain.Figure8showsthatthereis

alackofagreementbetweendrugsponsorcompaniesonthequalityofthedatapackagestheycurrently

receive.Reflectingonthis,theteamacknowledgedthatatthetimeofthesurvey,littleworkhadbeen

carriedouttodefinewhatconstitutes‘good’withrespecttosingle-usesystemchange-notificationdata

packages.Withoutalignmentbetweendrugsponsorsitisverydifficultforsupplypartnerstomeet

expectationsasthesemaydifferbetweendrugsponsorcompanies.Theimpactofthisontimelinesmay

beexacerbatedwhenrequestsforadditionaldataneedtobepassedonfromtier1suppliersthrough

thesupplychain.Teamagreedtoasubgroupcomposedofbothdrugsponsorsandsuppliersworking

todevelop guidance on data package recommendations.

Movingforward,thesurveyhasallowedcompaniesparticipatingintheBioPhoruminitiativetomeasure

thecurrentstatusofkeyaspectsofthechange-notificationprocessthattheirbestpracticeisintended

toaddress.Thisformsabaselineagainstwhichcontinuousimprovementeffortscanbemeasured.A

keyrevealedproblemhasbeenidentifiedpromptingthegrouptodevelopcontenttoaddressthis.The

intentionisnowtorunthissurveywithabroaderaudienceandcontinuetomeasuretheimpactthat

ourbest-practiceproposalishavingonchange-notificationsingle-usesystemswithinthebioprocessing

industry.Asmorecompaniesimplementthispractice,whenthissurveyisrepeateditwillbepossibleto

measurewhetherthegoalsofthecollaboration(>90%right-first-timechangenotifications)arebeing

metorwhetherthereisaneedtocorrectthecoursebyadaptingwaysofworkingorrevisingpublished

tools and guidance.

Scorecard survey ©BioPhorum Operations Group Ltd 30

Appendix

BioPhorumBPSAChangeNotificationScorecardFramework

Attribute 0 1 2 3 4 5

Categorizationand

timing

BioPhorum or

other industry

standards(ASTM,

USP,etc.)toassign

change categories

are not used.

Less than 25%

of changes are

categorized

correctly

according to the

industry standard.

25–50% of

changes are

categorized

correctly

according to the

industry standard.

50–75% of

changes are

categorized

correctly

according to the

industry standard.

75–90% are

categorized

correctly

according to the

industry standard.

At least 90%

are categorized

correctly

according to the

industry standard.

Changedescription

effectiveness(and

anticipatedenduser(EU)

impactassessment(IA))

A description

of the change is

not provided or

the description

provided does

notallowanEU

IA. There may

beasignificant

amountoffollow-

up required to

obtain proper

descriptions.

A change

description is

provided that

willallowtheEU

to complete the

right-first-time

(RFT)IAless

than 25% of the

time. There may

beasignificant

amountoffollow-

up required to

obtain proper

descriptions.

A change

description is

providedthatwill

allowtheEUto

complete the RFT

IA 25–50% of the

time. There may

beasignificant

amountoffollow-

up required to

obtain proper

descriptions.

A change

description is

providedthatwill

allowtheEUto

complete the RFT

IA 50–75% of the

time. There may be

a minor number of

follow-uprequired

to obtain proper

descriptions.

A change

description is

providedthatwill

allowtheEUto

complete the RFT

IA at least 75–90%

of the time.

The change

description

providessufficient

information to

allowtheEUto

complete the RFT

IA at least 90% of

the time.

Datapackagequality Data packages

are not sent

withthechange

notification,

and data is not

available.

Data package is

providedbutwith

major gaps and

reworkisrequired,

impacting

significantlyon

timelines for

implementation.

Data package is

providedwith

significantgaps

andrework

is required,

impacting on

timelines for

implementation.

Data package is

providedwith

minor gaps that are

easily addressed

withoutimpacting

on timelines for

implementation.

Data is available

withnogapsbut

itisnotalways

sentwiththe

notificationand

must be requested.

Dataisalways

available,withno

gapsandsentwith

thenotification

(rightfirsttime).

Workflow/

communication

effectiveness

Pre-notification

is not provided

(whenneeded).

The quality and

timing of the

notificationisnot

acceptable.

Timelines

for change

implementation

are not acceptable

most of the time.

SPOCs are not

used. There is

no opportunity

for bidirectional

feedback.

Notification(and

whenapplicable

pre-notification)

isprovidedwith

sufficienttime

to properly

implement less

than 25% of the

time.

Implementation

timelines are

acceptable less

than 25% of the

time.

SPOCs are not

used. There is

opportunity

for timely

and effective

bidirectional

feedback less than

25% of the time.

Notification(and

whenapplicable

pre-notification)

are provided

withsufficient

time to properly

implement less

than 25–50% of

the time.

Implementation

timelines are

acceptable 25% of

the time or more.

SPOCs are

used and there

is opportunity

for effective

bidirectional

feedback 25– 50%

of the time.

Notification(and

whenapplicable

pre-notification)

are provided

withsufficient

time to properly

implement

50–75% of the

time.

Implementation

timelines are

acceptable 50% of

the time or more.

SPOCs are

used and there

is opportunity

for timely

and effective

bidirectional

feedback 50–75%

of the time.

Notification(and

whenapplicable

pre-notification)

are provided,

withsufficient

time to properly

implement,

75–95% of the

time.

Implementation

timelines are

acceptable 75% of

the time or more.

SPOCs are used

and have timely

and effective

bidirectional

feedback 75–90%

of the time.

Pre-notification

(whenapplicable)

andnotification

are provided,

withsufficient

time to properly

implement, at least

90% of the time.

Implementation

timelines are

acceptable 90% of

the time or more.

SPOCs are used

and have timely

and effective

bidirectional

feedback at least

90% of the time.


ReferencesCarterJ,IsbergE,VogelJ,RestrepoS.2017a.Anindustryproposalforchangenotificationpracticesforsingle-use

biomanufacturing systems. Bioprocess Int. 15.

CarterJ,RestrepoS,VogelJ,IsbergE.2017b.AnIndustryProposalforChangeNotificationPracticesforSingle-Use

BiomanufacturingSystems.BioPhorum.http://bpsalliance.org/wp-content/uploads/2017/06/BPOG-BPSA-Change-

Notification-June-2017-FINAL.pdf.

WhiteT,OttK.2015.Management,notification,anddocumentationofsingle-usesystemschangeorders:Challengesand

opportunities. Bioprocess Int. 13.

Restreppo et al 2018 Aguidetotheclassificationofchangestosingle-usebiomanufacturingsystems BioPhorum.com

https://www.biophorum.com/a-guide-to-the-classification-of-changes-to-single-use-biomanufacturing-systems/


Permission to useThe contents of this report may be used unaltered as longasthecopyrightisacknowledgedappropriatelywithcorrectsourcecitation,asfollows“Entity,Author(s),Editor,Title,Location:Year”

DisclaimerThisdocumentrepresentsaconsensusview,andassuch it does not represent fully the internal policies of the contributing companies.

Neither BioPhorum nor any of the contributing companies accept any liability to any person arising from their use of this document.

Theviewsandopinionscontainedhereinarethatofthe individual authors and should not be attributed to theauthors’employers.

SINGLE-USE SYSTEMS CHANGE-NOTIFICATION SCORECARD … · 2019. 12. 15. · BioPhorum Operations...

Documents

Transcript of SINGLE-USE SYSTEMS CHANGE-NOTIFICATION SCORECARD … · 2019. 12. 15. · BioPhorum Operations...