Adopting threat modelling in agile software development ...

Adopting threat modelling in agile software development projects⋆Karin Bernsmeda, Daniela Soares Cruzesa, Martin Gilje Jaatuna and Monica IovanbaSINTEF Digital, Trondheim, NorwaybVisma, Romania

ART ICLE INFOKeywords:agilesoftwarethreat modellingData Flow DiagramsSTRIDEMS-TMT

ABSTRACTThe goal of secure software engineering is to create software that keeps performing as intended, evenwhen exposed to attacks. Threat modelling is considered to be a key activity to reach this goal, buthas turned out to be challenging to implement in agile teams. This paper presents results from fourdifferent studies, in which we have investigated how agile teams do threat modelling today. Study Ais based on observations and document analysis from five teams in a single organisation, Study B isbased on interviews with eight individuals from four different organisations, Study C is based on aquestionnaire survey of 45 students at two different universities, and Study D is based on interviewswith seven teams in a single organisation, supplemented with document analysis. Our results includefindings, challenges and current good practice related to the use of Data Flow Diagrams, STRIDE andthe Microsoft Threat Modelling Tool. We also cross-check our findings with previous relevant work,and provide recommendations for making the threat modelling activities more useful to agile teams.

1. IntroductionThreat modelling has been identified as one of the most

important activities when developing secure software [1, 2,3]. The main idea behind threat modelling is to think likean attacker during the design phase. A well-defined threatmodel helps to identify threats to different assets of a systemby utilizing well-grounded assumptions on the capabilitiesof any attacker interested in attacking such a system, therebyenabling software development teams to identify critical ar-eas of design that need to be protected.

Threat modelling is a wide concept that encompasses abroad range of techniques that can be utilized to make a sys-tem more secure. Our approach, which is based on the Mi-crosoft approach to threat modelling [4], consists of threemain elements [5]:

• Asset identification, which includes identifying infor-mation and/or services that are essential or critical forthe system;

• Creating a Data Flow Diagram (DFD), which is anoverview over how assets are stored, processed or oth-erwise interact with the system and that includes sys-tem interfaces and potential attack surfaces;

• Identifying threats that may affect one or more of theassets. Threats can be identified through the use of ex-isting frameworks, such as STRIDE [3], which is anacronym for the six threat categories Spoofing iden-tity, Tampering with data, Repudiation threats, Infor-mation disclosure, Denial of service and Elevation of

⋆This work was supported by the Science of Security in Agile Soft-ware Development (SoS-Agile) project, funded by the Research Council ofNorway (grant number 247678).

[email protected] (K. Bernsmed);[email protected] (D.S. Cruzes); [email protected](M.G. Jaatun); [email protected] (M. Iovan)

ORCID(s):

privileges, and further analysed and visualised in, forexample, attack trees [6].

These activities are often performed as part of a risk assess-ment, in which the threats with unacceptably high risk aremitigated through the identification, implementation and de-ployment of suitable countermeasures in the system.

Over time, various threat modelling approaches and me-thodologies have been developed, and are nowadays a nat-ural part of the process of designing, for example, secureweb applications [7]. In agile software development, how-ever, the practice of threat modelling has not yet been widelyadopted, and the practitioners have few sources of recom-mendations on how to include such practices in their pro-cesses. Many other security practices are also challenging toadopt in agile software development. The agile principles in-trinsically generate some of the challenges for security workin agile software development, as reported in the systematicliterature review by Oueslati et al. [8]. They identify thesechallenges and evaluate their causes with respect to the agilevalues, the agile principles, and the security assurance prac-tices. Oueslati et al. [8] found that the challenges are relatedto:

• Software development lifecycle, meaning that the se-curity related activities need to be applied for each de-velopment iteration and that the iteration time is lim-ited andmay not fit time consuming security activities;

• Incremental development, meaning that changes of re-quirements and design break system security require-ments, and that continuous code changes makes com-pleting the assurance activities difficult;

• Security assurance, meaning that security assessmentfavors detailed documentation, and that tests are, ingeneral, insufficient to ensure the implementation ofsecurity requirements;

Bernsmed et al.: Preprint submitted to Elsevier of 19

Adopting threat modelling in agile software development projects

• Awareness and collaboration, meaning that securityrequirements are often neglected, and that the devel-oper role must be separate from security reviewer roleto have objective results;

• Securitymanagement, meaning that organizations com-promise on security activities to accommodate accel-erated release schedule.

Studies in software security usually focus on software se-curity activities in general, and there are few empirical stud-ies focusing on specific security practices in agile softwaredevelopment. Researchers have not yet gathered evidenceon how to best include threat modelling in a software devel-opment process when you do not have an up-front design torely on. Our research aims to build evidence on the adoptionof threat modelling in agile projects, by gathering informa-tion on the challenges and experiences in different projectsthat we have been facilitating during the last few years. Themain research problem that we address is "How canwe adaptthe threat modelling approach to better suit agile softwaredevelopment projects?". The following research questionswere thus defined for our study:

• RQ1: How are agile teams doing threat modelling to-day?

• RQ2: What are the challenges / best practices of doingData Flow Diagrams?

• RQ3: What are the challenges / best practices of ap-plying STRIDE?

• RQ4: Is theMicrosoft ThreatModelling Tool helpful?• RQ5: How can we make the results from threat mod-

elling more useful to agile teams?In this paper we present the results from the triangulation ofthe results from four different studies, in which we have col-lected and analysed empirical information by means of ob-servations, interviews and surveys (for details on data sources,collection methods and sample sizes, see Table 1). Two ofthe studies [9, 10] were previously published by the authorsof this paper and reused in the analysis of this paper. Wehave then cross-checked our results with previous relevantwork, in order to further strengthen (or reject) our findings.

The paper is organised as follows. In Section 2, we presentbackground and related work. In Section 3, we explain themethodologies that we have used to gather data and triangu-late the results from the different studies. The four studiesare presented in Section 4. Section 5 presents the results,which are then discussed in Section 6. Finally, Section 7provides our conclusions and points to future work.

2. Background and related workThreat modelling was introduced as a concept by Mi-

crosoft around the turn of the century, providing another ar-row in the quiver of the Trustworthy Computing initiative. It

was formally documented in the book by Swiderski and Sny-der [4], and included as a component in the initial release ofthe Microsoft Security Development Lifecycle (SDL) [11].Threat modelling is also part of what McGraw calls Archi-tectural Risk Analysis [2].

Existing threat modelling approaches vary from concep-tual frameworks to practical tools. Myagmar, Lee and Yur-cik describe threat modelling as a process aiming to under-stand the complexity of a system and identifying all possiblethreats to the system [12]. According to Shostack [3], threatmodelling usually employs two types of models; one thatrepresents the system that is to be built and another one thatrepresents the actual threats to the system.

Regarding tools, Microsoft has made their Threat Mod-elling Tool (MS-TMT) [13] freely available, but there havebeen some doubts whether there are plans for further ver-sions after the current one (v7). There are other threat mod-elling tools that have yet to see a wide adoption in the mar-ket, in the following we will mention a couple of examples.A recent alternative is the OWASP Threat Dragon tool [14],which has support for Windows, Linux and MacOS in ad-dition to a web app. Threat Dragon currently leaves some-thing to be desired in user-friendliness of the drawing func-tion, but supports Confidentiality, Integrity and availability(CIA) analysis and privacy threat modelling (referred to asLINDDUN) in addition to the STRIDE methodology [15].Another tool is SPARTA, presented by Sion et al. [16], whichextends STRIDE threatmodelling usingData FlowDiagrams(DFDs) with more explicit countermeasures linked to eachidentified threat. SPARTA also performs simulations to es-timate the vulnerability of a solution, taking the capabilitiesof different types of attackers into account.

Regarding previous studies, a systematic mapping studyon security in agile requirements engineering from 2018 [17]provides some insights; one of the most common ways tohandle security requirements in an agile context is to proposemodifications to the methods that are already being used, orto introduce new artifacts to the existing set of artifacts. Itis also common to introduce guidelines to handle securityissues. The mapping study also identifies three major limi-tations of the agile security requirements approaches: lack oftime (for addressing security requirements in each release);lack of security skills and awareness; and lack of guidelinesfor collecting and addressing security requirements.

Behutiye et al. [18], performed a systematic mappingstudy of the management of quality requirements in agileand rapid software development. They found that securitywas one of the most frequently referenced quality areas, andconcluded that there is a need for more (and presumablybetter) tools and lightweight management strategies. Terp-stra, Dabeva and Wang [19] studied discussions related tosecurity requirements engineering on a LinkedIn discussiongroup, and identified coping strategies related to artefacts,human factors, and the agile process itself. The latter is par-ticularly interesting for our study, hinting at the need for ex-plicitly introducing new activities (e.g., threat modelling) inthe software development process.



Heijden, Broasca and Serebrenik [20] provide results froman interview study where they identified security challengesin large-scale agile software development projects.

In 2019, Weir et al. performed an action research study,in which they performed a series of lightweight interventions(workshops facilitated by a security expert), intending to im-prove the teams’ motivation to consider security and aware-ness [21]. The results show that the participating teamsmea-surably improved their abilities to deliver secure software.

Dodson, Souppaya and Scarfone [22] present high-level,methodology-agnostic guidelines for evolving a SoftwareDe-velopment LifeCycle (SDLC) into a Secure Software Devel-opment Lifecycle (SSDL), or as they say, a secure softwaredevelopment framework (SSDF). The framework consists ofpractices split into four groups, and in the group “ProduceWell-Secured Software" (PW), we find the practice “DesignSoftware to Meet Security Requirements and Mitigate Se-curity Risks" and the task PW.1.1: “Use forms of risk mod-elling, such as threat modelling, attack modelling, or attacksurface mapping, to help assess the security risk for the soft-ware." The NIST document provides numerous referencesfor each task; but interestingly whereas BSIMM is exten-sively cited, this task only refers to secure software engineer-ing activities from the Attack modelling (AM) practice, notthe Architectural Risk Analysis (AA) practice, which sub-sumes threat modelling.

3. MethodologyThreat modelling is a practice, and the purpose of our re-

search is to better understand howwe can adapt it to better fitagile software development projects. This is a real-life prob-lem and to find an answer, we needed to collect data from theintended users of this practice. Qualitative research is usefulfor exploring and understanding how individuals, or groupsof individuals, describe the aspects of a problem that theyare facing. Quantitative research, on the other hand, is oftenused to test theories, by examining the relationship amongvariables. However, both approaches have their limitations,and the weaknesses of one approach can be compensated forby the strengths of the others [23]. To address the researchquestions in this paper, we therefore found it useful to applya mix of both.

This paper consists of a collection of results from fourdifferent studies on threat modelling, in which the problemhas been addressed from different angles. Our approach canbe characterised as mixed methods research, which by East-erbrook et al. [24] has been described as an approach whereone employs data collection and analysis techniques asso-ciated with both quantitative and qualitative data. In thispaper, we have applied a concurrent triangulation strategy,which means that we have used different methods concur-rently, in an attempt to confirm, cross-validate or corrobo-rate our findings from the four different studies. Note thattwo of the studies that have been included in this paper (re-ferred to as Study A and Study B in this paper) have alreadybeen published in other venues ([9, 10]).

Table 1 provides an overview over the data sources usedin this paper. The "Data collection" column indicates whatmethods(s) have been applied to collect the data, "Time pe-riod" indicates when the studies were executed and "Datasample" indicateswhere the participating subjects came from."Authors involved" indicates who among the authors of thispaper were involved in the studies; Karin Bernsmed (KB),Martin Gilje Jaatun (MGJ), Daniela Soarez Cruzes (DSC)and/orMonica Iovan (MI), and "Reference" indicateswhether(and where) the results have been previously published. Wealso indicate what supplementary information has been in-cluded in the paper (see Appendix).

Table 2 provides an overview over the findings from eachof the studies, and how they contribute to our research ques-tions. The table also includes an overview over what meth-ods have been used to analyse the data. In addition to thefour studies, we have also used the relevant previous workpresented in Section 2 to further support (or reject) some ofthe findings that we have derived.

The studies presented in this paper are part of a biggerproject1, which investigates how to meaningfully integratesoftware security into agile software development activities.The subjects participating in these studies are hence associ-ated with companies and universities with which we have along-term collaboration.

4. Empirical StudiesIn the following we present the four studies (A-D) that

form the empirical basis for this article. Studies A and Bare based on observations in a single organisation, and in-terviews in four different organisations, respectively. StudyC is based on a questionnaire survey of university students,and Study D is based on interviews and document analysisin a single organisation.4.1. Study A: Threat modelling in agile teams (I)

Study A is a case study which was performed in an ag-ile software organization during 2017-2018. The intentionof the study was to investigate how development teams ap-ply threat modelling in their projects. The main focus of thestudy was on the challenges that the teams are facing, butwe also wanted to find out how can we adapt the threat mod-elling approach to better fit with agile development projects.

The results from this study have already been publishedin full in the paper by Cruzes et al. [9]. This section thereforeonly reproduces the main findings from the study.4.1.1. Industrial context

The company that participated in this study is an SMEwith less than 100 employees, who are stationed in threedifferent geographical locations (Norway, Poland, and Fin-land). The company has five product teams, who designand implement applications for fleet management, real timetransport information systems and travel planning and tick-eting systems. The company products are complex in the

1https://www.sintef.no/sos-agile/



Table 1An overview over the data sources used in this paper.

Data source Data collection Time period Data sample Authorsinvolved

Supplementarymaterial

Publications

Study A Observationsand documentanalysis

2017-2018 1 organisation,5 teams

DSC, MGJ, KB Observationtemplateincluded inAppendix.

Resultspublished in [9].

Study B Interviews 2018 4 organisations,8 individuals

KB, MGJ Interview guideincluded inAppendix.

Resultspublishedin [10].

Study C Survey 2019 2 universities,45 individuals

KB, MGJ Data FlowDiagramincluded inAppendix.

Unpublishedwork.

Study D Interviews anddocumentanalysis

2020 1 organisation,7 teams

DSC, MI Interview guideincluded inAppendix.

Unpublishedwork.

Table 2An overview over the findings from each of the studies.

Data source Main topics investigated Data analysis Main types of findings Contribution to RQ(s)

Study A TM, agile Coding withMaxQDA [25] andextracting findings.

Challenges,best-practices

RQ1, RQ2, RQ3

Study B TM, agile Inductive analysis [26] ofinterview transcripts.

Challenges,best-practices

RQ1, RQ2, RQ3

Study C MS-TMT, STRIDE Numerical analysis of sur-vey in Microsoft Excel.

Usability & usefulness RQ4

Study D Agile, DFD, MS-TMT,STRIDE

Coding and analysis withMaxQDA [25].

Challenges,best-practice, usefulness

RQ1, RQ2, RQ3, RQ4,RQ5

sense that they involve hardware and software integrated so-lutions, as well as mobile and web applications.4.1.2. Data collection and analysis

Observation was our primary source of information inthis study. More specifically, we observed eight threat mod-elling sessions, each lasting around 1-2 hours. The data wasmainly collected bymeans of observation templates (see Fig-ure 4 in Appendix), which was filled out by one of the au-thors of this paper after each session. In addition to the ob-servation templates, our data sources also include the arte-facts generated during the sessions (lists of assets, DFDs, listof threats and lists of relevant risks).

In addition to the data collected from the threat mod-elling sessions, we also arranged periodic meetings with thesecurity champions in the company every other week duringsix months. We also had a number of informal discussionswith the participants who attended the threat modelling ses-sions. In this study, we coded the data using the MaxQDAqualitative data analysis tool2 in an exploratory way, so we

2https://www.maxqda.com/

could see which themes would emerge from the observa-tions. Once themes were identified, we grouped them byphases (preparation, execution and post-execution) and endedup with a total number of 21 general main challenges en-countered. We then classified these issues according to thechallenges and, finally, the researchers interpreted and dis-cussed the findings together in order to reach consensus. Wethen generalized the findings into theoretical statements anddiscussed them in accordance with the literature.4.1.3. Main findings from Study A

Here we provide a summary of the main findings of thisstudy, which are related to our research questions. For moredetails on these findings, we refer to the paper where theywere originally published [9].RQ1: How are agile teams doing threat modelling to-day? In this organisation, threat modelling was performedin dedicated sessions, attended by the security champions,the development teams and in some cases also the productowner and a representative from the systems operations de-


https://www.maxqda.com/


partment. The baseline for the activities were pre-generatedlists with prioritized assets and Data Flow Diagrams, whichwe had asked them to create before the meetings. The list ofassets and the DFDs were then discussed and updated duringthe sessions. During the sessions, they also used STRIDE togenerate lists of threats (one list per asset). The teams wereasked to document the results after the sessions had ended,but most of them did not do this. When asked why, they saidthey did not see how this kind of documentation would beuseful.RQ2: What are the challenges / best practices of do-ing Data Flow Diagrams? It was challenging to moti-vate the teams to draw the Data Flow Diagrams and we alsoobserved that it often took a long time. In most cases, itwas the security champion who had prepared the diagrambefore the session started, with help from different peoplein his/her team. Some of the security champions felt it wasoverwhelming and that it was hard to produce good draw-ings that correctly reflected the design of the systems. Someteams spent a long time (more than 10 hours) on creatingthe DFD and they mentioned it felt like they were “takingtime from development activities" and that it had a negativeimpact on their productivity.

We also observed that it was hard to set the right levelof abstraction for the DFD, and that this had a negative im-pact on the discussions. When the DFD was too high-level,the discussion got unfocused and vague. In some cases theteam chose to draw a deeper view of the DFD during themeeting, hence taking time from the other planned activities(like threat identification). In other cases the DFD was toocomplex, leading to the need to arrange more meetings tobe able to cover all aspect of the DFD when identifying thethreats.

Further, it was challenging to map the interfaces in theDFD with software and systems delivered by other teams.Some teams had problems with drawing the interfaces withother internal systems in the company because the productdevelopment teams often comprised many different prod-ucts. They were also not sure how to deal with cross-cuttingconcerns. In practice, it was difficult to gather all relevantteams in onemeeting, or to knowwho should be invited fromthe other teams.

We also observed difficulties with linking the DFD to theactual code. The teams mentioned that they actually did notknow how the systemwas really implemented, and the DFDsthat they produced therefore were tainted by the uncertaintywhether the system was actually implemented that way, es-pecially where there was a lot of legacy code involved.

The last challenge we observed was that it was difficultto maintain the DFDs. Because of the lack of focus on doc-umentation in agile teams, they did not show motivation tomaintain or have a strategy to update the DFD. The teamsdid not feel like this was an activity that they would wantto do frequently. The main impression was that they woulddo it now, and then maybe in one year or something. It wasnot easy for the teams to foresee how often they would need

to perform a new threat modelling activity for their systems.We as researchers also had problems to state clearly how todecide.

Regarding best practices, we observed that including asecurity expert for facilitating the meetings helped the dis-cussions to be more focused and also more relevant to theparticipants. Also, as we mentioned before, this companyhas adopted the role of Security Champions in each team,and these were very important players in the whole process;theywere the ones driving the drawing of theDFDs, schedul-ing the meetings, documenting the threats found, and cre-ating and following up the risks identified in the meetings.Clearly, they need to be “coached/trained” to build the skillsneeded to perform these activities, and our role as researchersdoing action research with them, influenced this process andhelped them to get the needed skills.RQ3: What are the challenges / best practices of ap-plying STRIDE? During our observations we noticed thatmany times the use of STRIDE efficiently limited the discus-sions to the threats that were related to the trust boundariesand communication channels in the system, thereby neglect-ing potential threats relevant to other parts of the product.Further, we also noticed that occasionally the team was notable to identify any STRIDE-related threats at all, whichmaylead to a false sense of security when using this approach.

Another challenge was that the list of identified threatswas in many cases too vague to be usable. Following up onthe identified threats was therefore challenging. After eachsession we asked the Security Champions in each team toformalize the list of threats into risks. However, the Secu-rity Champions did not feel completely comfortable to writethem down, and sometimes we needed to ask them manytimes about it. The meeting also did not focus much on thedecision of impact and probability of the threats or on whichmitigation actions would be done, because it would take toolong. This was a follow-up that the Security Officer had to dowith the security champions. We also asked the team to con-tribute to the documentation of the threats, but most teamsdid not prioritize this.

Finally, we observed that the list of threats often createda time concern in the development teams. More specifically,the team members were not sure when they would have timeto prioritize threats from the parts of the product that theythought had already passed the “definition of done”.

Regarding best practices, as for the DFDs, the use of asecurity expert for facilitating the meetings and the inclusionof Security Champions in the teams turned out be a verypositive aspect.RQ4: Is the Microsoft Threat Modelling Tool helpful?The teams in this organisation did not use MS-TMT. Wetherefore do not have any relevant findings related to thistopic.RQ5: How can we make the results from threat mod-elling more useful to agile teams? Our observations inthis study revealed that there is a need to clarify to the teams



the benefits of doing threat modelling in their project, whilestill acknowledging the possible impacts in time and costs,not only for the meetings themselves, but also on the timeneeded to prepare the meetings and to follow up on the out-puts of the meetings. In addition, the positive side effectsof threat modelling should be highlighted, such as: betterdocumentation of the system, awareness of security issuesfor all team members, better confidence on the way securityis addressed in the team, and better visibility of the threatsfor other stakeholders such as the product owners, managersand the executives.4.2. Study B: Threat modelling in agile teams (II)

Study B is an interview study, which was performed in2018. The study focused on threat modelling in agile teams.The intention of the study was to examine how and to whatdegree Norwegian organisations perform threat modellingas part of their software development activities, what bestpractices they use, and what challenges they face.

The results from this study has already been published infull in the paper by Bernsmed and Jaatun [10]. This sectiontherefore only reproduces the main findings from the study.4.2.1. Industrial context

The participating organisations in this study are all basedin Norway. Their core product is software, and their devel-opment teams all employ agile methods. In this section, werefer to the four organisations that we interviewed as Organ-isation A, B, C and D, respectively.

OrganisationA is a supplier of IT and electronic ticketingsystems to the public transport sector. They have around 100employees, whereof the majority are stationed at their Nor-wegian headquarters. The organisation also operates two of-fices abroad, where some of the developers are working. Weinterviewed the Security Champion of one of the develop-ment teams stationed in Norway. This particular team is re-sponsible for maintenance of the organisation’s existing soft-ware product; they rarely develop any new services.

Organisation B is a provider of digital identification andauthentication services. The company has around 30 devel-opers, who are stationed in Norway as well as abroad. Threepersons participated in the interview; the Chief InformationSecurity Officer (CISO) and two of the security managers.

Organisation C is a small start-up, which offers prod-ucts and services to help improve the digital security of theircustomers. We interviewed the person responsible for inte-grating security in the organisation’s development and de-ployment processes (who referred to himself as the ”SecDe-vOp”), who in addition also is the Chief Executive Officer(CEO) of the company.

Organisation D is a software development company thatdelivers software and related services to customers in theenergy sector all around the world. They have around 20developers, whereof ten are stationed in Norway. Three per-sons participated in the interview: a technical consultant, theSoftwareDevelopmentManager and the Chief Architect (thelatter two participated through Skype).

4.2.2. Data collection and analysisIn this study we sought to extract in-depth information

on how the participating organisations perform threat mod-elling in practice. We therefore chose a qualitative researchmethod based on relatively few informants, which enabledus to perform a rich and detailed analysis of the selectedorganisations. More specifically, we performed a multiplecase study (involving the four previously mentioned organ-isations) and we used semi-structured interviews as the mainsource of information. Interviews are awell-known and pow-erful tool for information collection in qualitative research.They give the researchers insights into the research topicfrom the interviewees’ perspective [27]. We used what isreferred to in the literature as semi-structured interviews,which are driven by open questions, have a limited degree ofstructure, and tend to focus on specific situations and expe-riences made by the interviewee [28]. The interviews wereperformed either face-to-face or over Skype. No personaldatawas recorded during the interviews. The interview guidehas been included in Figure 5 in Appendix.

Since we wanted to derive patterns from our observa-tions, rather than evaluating existing hypotheses, we usedthe inductive research approach described in the paper byThomas [26] to interpret the collected data. The paper in-cludes a systematic set of procedures for analysing qualita-tive data and explains a straightforward approach for deriv-ing findings guided by research questions.4.2.3. Main findings from Study B

Here we provide a summary of the main findings of thisstudy, which are related to our research questions. For moredetails of these findings, we refer to the paper where theywere originally published [10].RQ1: How are agile teams doing threat modelling to-day? A common factor identified in all four of the inter-viewed organisations, was the involvement of developers inthe threat modelling activities. This appeared to be a suc-cessful approach, regardless of whether the developers wereonly contributing to the TM activities (as in Organisation A),or if they didmost of thework themselves (as inOrganisationB, C and D). However, the initial step of identifying assetswas not done by developers in any of these organisations; thecommon opinion appeared to be that this is an exercise forpeople higher up the “food chain”, who have the businessperspective and understand the value of the organisation’sassets.

All four organisations relied on the use of checklists, clear-ly defined processes, routines and the like, at least to somedegree. Organisation A used them because they felt theylacked experience to identify and assess all relevant threatsthemselves. Organisation B said checklists and the like wereparticularly useful to their newly employed developers. Or-ganisation C also advocated checklists, however without fur-ther specifying why. Organisation D used guidance docu-ments for threat modelling, which they had developed inter-nally.



All four organisations used an agile software develop-ment process with short iterations and they all pointed outthe benefit of doing threat modelling activities at regulartime intervals.RQ2/RQ3: What are the challenges / best practices ofdoing Data Flow Diagrams / applying STRIDE? Lackof motivation was identified as a common challenge in threeof the four organisation. In Organisation A, the securitychampion could at first not explain why they did threat mod-elling, even though he later stated that it is good for learningpurposes and that it eventually may lead to a more secureproduct. In Organisation B, the lack of motivation amongstsome of the developers was illustrated by their tendency toskip it when they have the chance. Organisation C, on theother hand, seemed to be highly motivated, but their moti-vation was related to having the right ”mindset” rather thanon going through all the steps of the threat modelling activ-ities. On the other hand, lack of motivation was not referredto as a challenge at all by the interviewees from Organisa-tion D who, in contrast stated that all of their involved em-ployees were motivated to participate in the threat modellingsessions.

Identifying relevant threats was another challenge iden-tified in all the four organisations that we interviewed, eventhough the reason why differed. In Organisation A, the se-curity champion felt that he lacked experience and that ex-isting guidelines were too shallow. He also considered thisas a time consuming activity. In Organisation B, the CISOsuspected that relevant threats could be overlooked, becausethe developers did not understand the value of the informa-tion they processed. Organisation C’s opinion was that do-ing threat identification in accordance to STRIDE was notworth the effort. Finally, Organisation D often found it hardto know whether, for example, publicly know threats wererelevant to them or not.

That threat modelling is time-consuming was also re-ported as a challenge by the first three interviewed organi-sations. In Organisation A, they found it hard to find timeto do these activities, in addition to all their other obliga-tions. Organisation B had a “worst case” example where alot of time had been spent on these activities. Finally, asmentioned above, Organisation C considered in particularthe threat identification and analysis part to be a waste oftime. Time was not perceived as a challenge in OrganisationD though; the reason for this is most likely that they alwayshave already existing DFDs to be used as a baseline and thatthe amount of documentation that they produce is minimal.

Finally, none of the four organisations had a clear defini-tion of when they were “done” with the threat modelling ac-tivity. Organisation A stopped when they felt they had spent“sufficient time” (without being able to quantify this). Or-ganisation B had a clear definition of when the risk assess-ment was done, but not of the individual threat modellingactivities that they did during this assessment. OrganisationC, on the other hand, stated that theywill never be done; theirapproach is to think about threats constantly, at all times; a

mindset that they seem to share with Organisation D.RQ4: Is the Microsoft Threat Modelling Tool helpful?In this study, it was only Organisation A that used MS-TMTon a regular basis. They found it to be helpful for creat-ing Data Flow Diagrams and for generating an initial set ofthreats, which they then supplemented by means of OWASPTop 10 [29] and with an internally pre-generated table thatincluded relevant threat actors and their motivations.RQ5: How can we make the results from threat mod-elling more useful to agile teams? The findings showthat, even though all of the four organisations consideredthreat modelling to lead to a more secure product, they allstruggled with practical aspects of the established theory.On the bright side, our study also revealed that many thingsworked really well. In particular involving the developers inthe threat modelling activities, using checklists and clearlydefined processes and routines, and triggering the threatmod-elling activities at regular time intervals stood out as bestpractices that all the four organisations had (at least partly)adopted with good results.4.3. Study C: Threat modelling using MS-TMT

In this study, we performed a controlled experiment, ascientific method for identifying cause-effect relationships,and thus a means to “acquire general knowledge about whichtechnology (process, method, technique, language or tool) isuseful for whom to conduct which tasks in which environ-ments” [30]. The experiment was performed in the fall of2019 and included approximately 100MSc and BSc studentsfrom the Norwegian University of Science and Technology(NTNU) and the University of Stavanger (UiS). The purposeof the experience was to investigate the students’ acceptanceand usage of two different threat modelling techniques; oneof them being the Microsoft Threat Modelling Tool (MS-TMT), thereby contributing to answering RQ4: Is the Mi-crosoft Threat Modelling Tool helpful?

4.3.1. Case: Digital exam systemThe experiment focused on a scenario in which a ficti-

tious university had decided to procure a digital exam sys-tem. The digital exam system has three main types of users:students, teachers and external censors, and it supports fourmain functions: 1) Creation of exams by teachers (externalsensors are also able to help), 2) Safekeeping of exams un-til the exact time the examination begins, 3) Examination,including hand-in of completed exams, and 4) Grading ofcompleted exams by teachers and external censors. In thescenario, we also assumed that the university wanted to adda new functionality to the digital exam system: 5) Storing ofresults (grades) in a database at the university.

Before the experiment started, we had already modelledthe first four functionalities in a Data Flow Diagram (seeFigure 6 in Appendix). The task of the students participat-ing in the experiment was then to do threat modelling ofthe new envisioned functionality (i.e., storing of results inthe database at the university). More specifically, we asked



them to 1) add the new functionality to the existing DFD,and 2) identify relevant threats to this new functionality, us-ing STRIDE. The intention of these two tasks was to let thestudents get hands-on experience with two of the core activ-ities in threat modelling, within a reasonable time, withouthaving to do any particular preparations, using a real-life ex-ample.4.3.2. Experiment set-up

The experiment started with a lecture, in which the stu-dents were introduced to the basic concepts of threat mod-elling, including how to draw Data Flow Diagrams (DFD)and how to do threat identification using STRIDE. The stu-dents were then introduced to the case described above. Thestudents were divided into two groups, where one group wasasked to update the existing DFD with the new functionalityand identify the relevant threats manually, using pen and pa-per; and the other groupwas asked to do the same task, but byusing the Microsoft Threat Modelling Tool (MS-TMT tool).It should be noted that the students were not randomly as-signed into these two groups; those who had access to a lap-top PC had been asked in advance to pre-install MS-TMT,make sure that it worked, and to bring their PCs to the lec-ture. These students were then assigned to the “MS-TMTgroup". The remaining students, who either did not own orhad not brought any PCwith them, were assigned to the “penand paper group". Incidentally, these two groups turned outto be almost equal in size.

The students were encouraged to work in pairs to solvethe task, but they were not allowed to interact with otherpairs. When they were finished, they were asked to handin their results and to fill out an evaluation sheet (one perstudent).4.3.3. Data collection

Two types of data sources were generated in this exper-iment; the threat models that the students produced; i.e., theupdated DFDs and the list of threats that they had identifiedas relevant for the new functionality, and their answers in theevaluation sheet.

We received in total 45 completed evaluation sheets; 20from the students who had used the “pen and paper" ap-proach to threat modelling and 25 from the students whohad used MS-TMT. The results from this data source willbe presented in the next subsection. Unfortunately, very fewstudents chose to hand in their threat models, and we weretherefore not able to use these as a source of data in our dataanalysis. No identifiable information or any kind of personaldata was collected in this experiment.4.3.4. Evaluation of user acceptance and usage

The Technology Acceptance Model (TAM) [31] identi-fies perceived usefulness and perceived ease of use as themain variables for determining user acceptance of new tech-nologies. To measure the students’ acceptance and usageof the two different threat modelling techniques, we used amodified version of the TAM, which is commonly referredto as “TAM2" [32]. Using TAM as the starting point, TAM2

Figure 1: The modified version of TAM that was used toevaluate the user acceptance of threat modelling in Study C.

incorporates additional key determinants of TAM’s perceivedusefulness and usage intention constructs, thereby allowingone to better understand the user acceptance of the eval-uated technology. More specifically, TAM2 also includes“Subjective norm", “Image", “Job relevance", “Output qual-ity", “"Result demonstrability", “Experience" and “Volun-tariness" as additional variables to the model.

Since the subjects in this study were students, we hadto make some modifications to the questions in the originalTAM2 paper [32]. More specifically, we chose not to includethe questions in the categories “subjective norm" and “im-age", since these are specifically focusing on how the sub-jects’ opinions are influenced by colleagues that they haveprofessional relations with. We also had to adapt the word-ing of those questions that were formulated for a work envi-ronment, to better fit into the context of the university stu-dents3. The value range used in the questionnaire was 1-5where 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 =agree, and 5 = strongly agree.

In addition, to include the TAM2 attribute “Experience"in our analysis we also asked the students whether they hadany previous experience in any of the following roles: 1)developer, 2) architect, 3) tester, and/or 4) product owner.

Figure 1 provides an overview over the evaluation modelthat we used.

An overview over the questions and the average score(mean values) of the answers from the students is providedin Figure 2. The average score (mean values) of the answersfrom the students who used the MS-TMT in Study C is pro-vided in Figure 3.4.3.5. Main findings from Study C

The findings from this study are mainly related to RQ4:Is the Microsoft Threat Modelling Tool helpful?

As can be seen in Figure 2, both modelling methods re-3For example, the TAM2 question “Assuming I have access to the sys-

tem, I intend to use it", which measures the subject’s intention to use thesystem in his/her daily job, was reformulated to “I intend to use the [threatmodelling/MS-TMT] in my studies". Similarly, the TAM2 question “In myjob, usage of the system is important", which measures the relevance of thesystem to the subject’s work position, was reformulated to “In my futurejob, using [threat modelling/MS-TMT] will be important".



Figure 2: The average score (mean values) of the answers fromthe students in Study C.

ceived a very good score in terms of their perceived use-fulness, in particular w.r.t whether the students think theywill improve the security of the system that is being mod-elled (4.50 for the pen and paper students and 4.12 for theMS-TMT-students, respectively). Regarding their intentionto use, the students who had modelled using pen and paperwere slightly more positive than the students who had usedthe MS-TMT (3.70 and 3.70, compared to 2.96 and 3.16).Both modelling methods received rather low scores on theirperceived ease of use (most of the average scores are below3.00, which is indicated by the red colour in the figure).

Regarding output quality and result demonstrability, thestudents who had usedMS-TMTwere consistentlymore pos-itive (all their average scores are above 3,00, as indicated bythe green colour shading) than the students who had mod-elled using pen and paper (whose average scores are consis-

Figure 3: The average score (mean values) of the answers fromthe students who used the MS-TMT in Study C.

tently below the MS-TMT students’ average scores in thesetwo categories).

The biggest difference in average score between the dif-ferent groups of students is for the question "I find [threatmodelling/MS-TMT] to be easy to use", where the studentswho had used pen and paper were far more negative than thestudents who had used MS-TMT (average score was 2.25 forthe former and 3.20 for the latter).

Looking further into the opinions of the students whousedMS-TMT (Figure 3), there are onlyminor differences inthe average score between the students who had previous ex-perience with software development (18 persons), comparedto the students who had not previews experience in this field(7 persons). However, we observe that the students with de-veloper experience were slightly more negative in their an-swers to the questions on their intention to use the tool inthe future. They were also somewhat more negative in theiranswers to the question on the output quality of the tool, andwhether they thought the tool was clear and understandable.Finally, these students also indicated that they had felt lesspressure to participate in the exercise that the students whodid not have any previous experience.

Based on the findings presented above, it appears that theMS-TMT indeed is a very “useful" tool (cf. the questions on“Perceived usefulness"), the result from the tool is easy tounderstand and present to others (cf. the questions on “Re-sults demonstrability"), but it is difficult to use the tool (cf.the questions on “Perceived ease of use"). We note that these



are our preliminary conclusions, but we have used them asa data source when triangulating the results from our differ-ent studies in Section 5. Threats to validity and limitationsassociated with this study will be discussed in Section 6.4.4. Study D: Threat modelling in agile teams

using MS-TMTStudyD is an interview study, supplemented by data gath-

ered from document analysis, in an agile software organiza-tion, which was performed during the spring of 2020. Theintention of the study was to investigate how developmentteams apply threat modelling (TM) in their projects, whichchallenges they face and what best practices they apply. Wealso wanted to study the teams’ experiences with the Mi-crosoft Threat Modelling Tool (MS-TMT).4.4.1. Industrial context

The company that participated in this study is a softwarecompany, which simplifies and digitizes core business pro-cesses in the private and public sector. Their headquarter islocated in Norway, but they have a presence across the en-tire Nordic region, as well as in Benelux and in central andeastern Europe. The company is an amalgamation of morethan 145 individual agile companies, each with its own cul-ture and way of working. Due to this diversity, each of thesub-companies is composed of one or more self-managedagile teams, which are responsible for the security of theirown services. A centralized Product Security Team (PST)is driving the software security efforts on an overall level,by creating and enforcing a secure development lifecycle,which is a standardized way of working with security acrossthe individual teams.4.4.2. Data collection and analysis

Seven teams participated in this study. They were care-fully selected to ensure diversity, in terms of different teamsizes, different types of software products, different geogra-phic locations and different maturity of security in their pro-jects. They also had varying histories regarding their teammembers and their products; some had been acquired by thecompany while others had been developed internally. Thedata collection was performed through a two-step process.In the first step, we collected and analysed the DFDs that theselected teams had produced in their threat modelling activi-ties. The intention of this first step was to understand upfrontthe approach for the DFDs from those particular teams’ per-spectives, and to identify further questions that we wouldfocus on during the interviews.

In the second step, we interviewed the security engineersfrom these seven teams. In this organisation, the “securityengineers" are team members who are responsible for pro-moting and supporting the adoption of security activities in-side the team, without breaking the agility, continuous deliv-ery, self-management and autonomy of their projects. Thesecurity engineers were asked about how they and their teammembers integrate threat modelling in their software devel-opment activities. More specifically, we asked about thedrawing of DFDs, their usage ofMS-TMT and STRIDE, and

about their impression and opinions of these activities over-all. They were also asked to give concrete examples of chal-lenges and best practices that they had adopted when doingthreat modelling in their projects. All the interviews wereperformed over Google Meet4 and lasted about 1 hour. Theinterview guide has been included in Figure 7 in Appendix.

We then transcribed the interviews and performed opencoding of our analysis of the DFDs and of the interview tran-scripts using the MaxQDA tool2. The coding resulted in22 codes from the interview transcripts and 19 codes fromthe document analysis of the DFDs. We then organised thecodes according to the five research questions, and analyzedthem with the intention of deriving relevant findings.4.4.3. Main findings from Study D

Here we provide the main findings of this study, whichare related to our research questions.RQ1: How are agile teams doing threat modelling to-day? Drawing Data Flow Diagrams (DFDs) is part of thesecurity development lifecycle in company D, and each teamis required to update their diagrams at least once a year. Mostof the times these diagrams are created and maintained bythe security engineers, sometimes with help from other se-nior team members, such as the system architect. In thiscompany, the Microsoft Threat Modelling Tool (MS-TMT)has been selected as the preferred tool to draw the DFDs.The teams have not received dedicated training for usingthis tool, but they receive support from the centralized PST(Product Security Team) during the review of the diagrams,or whenever they have questions.

When drawing the DFDs, the teams focus mostly on con-nections and integration to systems outside the scope of theirown system. Most of the development teams draws theDFDswhile at the same time doing analysis on the connections tofind possible problems with information disclosure. Someteams also looked at possible risks related to tampering withinformation. A few of the teams also added informationabout different types of users and logging to their DFDs.

In this company, there is no a mandatory threat identifi-cation and analysis activity included in the secure develop-ment lifecycle. Even though MS-TMT provides support fordoing threat analysis using STRIDE, most teams do not usethis functionality. However, the PST has designed a verydetailed questionnaire for gathering information about thesecurity of the products and services, including questionsabout input validation, denial of service, elevation of priv-ilege and other relevant attacks. The aim is to complementthe drawing of the DFDs and to make teams more aware ofpossible threats. According to the security engineers (as thesecurity champions are denoted in the company), they usu-ally do not find threats solely based on the DFDs, but whenthey do, they create Jira tasks to investigate these further.

Regarding the “definition of done" of the drawing of theDFDs, most of the security engineers mentioned that theydecided to stop when all the connections and integration to

4https://meet.google.com/


https://meet.google.com/


other systems and services had been included in the dia-grams. In order to do a review of the diagram, some of thesecurity engineers present it to the rest of their teams anddiscuss around it to make sure nothing was missing.

Regarding the time aspect, we did not find any evidencethat the data flow diagrams had been created or updated be-fore the actual implementation of the features in the code; inmost cases they were used to documented changes to the sys-tem or software, after they had been implemented. In gen-eral, the creation of the diagrams was seen as a time consum-ing and tedious task, which required two persons spendingat least two full days to complete it, occasionally even spreadover up to three weeks. This could be the reasonwhy updatesof the diagrams usually only are done once a year in this or-ganization, the exception being some teams who do updateswhenever new interactions are added to their systems.

Before drawing of DFDs was included as a compulsorypart of the security development lifecycle, most teams didnot have any formal drawing of their systems. Some of themhad architecture diagrams; however, according to the secu-rity engineers, those diagrams could not be reused, becausethey did not include connections and interactions with othersystems.RQ2: What are the challenges / best practices of doingData Flow Diagrams? During the interviews, the secu-rity engineers raised several challenges related to the draw-ing of the DFDs. Firstly, they affirm it is challenging to iden-tify which elements to include in the DFD and how detailedthat information should be. One team commented about thelack of security knowledge to do this type of work. They alsofound it hard to know when the DFD is good enough. Forbigger systems it is even more challenging, because the taskeasily becomes time consuming, it is hard to decide where tofocus, and the drawings often become crowded. One teamdivided their DFDs into different sub-diagrams: high level(including external connections) and low level (focusing onthe internal structure).

Overall, the interviewees stated that the task is time con-suming and boring, and the teams did not seem to understandthe usefulness of this activity. Many teams also commentedabout the lack of guidelines and templates to help them get-ting started with the process. As one interviewee expressedit: “What I lacked when we were doing this was the lack ofguidelines, because no one in our team had done this be-fore. At that point it would have been good to have got a bitmore guidelines on how to structure it and what to use it for.Because if we had started off doing it correctly from the be-ginning and looking at it such as first adding a service, thenconfigure it, generate a report, before adding new services.We might have gotten tips on how to proceed. But since wedid not have any guidelines, we just started to add every-thing and then when we created the report, it was so big thatwe thought we did not not have time to do it and we gave up.If we maybe we had done in a more structured way it wouldhave been more useful." Another challenge frequently men-tioned was the need to have a senior member, or the security

engineer, involved in the drawing of the DFD, especially incases where the person(s) who had produced the code hadleft the company.

When doing document analyses of the DFDs, we noticedthat some of the teams had struggled to correctly identify andmodel the trust boundaries of their systems, especially forcloud solutions. The reason is most likely that this concept isvery different in cloud environments compared to in-housesolutions. Also, the MS-TMT does not provide much sup-port, in terms of available elements, when modelling cloudsystems.

From the interviews we identified some recurring topicsthat the teams perceived as good practices:

• The drawing of the DFD is seen as a good practice.According to the teams, including all components andconnections in such a diagram is a good way of docu-menting their systems,

• Making sure that all connections and interactions withother systems are included, is a good criterion for “Def-inition of done" for the DFD;

• Asking for help from a security expert (in the PST orfrom another team) whenever needed;

• Evaluate when to involve other team members in thediscussions;

• Using the DFDs drawings for onboarding of new em-ployees. The diagrams can also be used in customermeetings;

• Updating the DFD every time there is a new integra-tion or any other big change to the system;

• Sharing of knowledge between the teams, in particularon experiences and lessons learnt from drawing theDFDs.

RQ3: What are the challenges / best practices of apply-ing STRIDE? In this study we can conclude that STRIDEdoes not seem to be a concept that is in the mind of the Secu-rity Engineers when they do the analysis of the DFD. Whenthey were asked about STRIDE they needed reminders orexplanation of what each of the letters mean. Teams men-tioned that, in general, except when using the Threat Mod-elling Tool, they do not use STRIDE as a concept. For exam-ple when asked if they use STRIDE, one Security Engineermentioned: “Not really those terms, I do not think we usethem. We do not use STRIDE outside of data flow diagram."Some mentioned that STRIDE seems to be more theoreticalthan practical.

Because the focus of the DFDs was mostly on the con-nections, the Security Engineers could not visualize how theycan do analysis on elevation of privilege using these dia-grams. Mostly they believe that elevation of privilege isalready handled by the authentication/authorisation mech-anisms that are implemented in their systems.



RQ4: Is the Microsoft Threat Modelling Tool helpful?In this company, the Microsoft Threat Modelling Tool (MS-TMT) is the default tool used for drawing the data flow dia-gram. However, since there is only aWindows version of thetool, some teams had decided to use draw.io5 instead. Whencomparing the two products, the teams mentioned the abilityof describing the security properties of each element in MS-TMT as an advantage. On the other hand, they found MS-TMT more difficult to use than draw.io when two or morepeople needed to collaborate to draw a diagram.

Regarding usability and usefulness, the teams found iteasy to learn to use the drawing functionality of MS-TMT,however, the rest of the functionalities were not appreciated.For example, they commented about having to add all prop-erties, whereof many do not make sense to them, before theycould generate a useful threat analysis. They also reportedthat the analysis results in many false positives. In addition,one team pointed out that the tool does not work well formodelling and analysing cloud-based solutions: "Becausethe tool is not very good for cloud. Sometimes I feel like thetool does not understand what we do. Does not understandthe communication level, a lot of misleading questions aboutthe properties of the elements. When you have the cloud youhave other levels of security. And we had to answer a lotof questions about each link you make that it did not makesense or we do not have to deal with those things anymorebecause we use the cloud. The communication protocols,asks different questions that are misleading. We spent a lotof time trying to understand what everything meant."

According to the security engineers, the teams tend togive up quickly when they realise that the analysis task istime consuming. They also find it boring and they claim theydo not find any useful information by analyzing the reportthat is generated by the tool. The few relevant findings thathad been discovered by the Security Engineers were threatsrelated to insecure connections. One security engineer said:"We tried to set properties in theMicrosoft Threat ModellingTools in the communications part. But that is not complete,so we have not used the analysis part of the tool. We triedearlier and we thought it was a lot of things to look at. So ifwe had more time we could have done but I don’t know howmuch this will give us."

Regarding the drawing functionality, the teams are pleasedto have a full overview of the system, which they think ishelpful, but they also mentioned several drawbacks:

• The list of pre-defined elements lacks many of the ser-vices and components that are commonly used nowa-days (for example, in cloud-based environments);

• The list of pre-defined data flows does not include two-way communication connections, hence forcing theteams to add two separate connections for each dataflow;

• Complexity, and lack of readability, quickly become aproblem when models grow large.

5https://drawio-app.com/

• There is a lack of filtering or other visualization of theproperties, especially when they need to update thediagram.

When askedwhether theywould recommend other teamsto use the MS-TMT, most of the interviewed security engi-neers said that yes they would recommend the tool to otherteams, pointing out that the diagrams provide a good over-view of the system, and can serve as a good instrument infurther discussions about security.RQ5: How can we make the results from threat mod-elling more useful to agile teams? According to the se-curity engineers, the Data Flow Diagrams (DFDs) would bemore useful if the PST could provide better guidelines forhow to create them, for example on how to split them intoseparate diagrams to represent different levels of abstrac-tions. The teams should also be better informed on how theycould use the DFDs for security discussions, in selling pro-cesses and for onboarding of new employees. The DFDswould also be useful if the teams could use them to supportthe verification of security when migrating products to othercloud providers or micro-services architectures.

Once the product is growing in complexity and size, andmore teams start to usemicro-service architectures, one chal-lenge is the identification of relevant components to includein the DFDs. Since most of the company products are storedin cloud environments, it will be helpful if MS-TMT couldbe integrated with existing cloud systems, in order to facil-itate self-discovery of the relevant system elements. Also,in order to reduce the number of false positives in the threatanalysis report, the tool should be better on modelling thedifferent cloud protection mechanism/services. This meansthat the tool has to continuously integrate new technologiesand stay up to date with the new development approaches.

As discussed, simply just drawing the DFDs did not helpmuch in identifying threats. Neither was STRIDE consid-ered to be particularly useful. The questionnaire, on theother hand, was of great help, since it asked concrete ques-tions, such as “How do you do input validation?", “Where doyou store your logs?", etc. Such questions made it easier toidentify relevant threats, which could then be added to the is-sues tracking systems (such as Jira) for further investigationand following-up by the teams and/or the PST.

One of the reasons why the teams found threat modellingto be time consuming and tedious, was because they werenot sure if what they did was what they were supposed todo. Therefore, having an security threat modelling expert todo a walk-through with the team, where they start drawingand analyzing the diagrams, may help them understand thelevel of abstraction that best suits their system. This wouldhelp the teams improve the readability of their diagrams, andeventually also help them identify and mitigate the threatsthat are relevant for their products.

Finally, to be more useful, there should be a better inte-gration between the threat modelling activities and the de-velopment pipeline, which today in this organization is seenas two separate activities.


https://drawio-app.com/


5. Results achieved from triangulationTo better answer the research questions, we triangulated

the results from the four studies presented in Section 4.1-4.4.In this section, we will go into the details of these results. Anoverview over which studies that have contributed to whichfindings is presented in Table 3-6.5.1. How agile teams are doing threat modelling

today.Data regarding the use of threat modelling in agile teams

(RQ1) were collected in Study A, B and D. The results thatwe derived from these studies are:Finding: Asset identification is not included in the threatmodelling activities Asset identification and prioritizationis usually viewed as the first activity in a threat modellingprocess [5]. However, as was shown in Study A, B andD, it is very rare that agile software development teams gothrough this particular activity every time they do threatmod-elling. In fact, developers are often not even involved inthis part at all; for example, in Study B, one of the intervie-wees explained that the assets had already been identified onthe management level and stored in the company repository,which could then be accessed by the developers wheneverneeded.Finding: The threat modelling is done by the develop-ers. The involvement of developers in the threat modellingactivities appeared to be common practice in all of the par-ticipating organisations in Study A, B and D.Finding: The threat modelling is not done as part ofthe daily activities. Another common factor identified inStudy A, B and D, was that threat modelling was not fullyintegrated as a part of the daily activities of the teams, butrather performed as a separate activity in dedicated sessions.The collected data also indicated that threat modelling wasby many perceived as an additional activity that took timeaway from the actual development work.Finding: Threat modelling is scheduled at regular timeintervals. Doing threat modelling at regular time intervalswas a common intention amongst a majority of the partici-pants in Study A, B and D, even though it varied how wellthey managed to fulfil it in practice. The organisation partic-ipating in Study A had just got started with threat modelling,but they planned to keep doing it on a regular basis. In StudyB, the interviewees from all the four participating organisa-tions pointed out the benefit of doing threat modelling regu-larly, even though it varied how often they managed to do itin practice. In Study D, the interviewees claimed they striveto do this on a regular basis, but with varying success. Thesecure lifecycle in this organization demanded that this ac-tivity should be done once a year and therefore the teamsfollowed the demands from the procedure. Further, somementioned that they also do the activity when they feel likethey have done too many changes to the architecture of the

system.Finding: Relevant threats are found through checklistsEven though the process for identifying threats varied in thethree studies, all of the participating organisations relied onsome kind of supporting material. In Study A, they usedSTRIDE to generate lists of relevant threats. In Study B, thefour interviewed organisations all informed that they usedchecklists, guidance documents and other types of support-ing documents to identify threats. Similarly, in Study D theparticipating organisation relied on a questionnaire to findrelevant threats. The participating organisations all seemedto agree that the checklists etc. helped them identify relevantthreats.5.2. Challenges and best practices of doing Data

Flow DiagramsData regarding doing Data Flow Diagrams (RQ2) were

collected in Study A and D, and to some degree also in StudyB. The results that we derived from these studies are:Challenge: Lack of motivation. This challengewas raisedby the participants in Study A, B as well as D. Commonconcerns were that threat modelling is considered to be timeconsuming and tedious, and that it is difficult to motivatethe developers to attend the sessions. In Study A, the partic-ipants also claimed such activities takes time from the devel-opment work and hence impacts their productivity. A relatedissue that was voiced in both Study A and D, was that somethe teams do not knowwhat to dowith the results afterwards.Challenge: Level of detail to include. The challenge ofsetting the right level of abstraction when drawing the DFDswas raised as an issue by participants from both Study A andD.Challenge: Producing accurate models. How to cor-rectly model the software that they were working on, as aDFD, was raised as a challenge in both Study A and D. Is-sues mentioned included difficulties to map the models withinterfaces to other systems (Study A), challenging to linkto actual code (Study A) and problems with defining trustboundaries (Study D).Best practice: Use DFDs to document the systems. InStudy D, it was revealed that the DFD is a good way of doc-umenting the system that is being developed, and that it alsocan be used for onboarding of new employees and for cus-tomer meetings. We note that this finding partly contradictswith the previously identified challenge of lack of motiva-tion.Best practice: Involve security expert. Both Study Aand D showed that involving a security expert when drawingthe DFDs is in general a good practice.



Table 3Triangulating the results from the different studies to answer RQ1.

Research Question Result Study A Study B Study C Study D

RQ1: How are agile teamsdoing threat modelling (TM)today?

Finding: Asset identification is not in-cluded in the threat modelling activi-ties.

x x x

Finding: The threat modelling is doneby the developers.

x x x

Finding: The threat modelling is donein dedicated sessions.

x x x

Finding: Threat modelling is scheduledat regular time intervals.

x x x

Finding: Relevant threats are foundthrough checklists and the like.

x x x



RQ2 (part I): What are thechallenges of doing DataFlow Diagrams?

Challenge: Lack of motivation. x x x

Challenge: What level of detail to in-clude.

x x

Challenge: Produce accurate models. x x

RQ2 (part II): What are thebest practices of doing DataFlow Diagrams?

Best practice: Use the DFDs to docu-ment the systems.

x

Best practice: Involve a security ex-pert.

x x

5.3. Challenges and best practices of applyingSTRIDE

Data regarding the use of applying STRIDE (RQ3) wascollected in Study A and D, and to some degree also in StudyB. The results that we derived from these studies are:Challenge: Limited scope. That STRIDE has a limitedscope was identified as a challenge in both Study A and D. InStudy A, the participating organisation claimed these typesof threats have a very narrow scope, they are too vague to beusable, and that it is difficult to translate them into risks. InStudy D, the interviewees could not even relate to the term;most of them they did not even remember what the acronymstands for. In Study B, one of the organisations even claimedthat using STRIDE to identify threats was a waste of time.Best practice: Generate initial set of threats. Despiteits limitations, one of the organisations in Study A informedthey found STRIDE useful for generating an initial set ofthreats, which they could then build upon and that this wassomething they had positive experiences from.5.4. Usefulness of the MS-TMT tool

Data regarding the usefulness of the Microsoft ThreatModelling Tool (RQ4) was collected in Study A, C and D.

The results that we derived from these studies are:Finding: The tool works well for drawing DFDs. Thatthe tool works well for drawing DFDs was shown in bothStudy A, where the organisation that used the tool on a regu-lar basis explicitly said they found it useful for creating suchdiagrams, and in Study D where the interviewees told thatthe diagrams they draw by means of the tool represent a niceoverview over their systems. The tool also received goodscores on its “perceived usefulness" and “results demonstra-bility" in the student experiment (Study C).Finding: The tool is easy to use. In Study D, it wasshown that the teams found the drawing functionality easyto learn. We note however that this finding contradicts withthe results from Study C, where the tool received rather lowscores on its “perceived ease of use". However, as was shownin Study C, using the tool to model new functionality in aDFD was perceived far more easier than using pen and pa-per to do the same better exercise.Finding: The tool lacks important elements. That thetool lacks important elements, especially formodelling cloudservices, was considered a main disadvantage by the inter-viewees participating in Study D. They also pointed out thatthe tool lacks elements for modelling common mitigation





RQ3: What are thechallenges / best practices ofapplying STRIDE?

Challenge: Limited scope. x x x

Best practice: Generate initial set ofthreats.

x



RQ4: Is the Microsoft ThreatModelling Tool helpful?

Finding: The tool works well for draw-ing DFDs.

x x x

Finding: The tool is easy to use. x x

Finding: The tool lacks important ele-ments.

x

Finding: The tool generates irrelevantthreats.

x

Finding: The tool lacks support for col-laboration.

x

mechanisms, such as firewalls.Finding: The tool generates irrelevant list threats inthe report. In Study D, it was pointed out that, in additionto having a limited scope (the challenge identified in RQ3),the tool generates lots of irrelevant threats, and hence falsepositives that the teams have to deal with.Finding: The tool lacks support for collaboration. Thelacking support for collaboration was identified as a majordisadvantage in Study D. Since many of the developers inthis organisation work remotely, using different platformsand technologies, they experienced challenges sharing infor-mation, not only between different teams, but also betweenmembers in the same team.5.5. Making the results more useful to agile teams

How tomake the results from threat modellingmore use-ful to agile teams was the fifth research question defined inSection 1 (RQ5). By analysing our collected data, and cross-checking it with previous relevant research (Section 2) wehave arrived at three main recommendations:

First, the threat modelling activities need to be better in-tegrated into the agile software development life cycle. Thishas already been recognised in several previous studies [21,20, 19], and our results further confirm this conclusion. Astarting point could be to integrate those parts of the activ-ities that can be automated, such as the generation of rele-vant threats when new functionality is being modelled, andthe generation of action points (such as Jira tickets) for thosethreats that have been reviewed and tagged to be relevant. Itwould also be valuable for the developers to have access tobetter, and more updated, templates that can be used to get

them started with DFDs, with a minimal amount of effort.Second, there is clearly a need for better tooling, to help

developers draw their DFDs and to help them identify andanalyse relevant threats, but MS-TMT does not seem to bean up to date fit. Threat modelling is a collaborative task, butthe tool does not support online collaboration. Even thoughthe focus in agile is onworking software over comprehensivedocumentation, there is still value in this type of documenta-tion. Having a tool that allows the developers to easily gen-erate, update and maintain their diagrams will support theirability to adapt to changing requirements and continuous de-livery, while at the same time helping them to deliver moresecure software. Also, existing tools and techniques need tobe kept up to date with the development of new technolo-gies, in order to be and to stay relevant for agile software de-velopment teams. This was particularly visible in our datafrom Study D, where the developers explicitly requested bet-ter integration with their cloud system and existing securitycountermeasures.

Third, the findings from our study includes the involve-ment of a security expert as an identified best practice whendrawing the DFDs (see Section 8.2). Adding an expert to thedevelopment teams has also recommended in other studies,such as the one by Terpstra et al. [19]. However, one shouldbe aware this is a quick fix that may not be practically feasi-ble in all organisation and which may not help the teams inthe long run. As was shown in the study by Weir et al. [21],a developer-centric approach, in which the security expert is“sensitizing the developers to their security needs, allowingthem to choose for themselves which tools and techniques touse", i.e., raising the awareness and the competence of theteams, is a better approach than telling the developers whatto do and how to do it.



6. DiscussionThere is a conflict between an academic desire to have

a threat modelling scheme that is as accurate and detailedas possible, and developers’ wish for tools that are painless,efficient and relevant. The former approach leads to scientif-ically satisfying solutions that end up being too cumbersometo actually be used in the real world. In this sense, this is aperfect example of “best" being an enemy of “good enough",to paraphrase Voltaire. Themost difficult challenge we see ishow to make the threat modelling activity an integrated partof the agile development life cycle and changing the mindset of the development team to constantly address security,in particular during big changes that happen on the designlevel of the products. Wewould like to emphasize the impor-tance of improving threat modelling as an activity for findingsecurity problems in the design early.

Our observations show that development teams performthis activity mostly for compliance and not because they seethat it improves the security of the product, and thus there islow motivation for performing the activities early in the pro-cess. Furthermore, in dealing with this challenge, the issueof how to organize and maintain the availability of securityexperts for better analysis of the DFDs is an open questionthat needs to be further investigated.

We have also observed that following up the output of thesessions is harder than one would expect, and just creatingissues on the issue tracking system does not always work,since issues that keep being ignored or deprecated eventu-ally disappear from the issue tracking system without havingbeen satisfactorily resolved. Development teams should alsoexpect that some threats that are found will not be mitigated,but rather be subsumed as part of the accepted risks in theproduct.

The Microsoft TMT is a free resource that gives goodsupport for the drawing of the DFDs and also support to de-scribe some important characteristics of the elements for se-curity. However, one should be aware that it is time consum-ing to get all the information needed for the tool to be ableto create a useful list of threats.6.1. Recommendations for Practice

In this section, we outline the recommendations basedon the cross-study analysis of the threat modelling activity,followed by a few general implications that we derived fromthe execution of our investigations. For the companies thatare interested in threat modelling we advise:Asset Identification

• Create a list of assets [33], involving infrastructurepersonnel; one possibility is to use different discoverysystems such as the ones used in configuration man-agement databases; if using infrastructure as code, thedevelopment team can extract these assets from there;

• Create an overview of cryptographic keys and othersecrets, and specify how they are used in the system.

• Create a list of systems/services that send or receivedata from the product;

Creating the Data Flow Diagram

• The diagrams should focus not only on the integra-tion with other systems, but also on other aspects thatshould be addressed, such as users, logging, storageand trust boundaries (the latter is particularly impor-tant in cloud environments);

• The organization can create templates and guidelinesto support the drawing of the DFDs in which the mostimportant aspects of the systems are highlighted;

• The guidelines should also contain the definition ofdone for the DFDs and examples on how the teamscan create different levels of DFDs;

• Senior developers and domain experts should be in-volved in the creation of the DFDs because they knowthe history of the architectural decisions.

TMT - Threat Modelling Tool

• Companies should investigate if there are new tools inthe market that have elements that better describe thecontemporary context of development, such as beingmore suitable to cloud;

• One important functionality that TMT has is the de-scription of the characteristics of the elements tailoredto the specific type of element. Search of this func-tionality in other tools.

• If the development team does not intend to specify allthe characteristics of the various elements, nor use theautomatic threat generation functionality, any conven-tional drawing tool can be used to draw the DFDs.

Threat modelling sessions

• When possible, the whole team should participate inthe threat modelling discussions, not only seniors.

• If possible, involve a security expert who can help tosupport the discussions, focusing on the most impor-tant scenarios for security [34].

• Repeat these meetings once a year (at least), or when-ever there is a substantial change in the design of theproduct.

• Appoint a team member (e.g., a security engineer), tomake sure these meetings will happen.

• Establish a definition of done for the threat modellingsession. For example, the security expert can use STRIDEas a way to define when the discussion is done, for ex-amplewhen all the letters in the acronym are discussedthen the meeting is done.



Output of the sessions

• The identified threats and issues from the modellingsession should have a responsible assigned by the ofthe meeting, preferably more than one team membershould be assigned to these issues for follow up.

6.2. Threats to validity and limitationsAs any empirical study, this study has limitations. In

the following, we discuss the reliability of our results alongwith threats to internal, construct and external validity, asdescribed by Runeson and Höst [35].

Reliability is related to the repeatability of a study, i.e.,how dependent are the data and analysis on the involvedresearchers [35]. To minimize this threat, four researcherswere involved in the design and execution of this multi-casestudy. It is well known that ‘what people say’ is often dif-ferent than ‘what people do’. The main benefit from thisapproach is therefore that by triangulating using evidenceconverging from different data sources, one will improve thevalidity of the final result [23, 24]. Furthermore, we devel-oped an explicit case study protocol to guide the investiga-tions on each study. Finally, the observations and findingswere verified with the companies’ representatives to avoidfalse interpretations and inconsistencies. Besides, the re-searchers have a long-term relationship with the case com-panies, which means that the researchers know more aboutthe context of the companies, not basing the findings andinterpretations only on the focused interviews.

Internal validity is related to factors that researchers areunaware of or cannot control regarding their effect on thevariables under investigation [35]. The main internal valid-ity threats related to this paper are the triangulation of theresults from independent studies, and the fact that for StudyA and B we used the scientific publications [9] and [10] toderive the findings, meaning that we did not go back to theprimary source of information (the original interview tran-scripts), although we had been involved in all the studies thatwere reported. Investigator bias was mitigated by involv-ing three researchers during the design of the interview andworkshop guides (investigator triangulation).

Construct validity reflects how well the measures usedactually represent the constructs the study intends to mea-sure [35]. The main threat to construct validity in our inves-tigation is that we used different methods to investigate thephenomena. This strengthens the results we have because itshows the phenomena from different perspectives. For someaspects we could show that independent of the method, wealways got to the same conclusions.

External validity is concerned with the generalization ofthe findings [35]. The main findings are strongly bound bythe context of the selected cases. To mitigate this threat, weconducted four studies, three in companies and one in an ed-ucational environment in which we could control some ofthe context in the studies. Furthermore, our main contribu-tion lies in the cross-study comparisons based on which weconcluded about findings on threat modelling. As such, itshall be valid despite the limitations of the cases. One limi-

tation in study C is the fact that the study is performed withstudents, and the validity of using students as subjects in asimplified example in a university setting has been discussedby Falessi et al. [36], Salman et al. [37] and Svahnberg etal. [38]. The consensus in these studies seems to be thatstudents are reasonable proxies for young developers start-ing out in the industry; to mitigate this possible age bias, wehave triangulated the results with other studies and focusedon the evaluation of the MS-TMT. The main contributionsof this paper may be of interest and applicable to researchersand practitioners that work in similar contexts. To allow thetransferability of the findings of this work, we detailed thedescription of the investigated cases, within the limits im-posed by the associated non-disclosure agreements.

7. ConclusionIntroducing software security activities in an agile de-

velopment life cycle is not an easy task. The results pre-sented in this paper contribute to the body of knowledge inhow threat modelling activities are being applied in the ag-ile context and what can be done to ease the process. Morespecifically, we have investigated and documented a numberof findings, challenges and best practices related to the useof Data Flow Diagrams, STRIDE and the Microsoft ThreatModelling Tool. The approach taken in our studies is to ob-serve, interview and survey, in order to gain a solid under-standing of how agile software development teams organisetheir daily work, so that we can base our recommendationson these. Study A is based on observations and documentanalysis from five teams in a single organisation, Study B isbased on interviews with eight individuals from four differ-ent organisations, Study C is based on a questionnaire sur-vey of 45 students at two different universities, and Study Dis based on interviews with seven teams in a single organi-sation, supplemented with document analysis. Based on theresults, we bring forward a list of recommendations that cansupport companies to improve their threat modelling strate-gies, as well as some implications for future research on thistopic. Further, we have proposed three approaches to makethe results from threat modelling more useful to agile teams.The results from our studies show that, while most of theconcepts of threat modelling have already been by adoptedby teams participating in our studies, they struggle with thepractical aspects of transforming these concepts into prac-tice.

Finally, we put forward the following future research di-rections:

• More empirical studies investigating how to integratethreat modelling in agile projects.

• Further development of tools for threat modelling thatincorporate the trends in technology that the softwareproducts are being developed upon.

• Further development of templates for better drawingsof DFDs.



• Studies that evaluate the effectiveness of threat mod-elling strategies for agile development, improving therelevance and the impact of the threat modelling ac-tivities for security.

References[1] C. Jeffries, Threat modeling and agile development practices (2012).

URL https://technet.microsoft.com/en-us/security/hh855044.aspx[2] G. McGraw, Software Security: Building Security In, Addison-

Wesley, 2006.[3] A. Shostack, Threat Modeling: Designing for Security, Wiley, 2014.[4] F. Swiderski, W. Snyder, Threat Modeling, Microsoft Press, Red-

mond, WA, 2004.[5] M. G. Jaatun, K. Bernsmed, D. S. Cruzes, I. A. Tøndel, Threat mod-

eling in agile software development, in: M. Felderer, R. Scandariato(Eds.), Exploring Security in Software Architecture and Design, IGIGlobal, 2019, pp. 1–14.

[6] B. Schneier, Attack trees, Dr. Dobb’s journal 24 (12) (1999) 21–29.[7] OWASP, Threat modeling (2020).

URL https://owasp.org/www-community/Threat_Modeling[8] H. Oueslati, M.M. Rahman, L. ben Othmane, Literature review of the

challenges of developing secure software using the agile approach, in:2015 10th International Conference on Availability, Reliability andSecurity, IEEE, 2015, pp. 540–547.

[9] D. S. Cruzes, M. G. Jaatun, K. Bernsmed, I. A. Tøndel, Challengesand experiences with applying Microsoft threat modeling in agile de-velopment projects, in: 2018 25th Australasian Software EngineeringConference (ASWEC), 2018, pp. 111–120.

[10] K. Bernsmed, M. G. Jaatun, Threat modelling and agile software de-velopment: Identified practice in four norwegian organisations, in:2019 International Conference on Cyber Security and Protection ofDigital Services (Cyber Security), 2019, pp. 1–8.

[11] M. Howard, S. Lipner, The Security Development Lifecycle, Mi-crosoft Press, Redmond, WA, 2006.

[12] S. Myagmar, A. J. Lee, W. Yurcik, Threat modeling as a basis forsecurity requirements, in: Symposium on requirements engineeringfor information security (SREIS), Vol. 2005, Citeseer, 2005, pp. 1–8.

[13] Microsoft, Microsoft security development lifecycle threat modelling(2017).URL https://www.microsoft.com/en-us/securityengineering/sdl/threatmodeling

[14] OWASP, Threat dragon (2020).URL https://threatdragon.org/

[15] E. Bygdås, L. A. Jaatun, S. B. Antonsen, A. Ringen, E. Eiring, Eval-uating threat modeling tools: Microsoft TMT versus OWASP ThreatDragon, in: Cyber Science 2021 – CyberSA for Trustworthy andTransparent Artificial Intelligence (AI), 2021, pp. 195–201.

[16] L. Sion, D. Van Landuyt, K. Yskout, W. Joosen, Sparta: Securityprivacy architecture through risk-driven threat assessment, in: 2018IEEE International Conference on Software Architecture Companion(ICSA-C), 2018, pp. 89–92.

[17] H. Villamizar, M. Kalinowski, M. Viana, D. Méndez Fernández,A systematic mapping study on security in agile requirements en-gineering, in: 2018 44th Euromicro Conference on Software En-gineering and Advanced Applications (SEAA), 2018, pp. 454–461.doi:10.1109/SEAA.2018.00080.

[18] W. Behutiye, P. Karhapää, L. López, X. Burgués, S. Martínez-Fernández, A. M. Vollmer, P. Rodríguez, X. Franch, M. Oivo,Management of quality requirements in agile and rapidsoftware development: A systematic mapping study, In-formation and Software Technology 123 (2020) 106225.doi:https://doi.org/10.1016/j.infsof.2019.106225.URL http://www.sciencedirect.com/science/article/pii/S095058491930240X

[19] E. Terpstra, M. Daneva, C. Wang, Agile practitioners’ understandingof security requirements: Insights from a grounded theory analysis,

in: 2017 IEEE 25th International Requirements Engineering Confer-ence Workshops (REW), 2017, pp. 439–442.

[20] A. Heijden, C. Broasca, A. Serebrenik, An empirical perspective onsecurity challenges in large-scale agile software development, in: Pro-ceedings of the 12th ACM/IEEE International Symposium on Empir-ical Software Engineering and Measurement, 2018, pp. 1–4. doi:

10.1145/3239235.3267426.[21] C. Weir, I. Becker, J. Noble, L. Blair, A. Sasse, A. Rashid, Interven-

tions for software security: Creating a lightweight program of assur-ance techniques for developers, in: 2019 IEEE/ACM 41st Interna-tional Conference on Software Engineering: Software Engineering inPractice (ICSE-SEIP), 2019, pp. 41–50.

[22] D. Dodson, M. Souppaya, K. Scarfone, Mitigating the risk of softwarevulnerabilities by adopting a secure software development frame-work (SSDF), NIST whitepaper (2020). doi:https://doi.org/10.6028/NIST.CSWP.04232020.URL https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.04232020.pdf

[23] J. W. Creswell, Research Design: Qualitative, quantitative and mixedmethods approaches, 4th Edition, Sage Publications Limited, 2014.

[24] S. Easterbrook, J. Singer, M.-A. Storey, D. Damian, Selecting empiri-cal methods for software engineering research, in: F. Shull, J. Singer,D. I. K. Sjøberg (Eds.), Guide to Advanced Empirical Software Engi-neering, Springer London, London, 2008, pp. 285–311. doi:10.1007/978-1-84800-044-5_11.URL https://doi.org/10.1007/978-1-84800-044-5_11

[25] VERBI Software GmbH, MAXQDA All-In-One Tool for QualitativeData Analysis & Mixed Methods (2021).URL https://www.maxqda.com/

[26] D. R. Thomas, A general inductive approach for analyzing qualitativeevaluation data, American journal of evaluation 27 (2) (2006) 237–246.

[27] M. Myers, M. Newman, The qualitative interview in IS research:Examining the craft, Information and Organization 17 (2007) 2–26.doi:10.1016/j.infoandorg.2006.11.001.

[28] C. Cassell, G. Symon, Essential Guide to Qualitative Methods in Or-ganizational Research, Sage Publications Limited, 2004.

[29] OWASP, Owasp top ten (2017).URL https://owasp.org/www-project-top-ten/

[30] D. Sjøberg, J. Hannay, O. Hansen, V. Kampenes, A. Karahasanovic,N.-K. Liborg, A. Rekdal, A survey of controlled experiments in soft-ware engineering, Software Engineering, IEEE Transactions on 31(2005) 733– 753. doi:10.1109/TSE.2005.97.

[31] M. Chuttur, Overview of the technology acceptance model: Origins,developments and future directions, Sprouts: Working Papers on In-formation Systems 9 (01 2009).

[32] V. Venkatesh, F. D. Davis, A theoretical extension of the technologyacceptancemodel: Four longitudinal field studies, Manage. Sci. 46 (2)(2000) 186–204.

[33] M. G. Jaatun, I. A. Tøndel, Covering your assets in software engi-neering, in: The Third International Conference on Availability, Reli-ability and Security (ARES 2008), Barcelona, Spain, 2008, pp. 1172–1179.

[34] G. Boström, J. Wäyrynen, M. Bodén, K. Beznosov, P. Kruchten, Ex-tending XP practices to support security requirements engineering,in: Proceedings of the 2006 international workshop on Software en-gineering for secure systems, ACM, 2006, pp. 11–18.

[35] P. Runeson, M. Höst, A. Rainer, B. Regnell, Case study research insoftware engineering: Guidelines and examples, John Wiley & Sons,2012.

[36] D. Falessi, N. Juristo, C.Wohlin, B. Turhan, J. Münch, A. Jedlitschka,M. Oivo, Empirical software engineering experts on the use of stu-dents and professionals in experiments, Empirical Software Engineer-ing (06 2017). doi:10.1007/s10664-017-9523-3.

[37] I. Salman, A. T. Misirli, N. Juristo, Are students representativesof professionals in software engineering experiments?, in: 2015IEEE/ACM 37th IEEE International Conference on Software Engi-neering, Vol. 1, 2015, pp. 666–676.

[38] M. Svahnberg, A. Aurum, C. Wohlin, Using students as subjects - an


https://technet.microsoft.com/en-us/security/hh855044.aspx

https://technet.microsoft.com/en-us/security/hh855044.aspx

https://owasp.org/www-community/Threat_Modeling

https://owasp.org/www-community/Threat_Modeling

https://www.microsoft.com/en-us/securityengineering/sdl/threatmodeling



https://threatdragon.org/

https://threatdragon.org/

https://doi.org/10.1109/SEAA.2018.00080

http://www.sciencedirect.com/science/article/pii/S095058491930240X


https://doi.org/https://doi.org/10.1016/j.infsof.2019.106225



https://doi.org/10.1145/3239235.3267426

https://doi.org/10.1145/3239235.3267426

https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.04232020.pdf



https://doi.org/https://doi.org/10.6028/NIST.CSWP.04232020

https://doi.org/https://doi.org/10.6028/NIST.CSWP.04232020


https://doi.org/10.1007/978-1-84800-044-5_11

https://doi.org/10.1007/978-1-84800-044-5_11

https://doi.org/10.1007/978-1-84800-044-5_11

https://doi.org/10.1007/978-1-84800-044-5_11

https://doi.org/10.1007/978-1-84800-044-5_11




https://doi.org/10.1016/j.infoandorg.2006.11.001

https://owasp.org/www-project-top-ten/

https://owasp.org/www-project-top-ten/

https://doi.org/10.1109/TSE.2005.97

https://doi.org/10.1007/s10664-017-9523-3


empirical evaluation, in: ESEM’08: Proceedings of the 2008 ACM-IEEE International Symposium on Empirical Software Engineeringand Measurement, 2008, pp. 288–290. doi:10.1145/1414004.1414055.

Appendix: Supplementary material

Figure 4: Study A: The observation template.


https://doi.org/10.1145/1414004.1414055


Figure 5: Study B: The interview guide.



Figure 6: Study C: The MS-TMT Data Flow Diagram that was used in the experiment.

Figure 7: Study D: The interview guide.


Adopting threat modelling in agile software development ...

Documents

Transcript of Adopting threat modelling in agile software development ...