Post on 03-Jan-2016
NIK’01, Tromsø, 26-28 Nov. 2001 1
An Empirical Study on the Utility of Formal Routines
to Transfer Knowledge and Experience
Reidar Conradi, NTNUTore Dybå, SINTEF Tele og data
conradi@idi.ntnu.no, phone +47 73.593444
Norsk Informatikk-konferanse 2001 (NIK’01)Tromsø, 26-28 Nov. 2001
NIK’01, Tromsø, 26-28 Nov. 2001 2
Contents
• Background• Research context and questions• Knowledge of routines• Use and updating of routines• Routines as a medium for transfer of
knowledge and experience• Discussion and implications• Limitations and recommendations for
future research• Conclusions and postscript
NIK’01, Tromsø, 26-28 Nov. 2001 3
Background
• Formal routines are emphasized in both quality assurance (QA) and software process improvement (SPI).
• At the same time; there is often a considerable skepticism among developers to learn from and adhere to formal routines.
• Therefore, this study was initiated to explore how developers and manager perceive formal routines as a medium to express and disseminate knowledge and experience.
NIK’01, Tromsø, 26-28 Nov. 2001 4
Research Context
SPIQ - Software Process Improvement for better Quality:• National project, funded in part by the Research Council of
Norway, 1997-99• Three research institutions and 12 IT companies, mostly
SMEs. • More than 20 SPI pilot projects were run in these companies.
Method:• Interviews with 23 persons (13 developers, 5 QA managers,
and 5 SW manager) in 5 companies participating in SPIQ (convenience sample).
• Qualitative and quantitative data collected by two last year M.Sc. students, supervised by the authors.
• Data analyzed by the students in cooperation with the authors.
NIK’01, Tromsø, 26-28 Nov. 2001 5
Research Questions1
Q1: What is the knowledge of the routines being used?
Q2: How are these routines being used?
Q3: How are they updated?
Q4: How effective are they as a medium for transfer of knowledge and experience?
1with sub-questions
NIK’01, Tromsø, 26-28 Nov. 2001 6
Software developers(n = 13)
Managers(n = 10)
Frequency Percent Frequency Percent
Little - - - -
Some 6 46 2 20
Much 7 54 8 80
Q1.1: Knowledge of Routines
About the same
NIK’01, Tromsø, 26-28 Nov. 2001 7
Q1.2: Degree of Involvement During Introduction of
Routines
Degree of involvement
Low High
Frequency Percent Frequency Percent
Developers 9 69 4 31
Managers 3 30 7 70
Most developers (9/13) had a LOW degree of involvement!!Most managers (7/10) had a HIGH degree of involvement!!
NIK’01, Tromsø, 26-28 Nov. 2001 8
Q2: Use of Routines
• In general, about 50% of the companies’ routines were in use
•The more experienced developers used the routines to a lesser extent than the more inexperienced developers do.
• It was a common agreement that:– “There is no point in having routines that are not considered
useful”•The status of the routines among the software developers
was highly divergent:– “The routines are generally good and useful, but some
developers are frustrated regarding their use.”– “The system is bureaucratic – it was better before, when we
had more freedom to decide for ourselves what should best be done.”
– “The routines are easy to use.”– “Routines are uninteresting and revision meetings are boring.”
NIK’01, Tromsø, 26-28 Nov. 2001 9
Q3: Updating of Routines
• None of the companies had scheduled revisions as part of the process for updating their routines.
• Most answers to this issue were rather vague.
• Some respondents explained that revisions were informally triggered.
• Other respondents did not know how to propose and implement changes to existing routines.
• Respondents from all of the companies said that the employees could participate in the revision activities if they wanted to.
NIK’01, Tromsø, 26-28 Nov. 2001 10
Software developers(n = 13)
Managers(n = 10)
Frequency Percent Frequency Percent
Yes - - 7 70
Both 7 54 2 20
No 6 46 1 10
Q4.1: Routines as a Medium for Transfer of Knowledge and
Experience
Again, strongly skewed, as for Q1.2.
Fisher’s exact test = 13.02 (p = 0.002, two-tailed)
NIK’01, Tromsø, 26-28 Nov. 2001 11
Q4.2: Alternative Mediafor Knowledge Transfer
Rank (1 is best) Medium Developers Managers
Experience base/newsgroups 1 1 Socialization 2 3 Discussion groups 3 2 Experience reports 4 3 On-the-job-training 5 6 Work with ext. consultants 6 - Group meetings 7 5
Similar ranking, but different perception of contents.
NIK’01, Tromsø, 26-28 Nov. 2001 12
Degree of Involvement vs. Assessment of Formal
Routines
Degree of involvement
Efficient medium? Low High
Yes - 7
Both 5 4
No 7 -
Again, strongly skewed, as for Q1.2 and Q4.1.
Fisher’s exact test = 14.71 (p < 0.0005, two-tailed)
NIK’01, Tromsø, 26-28 Nov. 2001 13
Discussion of Results
• Occupational culture– Managers and developers don’t seem to understand each
other– This leads to failure in organizational learning.
• Participation– All of the companies had a low degree of employee
involvement in the development of their routines.– Consequently, the routines were seen as coercive and not
enabling.
• Situated learning– Significant learning should not be divorced from its specific
context.– Developers viewed stories and tacit social activities (e.g.
newsgroups and discussion groups) as more flexible, adaptable and relevant than formal routines.
NIK’01, Tromsø, 26-28 Nov. 2001 14
Knowledge “Conversion”1
Externalization
Internalization Combination
Socialization
ExplicitTacitT
acit
Exp
lici
t
From
To
1Nonaka and Takeuchi (1995).
NIK’01, Tromsø, 26-28 Nov. 2001 15
Implications
• Studies of the effects of idealized formalizations should focus on their relevance for actual routines and their implementation.
• We must learn how to establish better cross-cultural dialogues between the different occupational cultures.
• Managers should balance discipline and creativity, in order to supplement formal routines with collaborative, social processes.
• Both managers and developers should maintain an open dialogue with feedback regarding the utility of formal routines.
NIK’01, Tromsø, 26-28 Nov. 2001 16
Limitations and Recommendationsfor Future Research
Validity:• The small sample and lack of randomness in the choice of
respondents may be a threat to external validity. • A major threat to internal validity is that we have not
assessed the reliability of our measures (e.g. degree of involvement)
Future studies:• Should examine the enabling features of formal routines in
much more detail; both cross-sectional and longitudinal studies.
• Should include a multiple respondent approach to cover all major occupational cultures.
• Should do supplementary, ethnographic studies on how developers really work and how their work relate to formal routines
NIK’01, Tromsø, 26-28 Nov. 2001 17
Conclusions
• Developers in general are positive to experience transfer and guidelines.
• Formal routines alone is not an efficient way to transfer knowledge and experience.
• Large differences between managers and developers regarding the perception of formal routines to transfer knowledge and experience.
• In spite of a small sample, we think that the results are representative for a large class of software companies.
• A more cooperative and open work atmosphere, with strong developer participation is needed.
• The major and most difficult work remains non-technical, that is, to build a learning software organization.
NIK’01, Tromsø, 26-28 Nov. 2001 18
Postscript• Quality assurance: under-taught in CS curricula.• Timothy C. Lethbridge:“The Relevance of Software
Education: A Survey and Some Recommendations”, Annals of Software Engineering, No. 6, 1998 identifies three under-taught topics:– Testing and Quality Assurance– Configuration Management– Process Standards
• Tor Stålhane, IDI: further interviews last summer in 15 Norwegian software companies on attitudes to formal QA systems:– Same pattern: disrespect for formal routines.– Quote:”Oh, quality assurance – that is what those
people in that other department is doing”.• So you will here more from us!