RESEARCH ARTICLE Open Access

Assessment of medical professionalismusing the Professionalism Mini EvaluationExercise (P-MEX) in a multi-ethnic society: aDelphi studyWarren Fong1,2,3* , Yu Heng Kwan4, Sungwon Yoon4, Jie Kie Phang1, Julian Thumboo1,3,4,Ying Ying Leung1,2,3 and Swee Cheng Ng1,2

Abstract

Background: The importance of medical professionalism and its assessment has been well documented in theliterature. However, there is currently no culturally-adapted tool to assess medical professionalism in Singapore. Thisstudy sets out to find consensus on relevance of the items from the Professionalism Mini Evaluation Exercise (P-MEX) for assessing medical professionalism in Singapore.

Methods: A two-round Delphi survey was completed by an expert panel consisting of program directors, associatedesignated institutional officials, and designated institutional official (n = 15) from residency programs in Singapore.Round 1 comprised of 23 items from the P-MEX rated on a 5-point scale (1 = Definitely include, 2 = Possibly include,3 = Neutral, 4 = Possibly exclude, 5 = Definitely exclude). In round 2, the experts received feedback from theprevious round, and were asked to re-rate the items which did not achieve consensus in the previous round. Thethreshold for consensus in the study was defined as 70% or greater agreement among experts.

Results: Completed questionnaires for both rounds were received from all 15 experts. In round 1, 18 items (78%)achieved consensus to be included. In round 2, 1 (4%) item achieved consensus to be included. However, none ofthe remaining items achieved consensus to be removed, and they exhibited stability in responses. A list of 19 itemscovering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Timemanagement and Inter-professional relationship skills) was obtained during the two-rounds of Delphi survey.

Conclusions: Nineteen items in the P-MEX had garnered consensus and is suitable for assessing medicalprofessionalism in Singapore. The findings of this study can provide guidance for faculty and institutions who wantto assess medical professionalism of their medical trainees.

Keywords: Professionalism, Singapore, Delphi, Assessment

© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you giveappropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate ifchanges were made. The images or other third party material in this article are included in the article's Creative Commonslicence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commonslicence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtainpermission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to thedata made available in this article, unless otherwise stated in a credit line to the data.

* Correspondence: warren.fong.w.s@singhealth.com.sg1Department of Rheumatology and Immunology, Singapore GeneralHospital, 20 College Road, Singapore 169856, Singapore2Duke-NUS Medical School, Singapore, SingaporeFull list of author information is available at the end of the article

Fong et al. BMC Medical Education (2020) 20:225 https://doi.org/10.1186/s12909-020-02147-9

IntroductionMedical professionalism has been shown to affect doc-tors’ relationships with their patients, quality of care,and ultimately health and illness outcomes [1]. In recentyears, medical professionalism has been increasingly em-phasized in medical undergraduate and post-graduatecurricula [2–5]. Previously ‘good medical practice’ hasbeen defined more broadly in terms of roles such as‘professional’ and ‘healer’, with various medical councils,academic and professional bodies having produced cleardocumentations on these roles [6–8]. For residents intraining, unprofessional behaviour during their trainingcorrelates with an increased risk of disciplinary actionlater on in their careers as physicians [9]. Fortunately,medical professionalism can be nurtured [10], but inorder for this to take place, it has to be adequatelyassessed [11]. One of the dominant frameworks of med-ical professionalism is that professionalism consists of aset of behaviours and competencies that can be masteredby the physician; and these behaviours and competenciescan be assessed [12]. We had previously performed asystematic review of the quality and utility of observer-based assessment tools that could be used in residencyprograms and had identified the Professionalism Mini-Evaluation Exercise (P-MEX) as one of the assessmenttools that could be useful in the assessment of medicalprofessionalism in our residency programs [13]. How-ever, Chandratilake et al. and Jha et al. have demon-strated differences in understanding of professionalismin physicians from different regions [14, 15]. Moreover,according to consensus statement and recommendationsfrom the Ottawa conference, the assessment of profes-sionalism varies across different cultures and hencecross-cultural validation of the assessment tool is im-portant [16]. The P-MEX was originally developed inCanada by Cruess et al [17], and when the P-MEX waspiloted in Japan [18] and Finland [19], additionalculturally-relevant items were added.Since the Ottawa report in 2011 [20], studies have ex-

plored the assessment of medical professionalism in vari-ous non-Anglo-Saxon/ Western contexts such as Korea,Japan and China [21, 22]. However, till date there hasnot been a study looking at the assessment of medicalprofessionalism in a multi-ethnic Asian context such asSingapore. Singapore’s multi-ethnic landscape comprisesof predominantly Chinese (74%), followed by Malay(13%), and Indian (9%) [23]. The importance of culturalperspectives and its effect on medical professionalismhas previously been highlighted by Jha et al [15]. The in-tercultural development continuum [24] and cultural fittheory [25] both highlight that professional behaviourbetween healthcare professionals can potential be influ-enced by the shared values and societal culture, and thisis because professional behaviours and values are socially

constructed [26]. Based on Hofstede’s cultural dimensiontheory, national culture consists of 6 dimensions: powerdistance, individualism versus collectivism, masculinityversus femininity, uncertainty avoidance, and long-termorientation versus short term normative orientation [27].Even among Asian countries, significant cultural differ-ence across these dimensions may exist [28].Our previous qualitative study with patients (health-

care recipients) and faculty (medical educators) hadidentified 23 items to assess medical professionalism,covering four domains of medical professionalism (Doc-tor-patient relationship skills, Reflective skills, Timemanagement and Inter-professional relationship skills)[29]. The aim of this study was to gain consensus amongthe stakeholders in residency programs on the items tobe used to assess medical professionalism in a multi-ethnic and multi-cultural Asian context. These itemscan form the basis for an assessment tool for the assess-ment of medical professionalism within residencyprograms.

MethodsDesignA modified Delphi technique with two iterative roundswas employed. To ensure strong retention of expert in-volvement, an upper limit of two rounds of investigationwas set in this study [8]. It is also acknowledged thathaving a planned number of rounds is an indicator ofgood quality in designing a Delphi study [13].

ParticipantsSingHealth is Singapore’s largest group of public health-care institutions, consisting of four public hospitals, fivenational specialty centres and a network of communityhospitals and polyclinics [30]. There are over 1600 fac-ulty and more than 900 residents in training.Fifteen experts from the SingHealth residency pro-

grams were selected according to age, gender, ethnicityand disciplines. The experts selected had at least 3 yearsof experience in the residency program. They also hadto be in involved in the disciplinary committees set upto investigate disciplinary issues related to residents, aswell as be involved in the training and assessment of res-idents, especially in the area of medical professionalism.Each potential expert was sent an invitation email intro-ducing the study objectives and the study procedures.Depending on the expert’s preference, either hardcopyor electronic questionnaire was provided upon agree-ment to participate in the study.

Round 1The first questionnaire contained 23 observer-baseditems assessing medical professionalism based on the re-sults of the previous survey and qualitative study (Fig. 1)

Fong et al. BMC Medical Education (2020) 20:225 Page 2 of 9

[29]. Two new items which emerged from the qualitativestudy [30]-(1) communicated effectively with patient and(2) demonstrated collegiality were included in this Del-phi survey in addition to the original 21 items. The ex-perts were asked to rate the level of agreement witheach item as assessment of medical professionalism on a5-point scale scored as follows: 1 = Definitely include,2 = Possibly include, 3 = Neutral, 4 = Possibly exclude,5 = Definitely exclude. The expert was also invited toprovide any comment on each item.

Round 2The authors reviewed the consensus rating and feedbackfrom Round 1. Respondents to round 1 were contactedand provided with the group and individual ratings fromround 1, as well as any feedback obtained from the ex-perts. The experts were asked to re-rate the items withless than 70% agreement rate in round 1. Similar toround 1, a 5-point scale (1 = Definitely include, 2 = Pos-sibly include, 3 = Neutral, 4 = Possibly exclude, 5 =

Definitely exclude) was used. The expert was also invitedto provide any comment on each item.

AnalysisConsensus was deemed to be achieved when 70% of theexperts chose to include (definitely include and possiblyinclude) or exclude (definitely exclude and possibly ex-clude) the item. There is no universally accepted thresh-old for defining consensus as part of the Delphi process,with thresholds for consensus ranging from 55 to 100%in the published literature [31]. A predefined consensuslevel is an indicator of good quality Delphi research [32]and the consensus level is influenced by the study aims[33]. A 70% threshold was considered appropriate forthis study and is consistent with other research using amodified Delphi technique [34, 35]. It was decided apriori that items with no consensus in the two rounds ofDelphi would be included in the list of items to be usedfor pilot assessment of medical professionalism in resi-dency program. Stability of responses was determined

Fig. 1 Domains and subdomains of medical professionalism

Fong et al. BMC Medical Education (2020) 20:225 Page 3 of 9

using Wilcoxon signed-rank test. Data were analyzed withStata SE15.0 (Stata-Corp, College Station, TX, USA).

EthicsThe SingHealth Centralized Institutional Review Boardapproved this study (Reference Number: 2016/3009).We obtained informed consent, which conformed to theprinciple outlined in the 1964 Declaration of Helsinki,from all the experts before the commencement of study.

ResultsA total of 15 experts participated in this study (67%male, median age 45 years (37 to 66 years)), consisting of11 program directors, three associate designated institu-tional officials (ADIO) and one designated institutionalofficial (DIO). Around half of the faculty members camefrom medical disciplines, the rest spread across a widespectrum of disciplines (surgical, diagnostic radiology,nuclear medicine and pathology, emergency medicine,pediatrics) (Table 1). All 15 experts participated in bothrounds (response rate = 100%). Table 2 and Fig. 2 illus-trate the summary of the results.

Round 1Completed questionnaires were received from all 15 ex-perts. In round 1, 18 items (78%) achieved consensus forinclusion into the modified P-MEX tool. They included:listened actively to patient, showed interest in patient asa person, recognized and met patient needs, ensuredcontinuity of patient care, maintained appropriateboundaries, communicated effectively with patient, dem-onstrated awareness of limitations, admitted errors oromissions, accepted feedback, maintained composure,was on time, completed tasks in a reliable fashion, wasavailable to colleagues, maintained appropriate appear-ance, demonstrated respect for colleagues, avoided de-rogatory language, maintained patient confidentiality,

demonstrated collegiality. The five remaining items (soli-cited feedback, advocated on behalf of a patient, ex-tended his/herself to meet patient needs, used healthresources appropriately, addressed own gaps in know-ledge and skills) did not reach consensus for either in-clusion or exclusion (Table 3).

Round 2Completed questionnaires were received from all 15 ex-perts. In round 2, 1 (4%) item (addressed own gaps inknowledge and skills) achieved consensus to be included.The 4 remaining items (solicited feedback, advocated onbehalf of a patient, extended his/herself to meet patientneeds, used health resources appropriately) did not reachconsensus for either inclusion or exclusion (Table 4).

ConsensusA total of 19 items from the two rounds of the Delphisurvey covering four domains of medical professionalism(Doctor-patient relationship skills, Reflective skills, Timemanagement and Inter-professional relationship skills)were selected from the two-rounds of Delphi survey.Four items did not achieve consensus, and stability wasobserved for these items (Supplementary Table 1) [36].Further rounds of Delphi were terminated as decided apriori.

DiscussionProfessionalism refers to a “set of values, behaviours andrelationships that underpins the trust that the public hasin doctors” [37] and there are often differences in howprofessionalism is defined and observed in differentcountries and cultures [38]. In Singapore, our residentscomprising multi-ethnic groups often start their trainingof medical professionalism in medical school, wherefoundational platforms of medical professionalism in-cluded clinical competence, capacity to address ethicaland legal issues and effective communication [37, 39],with virtues of medical professionalism such as honestyand integrity, responsibility and participation, respectand sensitivity, compassion and empathy taken as thefoundational values of medical professionalism [40].Thus, it is expected that these values continue to bereflected in their clinical practice upon graduation. Inour study, we were able to achieve consensus on 19

Table 1 Demographics of experts who participated in theDelphi survey

Characteristics Median (Range) orNumber (%)

Age, median (range) 45 (37–66)

Chinese, n (%) 13 (87)

Male, n (%) 10 (67)

Years as faculty, median (range) 12 (4–40)

Disciplines, n (%)

Medical disciplines 7 (47)

Surgery 5 (33)

Emergency medicine 1 (7)

Radiology, nuclear medicine, pathology 1 (7)

Paediatrics 1 (7)

Table 2 Summary of results from rounds 1 and 2 of the Delphisurvey

Total numberof items forscoring

Statements thatreached consensus(≥ 70%) and wereaccepted

Statements thatreached consensus(≥ 70%) and wereremoved

Round 1 23 18 0

Round 2 5 1 0

Fong et al. BMC Medical Education (2020) 20:225 Page 4 of 9

items covering four domains of medical professionalism(Doctor-patient relationship skills, Reflective skills, Timemanagement and Inter-professional relationship skills).All the 19 items, inclusive of the two new items of ‘com-municated effectively with patient’ and ‘demonstratedcollegiality’, were aligned with the aforementioned aspir-ational values that identified medical professionalism inour multi-ethnic society. In particular, our findingsunderscore the importance of recognising acceptable be-haviours according to social, religious and culturalnorms in a multi-ethnic and multi-cultural country suchas Singapore. For example, the item on maintaining ap-propriate boundaries received 100% agreement for inclu-sion into the assessment of medical professionalism.However, as the P-MEX was primarily designed to as-sess observable behaviours for use in the clinical set-ting, only 24 of the 142 behaviours were eventuallychosen to be evaluated [17], and this has also resultedin the P-MEX having a higher utility score when usedin the clinical setting to assess trainees [13]. Thus, forpractical reasons, it will not be able to assess all do-mains of professionalism completely e.g. altruism, ex-cellence and humanism [39, 41]. In Korea, a previousstudy had identified eight categories of unethical andunprofessional behaviors- (a) substandard practice, (b)

violation of work ethics, (c) misconduct related toconflict of interest, (d) dishonesty with patients, (e)violation of patient confidentiality, (f) lack of respectfor patients, (g) lack of respect for colleagues, and (h)misconduct in research [42], of which six categoriesof unprofessional behaviour seem to mirror the 19items found in our study.In our study, 4 items “solicited feedback”, “advocated

on behalf of a patient”, “extended his/herself to meet pa-tient needs” and “used health resources appropriately”did not achieve the level of agreement to be included forboth rounds, suggesting that these items may be lessrelevant in the local context as compared to other itemsin the list. This is largely in congruence with the findingsof our previous qualitative study which showed thatthese items were less relevant to patients and/or faculty[29]. Similarly, in a study to define professionalism inanaesthesiology, “resourcefulness” which is similar to“used health resources appropriately”, was deemed to beless important [43]. This may be because of the difficultyto agree as to what is “appropriate use of health re-sources”, as raised by participants in this Delphi surveyand in our previous qualitative study [29]. Modificationsto these items may be needed in the future after the pilottesting of the modified P-MEX in our local context.

Fig. 2 Delphi methodology and results

Fong et al. BMC Medical Education (2020) 20:225 Page 5 of 9

Table 4 Results from Round 2 of the Delphi survey

Item Percentage of experts who have chosen the category

Include Neutral Exclude

Domain: Doctor-patient relationship skills

Extended his/herself to meet patient needs 33 0 67

Advocated on behalf of a patient 60 27 13

Domain: Reflective skills

Solicited feedback 27 27 47

Domain: Inter-professional relationship skills

Addressed own gaps in knowledge and skills 93 7 0

Used health resources appropriately 53 7 40

Table 3 Results from Round 1 of the Delphi survey

Item Percentage of experts who have chosen the category

Include Neutral Exclude

Domain: Doctor-patient relationship skills

Listened actively to patients 93 7 0

Showed interest in patients as a person 73 0 27

Recognized and met patient needs 87 0 13

Extended his/herself to meet patient needs 27 13 60

Ensured continuity of patient care 87 7 7

Advocated on behalf of a patient 60 13 27

Maintained appropriate boundaries 100 0 0

Communicated effectively with patient 87 7 7

Domain: Reflective skills

Demonstrated awareness of limitations 93 7 0

Admitted errors/omissions 100 0 0

Solicited feedback 27 33 40

Accepted feedback 100 0 0

Maintained composure in a difficult situation 100 0 0

Domain: Time management

Was on time 87 7 7

Completed tasks in a reliable fashion 100 0 0

Was available to colleagues 73 13 13

Domain: Inter-professional relationship skills

Maintained appropriate appearance 80 13 7

Addressed own gaps in knowledge and skills 60 13 27

Demonstrated respect for colleagues 93 7 0

Avoided derogatory language 93 7 0

Maintained patient confidentiality 100 0 0

Used health resources appropriately 47 20 33

Demonstrated collegiality 100 0 0

Fong et al. BMC Medical Education (2020) 20:225 Page 6 of 9

In our study, the two new items derived from the pre-vious qualitative study- ‘communicated effectively withpatient’ and ‘demonstrated collegiality’ both achieved theconsensus to be included. This reflects the importanceof effective communication and collaborative practice inmedical professionalism locally, which is corroboratedby the development and incorporation of various com-munication and interprofessional courses in local under-graduate and postgraduate medical education [44–47].Effective communication with patients and other health-care personnel, as well as respect for patients and col-leagues were recognized as increasingly importantdomains of medical professionalism that need additionalattention and nurturing [48]. The emphasis on collegial-ity may also reflect the collectivist nature of Asian cul-ture based on the Hofstede’s cultural dimension theory[27, 42] and the increasing recognition that medical pro-fessionals practice in a community of practice [49].Other Asian countries exploring the use of the P-MEXshould consider including “communicated effectivelywith patient” and “demonstrated collegiality” to betterreflect the medical professionalism construct.Key strengths of this study include involvement of se-

nior faculty, including the DIO and ADIO, who are in-volved in the disciplinary committees handling issues oflapses in professionalism in the residency programs. Inaddition, faculty from a range of disciplines (both surgi-cal and medical specialties) participated in the Delphisurvey and of different ethnicities. There was also no ex-pert drop-out in the two rounds of Delphi survey.Limitations of this study include the arbitrariness of

the cut-off point utilised. The cut-off point of 70%adopted in our study was widely used in other studies[34, 35]. However, at this threshold, our findings shouldbe taken as the best achievable consensus given the lackof robust evidence in this field rather than as evidence ofabsolute unanimity. In addition, the study may have se-lection bias as the experts were selected by the re-searchers. However, considerable care was taken duringthe study to select the experts based on their experiencein handling of disciplinary issues in the residency pro-grams, and to include faculty of various years of experi-ence, gender, ethnicity and across various surgical andmedical disciplines. Moreover, the framework of medicalprofessionalism in the P-MEX containing 4 domains -Doctor-patient relationship, Reflective, Time manage-ment, and Inter professional relationship - may notcover entire continuum of professionalism. Therefore,we may have missed certain attributes that define med-ical professionalism such as social accountability and al-truism. Nevertheless, our qualitative [29] and surveystudies (under review) with patients and faculty mem-bers showed that the four domains of P-MEX broadlycovered medical professionalism in our setting with an

addition of 2 new sub-domains- (1) communicated ef-fectively with patient and (2) demonstrated collegiality.

ConclusionsA list of 19 items covering four domains of medical pro-fessionalism (Doctor-patient relationship skills, Reflect-ive skills, Time management and Inter-professionalrelationship skills) of the P-MEX was obtained from theDelphi study. The 4 other items (solicited feedback, ad-vocated on behalf of a patient, extended his/herself tomeet patient needs, used health resources appropriately)did not reach consensus for either inclusion or exclu-sion. The findings of this study can provide guidance forfaculty and institutions who want to introduce assess-ment of medical professionalism in the curriculum ofmedical trainees. In particular, other Asian countries ex-ploring the use of the P-MEX may consider incorporat-ing items such as “communicated effectively withpatient” and “demonstrated collegiality” to have a morecomprehensive and culturally pertinent assessment ofthe medical professionalism.

Supplementary informationSupplementary information accompanies this paper at https://doi.org/10.1186/s12909-020-02147-9.

Additional file 1: Supplementary table 1. Wilcoxon signed-rank testresults comparing responses from round 1 and round 2 of the Delphisurvey.

AbbreviationsADIO: Associate designated institutional officials; DIO: Designatedinstitutional official; P-MEX: Professionalism Mini Evaluation Exercise

AcknowledgementsThe authors thank all the participants for their time and contribution to thestudy.

Authors’ contributionsWF, YHK, SWY, JT, YYL and SCN conceptualized the study design, and draftedthe survey questionnaire. WF, YHK and JKP acquired the data. WF, YHK, SWYand JKP wrote the initial manuscript draft. All authors critically reviewed andapproved the final manuscript. SCN, YYL, JT and WF are consultantphysicians and medical educators at Singapore General Hospital. SWY, JT,YYL and YHK are mixed methods researchers with extensive experience. JKPis research coordinator at Singapore General Hospital.

FundingThis research was supported by SingHealth Duke-NUS Medicine AcademicClinical Programme Education Support Programme Grant (Ref No: 03/FY2017/P2/03-A47). Funder was not involved in the design, delivery or sub-mission of the research.

Availability of data and materialsThe datasets used and/or analysed during the current study are availablefrom the corresponding author on reasonable request.

Ethics approval and consent to participateThe SingHealth Centralized Institutional Review Board approved this study(Reference Number: 2016/3009). We obtained informed consent, whichconformed to the principle outlined in the 1964 Declaration of Helsinki, fromall participants before study commencement.

Fong et al. BMC Medical Education (2020) 20:225 Page 7 of 9

Consent for publicationNot applicable.

Competing interestsThe authors declare that they have no competing interests.

Author details1Department of Rheumatology and Immunology, Singapore GeneralHospital, 20 College Road, Singapore 169856, Singapore. 2Duke-NUS MedicalSchool, Singapore, Singapore. 3NUS Yong Loo Lin School of Medicine,National University of Singapore, Singapore, Singapore. 4Program in HealthServices and Systems Research, Duke-NUS Medical School, Singapore,Singapore.

Received: 8 April 2020 Accepted: 9 July 2020

References1. Blasi ZD, Harkness E, Ernst E, Georgiou A, Kleijnen J. Influence of context

effects on health outcomes: a systematic review. Lancet. 2001;357:757–62.2. Medicine AFABoI. Medical professionalism in the new millennium: a

physician charter. Ann Intern Med. 2002;136:243.3. Swick HM, Szenas P, Danoff D, Whitcomb ME. Teaching professionalism in

undergraduate medical education. JAMA. 1999;282:830–2.4. Cruess RL, Cruess SR. Teaching professionalism: general principles. Med

Teach. 2006;28:205–8.5. Tweedie Jude JH. Dame Jane Dacre: advancing medical professionalism

London Royal College of Physicians; 2018.6. Blank L, Kimball H, McDonald W, Merino J. Medical professionalism in the

new millennium: a physician charter 15 months later. Ann Intern Med. 2003;138:839–41.

7. Frank JR. The CanMEDS 2005 physician competency framework. Betterstandards. Better physicians. Better care. Ottawa: The Royal College ofPhysicians and Surgeons of Canada; 2005.

8. Tomorrow’s Doctors. General Medical Council 2009.9. Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional

behavior in medical school is associated with subsequent disciplinary actionby a state medical board. Acad Med. 2004;79:244–9.

10. O'Sullivan H, van Mook W, Fewtrell R, Wass V. Integrating professionalisminto the curriculum: AMEE Guide No. 61. Med Teach. 2012;34:e64–77.

11. Lynch DC, Surdyk PM, Eiser AR. Assessing professionalism: a review of theliterature. Med Teach. 2004;26:366–73.

12. Irby DM, Hamstra SJ. Parting the clouds: three professionalism frameworksin medical education. Acad Med. 2016;91:1606–11.

13. Kwan YH, Png K, Phang JK, Leung YY, Goh H, Seah Y, Thumboo J, Ng APSC,Fong W, Lie D. A systematic review of the quality and utility of observer-based instruments for assessing medical professionalism. J Grad Med Educ.2018;10:629–38.

14. Chandratilake M, McAleer S, Gibson J. Cultural similarities and differences inmedical professionalism: a multi-region study. Med Educ. 2012;46:257–66.

15. Jha V, McLean M, Gibbs TJ, Sandars J. Medical professionalism acrosscultures: a challenge for medicine and medical education. Med Teach. 2015;37:74–80.

16. Hodges B, Paul R, Ginsburg S, The Ottawa Consensus Group M. Assessmentof professionalism: from where have we come - to where are we going? Anupdate from the Ottawa consensus group on the assessment ofprofessionalism. Med Teach. 2019;41:249–55.

17. Cruess R, McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The professionalismmini-evaluation exercise: a preliminary investigation. Acad Med. 2006;81:S74–8.

18. Tsugawa Y, Tokuda Y, Ohbu S, Okubo T, Cruess R, Cruess S, Ohde S, OkadaS, Hayashida N, Fukui T. Professionalism mini-evaluation exercise for medicalresidents in Japan: a pilot study. Med Educ. 2009;43:968–78.

19. Karukivi M, Kortekangas-Savolainen O, Saxén U, Haapasalo-Pesu K-M.Professionalism mini-evaluation exercise in Finland: a preliminaryinvestigation introducing the Finnish version of the P-MEX instrument. JAdv Med Educ Prof. 2015;3:154–8.

20. Hodges BD, Ginsburg S, Cruess R, Cruess S, Delport R, Hafferty F, Ho M-J,Holmboe E, Holtman M, Ohbu S. Assessment of professionalism:recommendations from the Ottawa 2010 conference. Med Teach. 2011;33:354–63.

21. Kwon H-J, Lee Y-M, Lee Y-H, Chang H-J. Development an instrumentassessing residents’ attitude towards professionalism lapses in training.Korean J Med Educ. 2017;29:81.

22. Wang X, Shih J, Kuo F-J, Ho M-J. A scoping review of medicalprofessionalism research published in the Chinese language. BMC MedEduc. 2016;16:300.

23. Noor NM, Leong C-H. Multiculturalism in Malaysia and Singapore:contesting models. Int J Intercult Relat. 2013;37:714–26.

24. Hammer MR. Additional cross-cultural validity testing of the interculturaldevelopment inventory. Int J Intercult Relat. 2011;35:474–87.

25. Lu L. " cultural fit": individual and societal discrepancies in values, beliefs,and subjective well-being. J Soc Psychol. 2006;146:203–21.

26. Goldie J. The formation of professional identity in medical students:considerations for educators. Med Teach. 2012;34:e641–8.

27. Hofstede G. National cultures in four dimensions: a research-based theory ofcultural differences among nations. Int Stud Manage Organ. 1983;13:46–74.

28. Paik Y, Vance CM, Stage HD. The extent of divergence in human resourcepractice across three Chinese National Cultures: Hong Kong, Taiwan andSingapore. Hum Resour Manag J. 1996;6:20–31.

29. Fong W, Kwan YH, Yoon S, Phang JK, Thumboo J, Leung YY, Ng SC.Assessment of medical professionalism: preliminary results of a qualitativestudy. BMC Med Educ. 2020;20:27.

30. Ong B-H, Gonzalez-Rivas D. The inaugural SingHealth uniportal video-assisted thoracoscopic surgery masterclass in Singapore. J Vis Surg. 2018;4.http://jovs.amegroups.com/article/view/21361/20816.

31. Williams PL, Webb C. The Delphi technique: a methodological discussion. JAdv Nurs. 1994;19:180–6.

32. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus groupmethods such as Delphi and nominal group in medical education research.Med Teach. 2017;39:14–9.

33. Vernon W. The Delphi technique: a review. Int J Ther Rehabil. 2009;16:69–76.34. Robinson KR, Leighton P, Logan P, Gordon AL, Anthony K, Harwood RH,

Gladman JR, Masud T. Developing the principles of chair based exercise forolder people: a modified Delphi study. BMC Geriatr. 2014;14:65.

35. van Steenkiste BC, Jacobs JE, Verheijen NM, Levelink JH, Bottema BJAM. ADelphi technique as a method for selecting the content of an electronicpatient record for asthma. Int J Med Inform. 2002;65:7–16.

36. Chalmers J, Armour M. The Delphi Technique. In: Liamputtong P, editor.Handbook of Research Methods in Health Social Sciences. Singapore:Springer Singapore; 2019. p. 715–35.

37. Physicians WPotRCo. Doctors in society. Medical professionalism in achanging world. Clin Med (London, England). 2005;5:S5.

38. Hafferty FW, McKinlay JB. The changing medical profession an internationalperspective; 1993.

39. Stern DT. Measuring medical professionalism. New York: Oxford UniversityPress; 2006.

40. Macneill P, Joseph R, Lysaght T, Samarasekera DD, Hooi SC. Aprofessionalism program in medical education and training–From broadvalues to specific applications: YLL School of Medicine, Singapore. MedicalTeacher. 2020:1–1.

41. Ho M-J, Yu K-H, Pan H, Norris JL, Liang Y-S, Li J-N, Hirsh D. A tale of twocities: understanding the differences in medical professionalism betweentwo Chinese cultural contexts. Acad Med. 2014;89:944–50.

42. Chang H-J, Lee Y-M, Lee Y-H, Kwon H-J. Investigation of unethical andunprofessional behavior in Korean residency training. Teach Learn Med.2015;27:370–8.

43. Kearney RA. Defining professionalism in anaesthesiology. Med Educ. 2005;39:769–76.

44. Jacobs JL, Samarasekera DD, Chui WK, Chan SY, Wong LL, Liaw SY, Tan ML,Chan S. Building a successful platform for interprofessional education forhealth professions in an Asian university. Med Teach. 2013;35:343–7.

45. Chua AZ, Lo DY, Ho WH, Koh YQ, Lim DS, Tam JK, Liaw SY, Koh GC. Theeffectiveness of a shared conference experience in improvingundergraduate medical and nursing students’ attitudes towards inter-professional education in an Asian country: a before and after study. BMCMed Educ. 2015;15:233.

46. Wong M, Koh D, Phua K, Lee H. Teaching community, occupational andfamily medicine at the National University of Singapore: past, present andfuture. Ann Acad Med Singap. 2005;34:102C–7C.

47. Samarasekera DD, Ooi S, Yeo SP, Hooi SC. Medical education in Singapore.Med Teach. 2015;37:707–13.

Fong et al. BMC Medical Education (2020) 20:225 Page 8 of 9

48. Ong YT, Kow CS, Teo YH, Tan LHE, Abdurrahman A, Quek NWS, Prakash K,Cheong CWS, Tan XH, Lim WQ, et al. Nurturing professionalism in medicalschools. A systematic scoping review of training curricula between 1990–2019. Med Teach. 2020:1–14.

49. Wenger E. Communities of practice: learning, meaning, and identity:Cambridge University Press; 1999.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.

Fong et al. BMC Medical Education (2020) 20:225 Page 9 of 9