ISPUB.COM The Internet Journal of Advanced Nursing PracticeVolume 10 Number 2

1 of 9

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research GuideS Northam, J Lakomy

Citation

S Northam, J Lakomy. Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide.The Internet Journal of Advanced Nursing Practice. 2008 Volume 10 Number 2.

Abstract

In order to promote safety and progress in health care, advanced practice nurses (APNs) need to base practice on bestevidence. Evidence-based practice remains more an ideal rather than a reality. This article presents as APN Research Guide forreading and critiquing quantitative research to assist APNs to make clinical decisions based on the appraisal of the evidence forrelevance, safety, and applicability to their practice. Quantitative and qualitative data from nurses indicate the critique enhancestheir understanding of the quality and utility of research. The step-by-step process of doing the critique in small parts buildsconfidence as each section is completed. Finally, scoring the research article on a 100 point scale helps APNs determine thequality of the research.

INTRODUCTION

Reading and understanding research is challenging and isoften cited as a barrier to research application (1-3).Generally novices require more time to read, understand, andcritique studies than experienced advance practice nurses(APNs) (4), but many find the research critique to be ahelpful learning activity (5). Lack of familiarity with termsand lack of confidence in ability to understand the findingscan make the experience of reading and critiquing researchintimidating and frustrating. Few nurses regularly readresearch; and alarmingly, Wood (6) recently noted thattrepidation, antipathy, and apathy toward research have notimproved in the past two decades (7). Regular reading togain insight into best evidence useful for practice is essentialif nursing is to meet the goal of improving health carethrough the implementation of evidence-based practice (8).Thus, strategies to improve the skills of APNs in reading andunderstanding research can make the experience lessoverwhelming, time consuming and frustrating. Recentfindings have shown that reading research increases criticalthinking skills (9), thus benefiting not only patients but alsonurses. This article reviews an APN Research Guidedevised, used and refined for 12 years by nurses who haveevaluated the guide positively. Data are included to supporttheir evaluations. Understanding research is an essential firststep towards evidence-based practice (10-12).

This APN Research Guide fills a major gap in critique

guides. This guide is short (4 pages), easily printed withspace for responses, and contains the first scoring criteriasince Duffy's criteria was published in 1985 (13). The pointallocations for this critique were derived by the authors whoare experienced clinicians and researchers. While othercritique guides exist, the ones in research texts are generallylong, presented as text boxes within chapters, and lack spacefor readers to include notes as they critique.

Polit and Beck (2004) used 138 questions within 18 textboxes to guide readers in evaluating published quantitativeresearch reports. Burns and Grove (2003) included 38sections with 150 questions: 19 sections with 56 questions inthe comprehension guide; 11 sections with 86 questions inthe comparison and analysis guide; and 8 questions in theevaluation guide.

The Burns and Grove guide includes more directingquestions for evaluating the literature review, separatesevaluation of variables from measurement and validityassessment, and includes questions about missing studyelements. The reader is also guided through the process ofarticle review and critique steps twice rather than once. BothPolit and Beck (2004) and Burns and Grove (2003)recommend that readers summarize study strengths andweaknesses but do not recommend scoring research studies.The multiple sections displayed as text boxes in severalchapters cannot be quickly printed and easily used by nursesin practice.

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

2 of 9

Recently Daggett, Harbaugh and Collum (2005) published asingle page worksheet devised to teach baccalaureatenursing students to critique research by briefly noting partsof the research. The worksheet is utilitarian in its simplicitybut is more geared to noting the parts of the research ratherthan critiquing the study or determining the utility forevidence-based practice. Cutcliffe and Ward (2007) arguethat no one best guide to reading and critiquing researchwould suit all nurses. These authors review various methodsof critique, note the strengths and limitations of each, andend with a presentation of a new approach used by theNetwork for Psychiatric Nursing Research (NPNR) Journalclub. The NPNR Journal club approach can be used for bothqualitative and quantitative research and includes elevenguidelines. The NPNR approach lacks guidelines inreviewing the scientific method of quantitative research,derives no scoring of the research report but encouragesnurses to support a critique by listing strengths andlimitations.

In many areas of life from kindergarten to deliveringadvanced patient care, steps are often created that break thetask into small parts. Small parts create the impression thewhole task is not overwhelming and provide a mentaladvantage and organization for completion. The APNResearch Guide accomplishes that goal of breaking the taskof reading and understanding research into small parts.

Throughout the APN Research Guide, point allocations arenoted among the steps of research to enable readers tocalculate an overall critique score, which has not been donesince Duffy's critique in 1985 (13). While Duffy's maximumscore was 306, the maximum on this APN Research Guide is100, a more traditional score. The higher the score, the betterthe nurse appraises the parts of the research study andevaluates its strengths overall. A rating over 80 wouldindicate the research has high quality and should be highlyconsidered when the APN is making a clinical decision inthe practice setting. A future article will provide readers withan example of a critique with a high score and utility forimplementation in evidence-based practice by advancedpractice nurses.

THE MAJOR STEPS OF THE RESEARCHPROCESS

THE PROBLEM

Although simple sounding, discerning the research problemcan be difficult and may be confused with the purpose. Forexample, while the purpose of the research may be to

identify ways to safely see more patients in a busy clinicsetting, the problem may be long wait times to get labresults. Discerning the problem allows for a much morefocused way to view how to study, measure, and improve thesituation. The problem is the one clearly addressed by thestudy rather than an expansive area such as efficient healthcare delivery. Identifying the problem also lets advancedpractice APNs see if this research study is relevant toproblems or issues in their own sphere of influence andpractice.

REVIEW OF THE LITERATURE

Generally the initial narrative of the research article, justafter the abstract, lays the groundwork for what is knownabout the topic. The discussion of existing literature citesprior research and helps the APN understand what is alreadyknown about the problem. Some journals restrict the lengthof articles and thus the discussion of existing literature maybe brief. Classics are well known works in an area. Forexample, if the study is on dreams and their meaning, Freudlikely would be included since he wrote extensively ondreams. Beginning readers may be unfamiliar with classics,but this section serves as a reminder that there is classicresearch on most issues in nursing and allows the APN tomake a judgment call as to whether or not the review of theliterature is sufficient if changes in practice are beingconsidered.

THEORETICAL FRAMEWORK

Just like a long road trip requires a map, a research studyneeds an organizational structure or framework, commonlycalled the theoretical framework. Quantitative researchinvolves deductive reasoning flowing from the framework toa testable hypothesis designed to support or refute theory.Evaluation of the reasoning or conclusions that were reachedby the researcher is important. Strong research clearlyexplains the underlying theory to be tested by the researchand sometimes supports the theory with a visual model.

RESEARCH QUESTIONS AND HYPOTHESES

All quantitative research involves either research questionsand/or hypotheses, and all research reporting significanceused statistical analysis to test hypotheses. Frequently theresearch questions are noted at the end of the article'sintroduction. A research question is the researcher'sunderlying probe as to why something is happening (or nothappening) in certain situations. For example, the questionmay be as simple as “Will providing elderly patients withpedometers increase strength and endurance in walking

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

3 of 9

exercises?” A hypothesis that flows from the theoreticalframework is more of a speculation about what will happenif certain situations occur. The researcher has more of aburden with hypotheses which must be supported or not. Anexample of a hypothesis would be “Providing pedometers toelderly clients will increase the amount and quality ofwalking by 20% over a six month period.” Beginningreaders may find that selecting journals that include ahypothesis heading within the abstract is helpful. If thehypothesis is not clearly stated some sleuthing may beneeded. A quick check in the results section may report thehypothesis along with the type of statistics used to test thehypothesis. Knowing the research question or hypothesiswill help the APN understand what to look for in thediscussion of findings.

RESEARCH DESIGN

Quantitative research generally progresses along acontinuum. When little is known about a phenomenon,researchers may simply aim to describe it and are thus at thedescriptive end of the continuum. When variables are knownand surveys can be designed or existing surveys can be usedto examine relationships among variables, the design is atthe survey point, sometimes also called correlation research.Research in which the researcher directly plans themanipulation of an independent variable and measurementof a dependent variable is at the experimental end of thecontinuum. This distinction is important as APNs try todecide whether or not to recommend a change in practicebased on the appraisal of the evidence. Descriptive andcorrelation research can show relationships and situationsthat are similar to those APNs face each day and can offersome alternatives. However, before implementing evidence-based practice, the APN must appraise the evidence forsoundness, evaluate the sample size for adequacy, concludethe findings are strong enough to warrant a consideration ofchanging the ways things are done, and determine that theproposed intervention is congruent with patient preferences.Quasi-experimental means “almost, but not quite,experimental.” When human subjects are studied, sometimesall of the variables cannot be controlled like can be withanimal models. Using quasi-experimental does not mean theresearch is not sound. Whether or not the study is a quasi-experimental design is dependent on the use ofrandomization and/or a control group. This section of thecritique includes brief questions applying common researchterms.

POPULATION

Research texts generally restrict the population to the groupfrom which the sample was drawn. Research rarely usesfindings for the exact same population but rather evidence-based research efforts are geared to moving research to morewidespread, yet appropriate, use. This section is designed toencourage the APN to think about the demographics of thepopulation and who could benefit from the findings. Forexample, a study involving mothers receiving group prenatalcare that involves a New England population of 1000Whites, Blacks, and Hispanics ages 16-42 can be used inother areas of the US but lacks knowledge about Asians.Clearly, research done on one or two ethnic or racial groupscan hardly be generalized to all. The challenge is torecognize best evidence and improve practice through theappropriate generalization of research.

VARIABLES

In order to determine the usefulness and appropriateness ofapplying research findings in the clinical setting, the APNmust determine what is being tested. The variables must bedefined. One researcher might describe “lifting risk” as thechance of dropping a patient on the floor; another researchermight describe the same variable as the chance the APN willget a back injury. Both are essentially right, but the readermust know exactly what is being discussed, tested andrecommended; therefore, quantitative research involves bothconceptual and operational definitions. The conceptualdefinitions are the ones mentioned previously; theoperational definition is how the researcher measured thevariable, such as a patient fall risk scale or a back injuryscale. The research consumer must decide if themeasurement of the variable makes sense and truly reflectsthe concept of interest. This discussion pertains to constructvalidity. APNs can often discern the variables by referringback to the research question and /or hypothesis. The titlealso generally includes the key variables. The table withinthe critique provides a place to list the name of the variable(column 1), then note how the variable was measured oroperationalized (column 2), and if that method of measuringthe variable is valid. More than yes or no answers arenecessary and require critical thinking. The key criticalthinking is whether the measurement of variables done in thestudy enables the APN to know about the underlyingconcepts of interest. Actual surveys are not included inpublications but sometimes sample survey items areincluded. The APN can evaluate the clarity andappropriateness for the population of sample items. The total

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

4 of 9

number of survey items can also be noted. More itemssometimes improve the ability to capture the concept, butcertainly expansive, time consuming surveys are likely toundermine completion due to fatigue. Evaluating validitycan be difficult, but practice will increase the APN’s skill.

RELIABILITY

The term reliability has the same basic meaning in researchas in other areas such as employment, appliances, andautomobiles, among other examples. But the myriad of typesof reliability, particularly the common internal consistency,takes the term beyond the general understanding. Thissection in the APN Research Guide was designed to narrowdown reliability to the three types: stability, internalconsistency, and equivalence. Stability means that similarresults are obtained on two separate administrations of theinstrument. Homogeneity or internal consistency states thatthe items on the instrument measure the same trait. Inter-rater reliability, a type of equivalence, means that two ormore independent researchers recorded or observed theevents in the same manner. This section also includes asection for APNs to note any actions taken to enhancereliability. Generally this area is a challenging part ofcompleting the guide.

PILOT

Researchers often do a dry run of a study to work out anyproblems before undertaking a large scale, extensive study.The pilot may simply involve testing surveys using a smallgroup to discern item clarity and the time required tocomplete them. Pilots are also helpful when effect sizes areunknown and initial data can assist in examining for aneffect size to determine the sample size required in a largerstudy to discern the effect.

SAMPLE

The number of individuals who actually participated andprovided data are considered to be the sample and aregenerally included in every research study. When surveysare used, individuals may elect to answer some items but notothers and this occurrence explains why the n (sample sizenotation) varies within tables or the article narrative.Randomization may occur in either selection or assignment.Random selection begins with a source of individuals andrandomly selects a sample from it. Telephone surveys, forexample, randomly select a designated sample size from allavailable telephone numbers. Random assignment placesindividuals into groups using a method such as a coin toss.There are a variety of methods of random assignment and

random selection but neither is haphazard. The goal ofrandomization is to allow individuals an equal chance ofselection or assignment and reduce bias. Randomizedcontrolled designs (RCT) are the classic experimentalresearch design and involve random assignment into groups.Sample demographics, often presented in a table, can becompared to the demographics of the population to evaluateif the sample is representative.

DATA COLLECTION

The actual steps involved in gathering data are generallyaddressed very briefly within the research article, primarilyto save space. The reader may be unable to discern whocollected the data or the circumstances of data collection.Acknowledgements at the beginning or end of the articlemay note the names of individuals who gathered data but allauthors may not have actively participated in the datacollection. As much information as possible can be includedin this section but incomplete insight into the “who, what,when, where and how” is common.

LIMITATIONS

The limitations section primarily pertains to internal andexternal threats to validity. Discerning threats helps toevaluate rival hypotheses, which are alternate explanationsfor changes in the dependent variable other than theindependent variable. Controls are any actions taken by theresearcher to understand or minimize the threats to validity.

PREPARING DATA FOR ANALYSIS

This section was devised as a step before considering thestatistical analysis. The variables listed here should directlymirror each variable listed in section 7 of the APN ResearchGuide where the variables, their measurement and validitywere presented. The goal is to think of what the data willyield in terms of a number and whether an average isfeasible. Non-parametric statistics are used when no averagescore is possible or meaningful such as with demographicvariables of race or marital status. Parametric statistics areused when an average score is possible and meaningful withthe caveat that other assumptions for each parametricstatistic must also be met. So by determining whether ameaningful mean is possible from the data, the reader gets abeginning idea of whether non-parametric or parametricstatistics will be used in analysis.

DATA ANALYSIS

Many research reports include a variety of statistics fromdescriptive statistics used to report demographics to

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

5 of 9

inferential statistics used to test hypotheses. At this point inthe critique the APN should refer back to the hypothesis andlook for reported significant or non-significant findings.Inferential statistics involve probability testing and report a pvalue which equates to the likelihood of error. In researchwith minimal risk, a preset alpha of .05 is generally used andindicates a 5% chance of error, or 5 out of 100 times theresults are due to chance. A more conservative preset alphaof .01 is used for research with more risk such as clinicaltrials involving new medications. Hypothesis testing is donebased upon the wording of the hypothesis, the number ofgroups and the number of variables. Hypotheses whichpredict a relationship between two continuous variables aretested using Pearson's Product Moment Correlation.Hypothesis predicting a difference in two groups on a singlecontinuous outcome variable are tested using a parametric ttest. Differences in more than two groups on a singlecontinuous variable are tested using ANOVA. When anumber of independent variables are used to predict a singlecontinuous outcome variable, multiple regression is used andif the outcome variable is dichotomous, logistic regression isappropriate. Many research studies involve testing of morethan one hypothesis and thus the results section includesfindings using a variety of statistical tests. The pre-set alphais the level of error the researcher felt reasonable beforebeginning the research and is often omitted from theresearch report. Since a 5% chance of error is common, anyreported p values less than .05 that are termed significant(sometimes using an asterisk) imply a preset alpha of 0.05.Sometimes notations under tables help to discern the presetalpha. A single asterisk is conventionally used to denotesignificance found at the ≤05 level; two asterisks denotesignificance at ≤.01, and three asterisks denote significanceat ≤.001. Significant findings indicate the findings areunlikely due to chance and similar results can be expectedfrom similar populations, thereby facilitating generalizationof the findings.

INTERPRETING THE RESULTS

The results of research are interpreted based upon thestatistical findings, their credibility and sample size. Whenfindings are statistically significant and a hypothesis wastested, the results support the hypothesis. For example, in astudy that hypothesized women in group prenatal careexperience more maternal stress reduction during pregnancythan standard prenatal care, a p value of <.05 and a lowermean stress level in the group prenatal care group supportsthe hypothesis. If the results make sense the reader evaluatesthem as credible. Studies may test more than one hypothesis

and the most important one is often reflected in the journalarticle title. The groups would need to be fairly similarinitially on stress levels as well as demographic variables toascribe the changes to the independent variable, the type ofprenatal care. Large samples lend credibility to the findingsand small samples undermine credibility. At least 30 peopleper variable and even group sizes (17) are desired. Skilledclinicians reading in an area of expertise should trust theirability to interpret research findings.

COMMUNICATION OF FINDINGS

Internationally, researchers strive to reduce the time fromresearch to implementation into practice. While all researchcannot be easily implemented, this section prompts the APNto evaluate who should know about the research and whetheror not implementation is appropriate. The actions shouldclearly be derived from the research. Small, simpleinterventions such as the use of music to reduce proceduralstress may be easily implemented in practice. Newprocedures, such as the method of caring for wounds,generally require more evidence and policy changes.Research reports often include a section on implications forpractice which summarize recommendations for using thestudy findings.

ETHICS

Four areas are included to evaluate the three basic tenets ofresearch ethics: beneficence, respect for human dignity, andjustice. APNs are directed to comment on institutionalreview board (IRB) review, informed consent, risks,benefits, and the qualifications of the researchers. Anyreference to grant funding, particularly US governmentfunding from institutions like the National Institute of Health(NIH) or the American Cancer Institute, imply IRB approvalsince such approval is required at the time of proposalsubmission. The use of a consent form also implies IRB orhuman subjects review board (HSRB) approval.

Risks are rarely enumerated within research article reports sothe reader must consider the risks and weigh them againstthe benefits. The qualifications of the researchers aredetermined by noting the educational and professionalcredential notations after the author names or within afootnote about the authors. Anonymity exists if theresearcher never saw the research subject. More oftenanonymity did not exist within the research study butconfidentiality was protected.

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

6 of 9

INTERPRETING THE CRITIQUING SCORE

Once the APN has completed the APN Research Guide, ascore is obtained. If the score is less than 70, no change isrecommended for practice. If the score is between 71 – 79,the APN should utilize this research cautiously in thepractice setting. If the score is 80 or greater, the APN maymore comfortably include the findings in the clinicaldecision process for evidence- based practice. Scoring on a100 point scale is common in graded educational projectsand the 71-79 scores equate to average work, 80 or greater isgood, and 90 or greater is excellent. The cut off scores werederived from the authors' educational experience and usedsuccessfully by clinicians enrolled in graduate education. Apilot study of the scoring was conducted using 4 nursinggraduate students enrolled in a research course that read andrated a published primary research report 5 weeks into thecourse. Inter-rater reliability was calculated as .89 (M =89.5; SD = 5.26; Std. Error = 2.63) and scoring ranged from85-97 without any specific training. The nurses were simplyasked to read and use the APN Research Guide, rate eachsection, and then create a sum score. The high inter-raterreliability reflects the good consensus in scoring the researchand indicates there was general agreement on the quality ofthe published research report. The nurses agreed the articlewas good and amenable to implementation for practice.

EVALUATION OF THE CRITIQUE

This critique guide was devised to help APNs read andevaluate research. The guide has been very positivelyevaluated and APNs suggested this article as a way to sharethe tool. To gather some recent evaluation data, ananonymous online evaluation was completed by APNs(N=12), 98% strongly agreed and 8% agreed that the guideassisted their understanding of research. Qualitativecomments included: 1) “I enjoyed the critiques--theychanged my way of reading professional articles and studies.They made me think a little more critically”; 2) “It hashelped me to become more comfortable reviewing researchand discussing it with others. Information I have seen beforenow makes more sense to me”; 3) “Even though I started outfearing research, writing the critiques made me understandthe process so much better. Now, I look for articles to see ifthey are as great as they say they are. What a great learningexperience.”

SUMMARY

Quantitative research involves a deductive process outlinedwithin this critique. The goal is to break the process downinto small parts with questions to guide APNs in doing

critical appraisal of the evidence. Even when a section ortwo are challenging, the overall ease of completion fostersthe understanding and confidence that are essential for APNsstriving to improve their skills in reading and using bestevidence in practice.

ACKNOWLEDGEMENTS

Special thanks to Dr. Lynn Wieck for editorial input and Dr.Cheryl Beck for review and input.

Figure 1

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

7 of 9

Figure 2 Figure 3

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

8 of 9

Figure 4

Figure 5

References

1. Oermann M. Brief reports for disseminating systematicreviews to APRNs. Clinical APRN Specialist: The Journalfor Advanced Nursing Practice, 2006;20;5:233-238.2. Retsas A & Retsas A. (2000). Barriers to using researchevidence in nursing practice. Journal of Advanced Nursing,2000;31;3:599-606.3. Walsh K. How APRNs perceive barriers to researchimplementation. Nursing Standard, 1997;11;29:34-39.4. Millor G, Levin R, Carter E, Doswell W, Jacobson L,Shortridge L. Cognitive skills and developmentcompetencies for conducting research critiques. Journal ofNew York State APRNs Association, 1991;22;3:12-16.5. Dobratz M. Putting the pieces together: Teachingundergraduate research from a theoretical perspective.Journal of Advanced Nursing, 2003;41;4:383-392.6. Wood M. Nursing practice research and evidence-basedpractice. Nursing Research, 2006;15;2:83-85.7. Diers D. Research in nursing practice. New York:Lippincott; 1979.8. Melnyk, B. M. & Fineout-Overholt, E. (2005). Evidence-based practice in nursing & healthcare. Philadelphia:Lippincott Williams & Wilkins.9. Chen F & Lin M. Effects of a nursing literature readingcourse on promoting critical thinking in two-year nursingprogram students. Journal of Nursing Research,2003;11;2:137-148.10. Oermann MH, Galvin EA, Floyd JA, Roop JC.Presenting research to clinicians: Strategies for writing aboutresearch findings. APRN Researcher, 2006;13;4:66-74.11. Broome B & Barkley T. Decoding the meaning ofresearch: applications in acute care. APRN PractitionerForum, 2001;12;3; p127-137.12. Hutchinson AM, Johnston L. Bridging the divide: asurvey of APRNs' opinions regarding barriers to, andfacilitators of research utilization in the practice setting.Journal of Clinical Nursing, 2004;13:3: p304-315.13. Duffy, M. A research appraisal checklist for evaluatingnursing research reports. Nursing Healthcare, 1985;6:10p539-47.14. Polit DF, Beck, CT. Nursing research: Principles andmethods. Philadelphia: Lippincott; 2004.15. Burns N, Grove SK. The practice of nursing research:Conduct, critique & utilization. Elsevier; 2005.16. Daggett LM, Harbaugh BL, Collum LA. A worksheet forcritiquing quantitative nursing research. Nurse Educator,2005:30(6), p255-258.17. Grinnell RM. Social work research and evaluation:Quantitative and qualitative approaches Itasca, IL: Peacock,2001.

Promoting Advanced Practice Nurse Buy-In For Evidence-Based Practice: An Apn Research Guide

9 of 9

Author Information

Sally Northam, PhD, RNProfessor, ollege of Nursing and Health Sciences The University of Texas at Tyler 3900 University Blvd. Tyler, Texas

Janet Lakomy, PhD, RNAdjunct Associate Professor, College of Nursing and Health Sciences The University of Texas at Tyler 3900 UniversityBlvd. Tyler, Texas