Critical evaluation (web version)
-
Upload
durhamlibrarydtp -
Category
Documents
-
view
114 -
download
1
description
Transcript of Critical evaluation (web version)
![Page 1: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/1.jpg)
Critical Evaluation
of Research Information
James Bisset ([email protected] ) Academic Liaison Librarian (Research Support)
![Page 2: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/2.jpg)
Session outline- Importance of evaluation- Forms of value - Group activity
- Evaluating Research Information
![Page 3: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/3.jpg)
Session outline- Importance of evaluation- Forms of value - Group activity
- Evaluating Research Information
Self awareness –
what do you bring
to the table
![Page 4: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/4.jpg)
Session outline- Importance of evaluation- Forms of value - Group activity- Evaluating Research Information
Self awareness –
what do you bring
to the table
Some practical
tips and
evaluation criteria
![Page 5: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/5.jpg)
Part 1 Importance
of evaluation
![Page 6: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/6.jpg)
• Much training is about directing you to the right information = searching and retrieval
• As postgraduate researchers you have to be critical and reflect on what you find.
• Be aware of your impact on your own research, and the research of others.
• What defines your evaluative criteria?
The need to evaluate information
![Page 7: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/7.jpg)
• Resources are interconnected and they evolve
• Information resources are transformed into knowledge
• Knowledge becomes a resource
• Therefore prior knowledge shapes what we go on to create
Ecology of Resources
![Page 8: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/8.jpg)
Role of the researcher
• In theory we can select almost any information to complete a task
• In practice we filter it by selecting resources we think most appropriate
• Motivation - affected by the learning we have already done
![Page 9: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/9.jpg)
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement• People• Technologies• Cost• Skills• Copyright, IP
![Page 10: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/10.jpg)
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement• People• Technologies• Cost• Skills• Copyright, IP
![Page 11: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/11.jpg)
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement• People• Technologies• Cost• Skills• Copyright, IP
![Page 12: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/12.jpg)
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement• People• Technologies• Cost• Skills• Copyright, IP
![Page 13: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/13.jpg)
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement• People• Technologies• Cost• Skills• Copyright, IP
![Page 14: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/14.jpg)
• You need to be critical and reflect on all of the sources you find and use.
• You have a professional responsibility, as your research will impact on others.
• You are creating knowledge…… which evolves, and will shape what others create…… similarly, the information you discover will shape the knowledge you create.
• You need to be aware of the filters already impacting upon the information you use.
Part 1 Summary
![Page 15: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/15.jpg)
Part 2 Forms of value
![Page 16: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/16.jpg)
• Filtering process = value judgement– By researcher–Made on their behalf
• What forms of value are there and how do they work together to create information literate researcher?
Forms of Value
![Page 17: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/17.jpg)
Objective form of value
• Scientific measures of validity or reliability
• Exists so that personal values don’t unduly influence work
• Omit this scheme of value and we risk information (and knowledge formed from it) becoming counterknowledge (Thompson, 2008).
![Page 18: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/18.jpg)
Intersubjective form of value
• Based on the shared values of a community e.g. morals, ethics, laws, economics
• Allows for discussion of scientific method as it can’t explain everything
• Acceptance in a community• If we omit this then values are
relativist
![Page 19: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/19.jpg)
Subjective form of value
• Decisions you make– Is this what I want, do I need this, is it
relevant?
• Privileges you as the researcher in the decision making process
• If we omit it we get groupthink (Janis, 1972) or battery cognition (Blaug, 2007)
• Importance therefore of asserting individual criticality
![Page 21: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/21.jpg)
Cognitive biases• Subjectivity is vulnerable to bias &
hunches• Concept of cognitive bias was
developed in 1970s by Tversky and Kahneman
• Four main groups- Social - Probability/belief - Memory - Decision making
![Page 22: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/22.jpg)
Social biases• Ascribe positive or negative traits to
self, individuals or groups• Loading values or anticipating action
based on prior experience or a bias against self, individuals or groups
• Academic impact: need to verify information and not rely on own views; important to remember when analysing human subjects
![Page 23: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/23.jpg)
![Page 24: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/24.jpg)
![Page 25: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/25.jpg)
Memory biases• How you perceive past events• False memory, positive memory,
imbalanced memory• E.g. A Photo, a Suggestion, a False
Memory • Academic impact: importance of
accurate record keeping and note taking
![Page 26: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/26.jpg)
Memory biases
![Page 27: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/27.jpg)
Memory biases
![Page 28: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/28.jpg)
Probability and belief
• To disregard or to pay too much attention to probability
• Academic impact: need to treat each research finding as distinct and to judge it in its own right
![Page 29: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/29.jpg)
![Page 30: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/30.jpg)
![Page 31: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/31.jpg)
![Page 32: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/32.jpg)
![Page 33: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/33.jpg)
Decision-making biases
• Influences on your decisions by own biases or those of a group
• Academic impact: need to be objective and consider all possible routes of enquiry and treat all research findings as valid until proved otherwise e.g. Semmelweis reflex
![Page 34: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/34.jpg)
Cognitive biases• On your table, group the
forty cards into four piles of ten• Social • Memory • Probability• Decision
![Page 35: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/35.jpg)
Three forms of valueSubjectiv
e
Inter-subjectiveObjective
![Page 36: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/36.jpg)
• Different concepts of ‘value’ and the need to evaluate in terms of each of these. - need to be objective and look at the measurable facts - need to be inter-subjective and apply to the wider context - need to be subjective, and assess the value based upon our own needs
• Explored some of the key bias which may subconsciously be impacting upon how your (subjective) measure of ‘value’ might effect how you filter information and your evaluative criteria.#
Part 2 Summary
![Page 37: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/37.jpg)
Part 3 Evaluation of Research Information
![Page 38: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/38.jpg)
Evaluating information In a literature review you need to
evaluate:• Relevance to the topic• Authority of the author, publisher etc• Objectivity• Presentation• Method of production and methodology• Currency
![Page 39: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/39.jpg)
Evaluating information In a literature review you need to
evaluate:• Relevance to the topic• Authority of the author, publisher etc• Objectivity• Presentation• Method of production and methodology• Currency
![Page 40: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/40.jpg)
• Read the abstract, introduction or summary.• Scan the bibliographic information which may
highlight key subject areas not specifically alluded to.
• Emphasis may not be clear until you read in full.
Relevance to the topic
![Page 41: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/41.jpg)
![Page 42: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/42.jpg)
• Read the abstract, introduction or summary.• Scan the bibliographic information which may
highlight key subject areas not specifically alluded to.
• Emphasis may not be clear until you read in full.
Relevance to the topic
Be aware of what is filtering your choices… - Vocabulary and broadness of interpretation. Are you under-estimating the value of a source because it doesn’t match your choice of keywords precisely?
![Page 43: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/43.jpg)
• Are the authors acknowledged experts in the field? - frequently cited? - have you or colleagues heard of them? - do they have an h-index? - can you find any profile information where they work?
• Where is it published? - Impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige)
Authority
![Page 44: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/44.jpg)
• Are the authors acknowledged experts in the field? - frequently cited? - have you or colleagues heard of them? - do they have an h-index? - can you find any profile information where they work?
• Where is it published? - Impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige)
Authority
Be aware of what is filtering your choices y - Is the prestige of the author impacting on how you evaluate the content?
![Page 45: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/45.jpg)
• Is the subject controversial?
• If there are differing views on the subject area, does the author consistently fall into one ‘camp’?
Objectivity
![Page 46: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/46.jpg)
• Is the subject controversial?
• If there are differing views on the subject area, does the author consistently fall into one ‘camp’?
Objectivity
Be aware of what is filtering your choices - Does the author demonstrate any hidden bias on the topic? - Evaluate yourself? Are you subconsciously over-valuing the resource because it confirms your own prejudices? Are you being objective?
![Page 47: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/47.jpg)
• Is the article peer-reviewed?• Can you identify the editor/editorial
board for the publication?
Method of production
![Page 48: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/48.jpg)
• Is the article peer-reviewed?• Can you identify the editor/editorial
board for the publication?
Method of production
Be aware of what is filtering your choices… - Be aware of editorial policy which may decide what is published.
![Page 49: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/49.jpg)
2.23.39 – 2.25.56http://
news.bbc.co.uk/1/hi/help/3681938.stm
![Page 50: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/50.jpg)
• Various criteria you can assess a resource by.
- a lot more ‘citation’ tools available for journal literature.
• How much time do you realistically have?
Part 3 Summary
![Page 51: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/51.jpg)
Bibliography• Blaug, R. (2007) ‘Cognition in a hierarchy’, Contemporary Political Theory. 6: 24–44
• Goldacre, B. (2008) Bad science. London: Harper Press.
• Janis, I. (1972) Victims of groupthink: a psychological study of foreign-policy decisions and fiascoes. Boston: Houghton Mifflin.
• Kahneman, D. and Amos, A. (1972) ‘Subjective probability: a judgment of representativeness’, Cognitive Psychology. 3(3): 430-454.
• Luckin, R. (2010) Redesigning learning contexts: technology-rich, learner-centred ecologies. Abingdon: Routledge.
• Strange, D., Hayne, H. and Garry, M. (2008) ’A photo, a suggestion, a false memory’, Applied Cognitive Psychology. 22: 587–603.
• Thompson, D. (2008) Counterknowledge: How We Surrendered to Conspiracy Theories, Quack Medicine, Bogus Science and Fake History. London: Atlantic Books.
• Whitworth, A. (2009) Information Obesity. Oxford, UK: Chandos. In particular chapter 2.
• Whitworth, D (2010) “The three domains of value: Why IL practitioners must take a holistic approach” Available at: http://prezi.com/rxqnzpoooolb/the-three-domains-of-value-why-il-practitioners-must-take-a-holistic-approach/
• http://www.informationliteracy.ie/
![Page 52: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/52.jpg)
Image Credits[Slide 5, 38] Via Flickr Creative Commons, by Martin LaBar. Original available at http://www.flickr.com/photos/32454422@N00/163107859/
[Slide 3] Via Flickr Creative Commons, by Kevin Dooley. Original available at http://www.flickr.com/photos/12836528@N00/2577006675
[Slide 10] Via Flickr Creative Commons, by shellorz. Original available at http://www.flickr.com/photos/59198719@N00/2192821345
[Slide 13] Via Flickr Creative Commons, by Richard Cocks. Original available at http://www.flickr.com/photos/richardland/3999234316/
[Slide 12] Via Flickr Creative Commons, by Photo Extremist. Original available at http://www.flickr.com/photos/thevlue/4839060646/
[Slide 11] Via Flickr Creative Commons, by What Dave Sees. Original available at http://www.flickr.com/photos/whatdavesees/2487875504/
![Page 53: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/53.jpg)
Image Credits[Slide 14] Via Flickr Creative Commons, by vl8189. Original available at http://www.flickr.com/photos/27630470@N03/
[Slide 15] Via Flickr Creative Commons, by opensourceway. Original available at http://www.flickr.com/photos/47691521@N07/4371001458/
[Slide 17] Via Flickr Creative Commons, by otherthings. Original available at http://www.flickr.com/photos/18619970@N00/3057937540
[Slide 30-31] Photo provided by colleague
[Slide 54] ‘Vitae®, © 2010 Careers Research and Advisory Centre (CRAC) Limited‘ Available at www.vitae.ac.uk/rdf
![Page 54: Critical evaluation (web version)](https://reader033.fdocuments.net/reader033/viewer/2022061303/54935e5dac795925288b4a16/html5/thumbnails/54.jpg)
Measuring Researche
r Developm
ent
Vitae Researcher Development Framework [see image credits]