1

Societal relevance of research

Peer Review WU research groups 2015

Theo Jetten

Who is Theo Jetten?• Executive Secretary of one of the Wageningen Graduate Schools (0.6)

– Safeguarding the quality of research and researchers within 20 universitygroups

– Development and implementation of a local WU format of the Standard Evaluation Protocol 2015-2021 for publicly funded research organizations

– Composition of documents at institute level, commenting documents at research group level

– Responsible for overall programme visit panel / presentation institute.

• Associated to library Wageningen UR since June 2015 (0.4)– Use of bibliometric systems, data management, OA– Interface of daily research practice and use of digital information

systems

• Background – MSc Phytopatology (1987), PhD Meteorology (1992), postdoc Vector borne

diseases until 1995 and many committees/projects after this– Focus on models and data

Knowledge Transfer

• Effective transfer of knowledge from the research sector to other sectors for use in the creation of economic, cultural, social, and personal value in society

• Essential in order to benefit more from the rapidly accumulating research efforts and results.

from Finne et al, A Composite Indicator for Knowledge Transfer, European Commission’ Expert Group on Knowledge Transfer Indicators

Peer Review 2015

• According to Standard Evaluation Protocol SEP 2015 -2021– determined by KNAW, VSNU and NWO – local format adapted from SEP– written documention and interviews with groups (incl. short presentation)

– June 2015, 77 Research groups Wageningen University within 5 GraduateSchools assessed by external independent peer review panels

- focus on three SEP criteria– Research quality– Relevance to society– Viability, using a SWOT analysis

- categories: World leading/excellent (1), Very good (2), Good (3), andUnsatisfactory (4);

- a research group is assessed within the context of its research domain

Peer Review 2015

The evaluation system aims to achieve three generic objectives:

• Improvement in the quality of research through an assessment carried out according to international standards of quality and relevance;

• Improvement in research management and leadership;

• Accountability to the higher management levels of the research organisations and to the funding agencies, government and society at large.

Both retrospective as well as prospective

Societal relevance

Narrative on relevance to society: short ( 500 words, excl. examples)

• to describe the most convincing examples of relevance, impact or added value to society achieved by means of the scientific work of the research unit.

• refer to the societal relevance of the domains in which your students will pursue their professional careers

• describe specific targets of the past six years and the targets identified for the next 5-10 years.

Concise, supported by prioritized examples (most important per category) and indicators

Quality and Relevance SEP

Research quality Relevance to society

Demonstrable

products

Demonstrable

use of

products

Demonstrable

marks of

recognition

Societal relevance

Demonstrable products - Research products for societal target groups

• professional publications

• publications for the general public

• instruments, infrastructure

• datasets, software tools, or designs developed

• reports (e.g. for the sake of policymaking)

• outreach activities for a general audience (lectures, exhibitions)

• activities with respect to primary and secondary education

Societal relevance

Demonstrable products - Research products for societal target groups

• professional publications

• publications for the general public

• instruments, infrastructure

• datasets, software tools, or designs developed

• reports (e.g. for the sake of policymaking)

• outreach activities for a general audience (lectures, exhibitions)

• activities with respect to primary and secondary education

Within CRIS (e.g. Pure) registered as research output/activities

• depending on the group;

• metrics: absolute numbers; sometimes downloads

Societal relevance

Demonstrable use of products - Use of research products by societal groups

• patents/licences

• use of research facilities by societal groups

• projects in cooperation with societal groups (European Union, governments, international funds)

• contract research (including consultancies), co-publications

• overview first jobs/present jobs of alumni (MSc and PhD / only in general terms when referring to the domains)

• participation in training courses

Societal relevance

Demonstrable use of products - Use of research products by societal groups

• patents/licences (list of patents/licences or contribution to)

• use of research facilities by societal groups (description)

• projects in cooperation with societal groups (European Union, governments, international funds) (description major ones, 1st/2nd/3rd

fte)

• contract research (including consultancies) (description major parties, overall budgets group)

• co-publications (descriptive, systems such as Scival offer tools)

• overview first jobs/present jobs of alumni (MSc and PhD / only in general terms when referring to the domains) (alumni office, enquiry)

• participation in training courses (description courses offered forprofessionals)

Societal relevance

Demonstrable marks of recognition - Marks of recognition by societal groups

– public prizes

– valorisation / Technology Transfer funding

– number of appointments/positions paid for by societal groups (or part-time appointments outside academia)

– membership of civil society advisory bodies

– media exposure as presentations on radio or TV, interviews, invited opinion articles, followers in the social media (e.g. YouTube, Twitter, Facebook, LinkedIn)

Societal relevance

Demonstrable marks of recognition - Marks of recognition by societal groups

– public prizes

– valorisation / Technology Transfer funding

– number of appointments/positions paid for by societal groups (or part-time appointments outside academia)

– membership of civil society advisory bodies

– media exposure as presentations on radio or TV, interviews, invited opinion articles, followers in the social media (e.g. YouTube, Twitter, Facebook, LinkedIn, ResearchGate)

CRIS offers possibilities for registration; often not used

Societal relevance

The responsibility for presenting a narrative and background material is with a research group

External advice about whether narrative is appealing and suggestions to complete background material (comprehensive but presented in a concise way with a focus on highlights).

Societal relevanceObservations

• From their own point of view groups are well able to demonstrate therelevance of their research, however, legitimacy of their work(appreciation by others) does need more attention.

• Documentation needs to be evidence based.

– in the context of the research domain of the group

– comprehensive

– impact activities important (numbers)

– not neccessary to complete all categories

• Results: world leading (1); very good (2); good (3)

1 2 3

29 45 3

38% 58% 4%

Societal relevance

Impact world leading.....

• N2Africa: Gate Foundation (250,000 farmer fields)

• book edible insects: 6 million downloads

• statistical Software package CANOCO 12,000 citations

• nematode diagnostics company clear detections: market opportunities > 80 M€

• advisor to minister and Dutch Parliament agri-environment schemes and Nature Conservation Act.

• coordinator of the Mobile DNA labs national initiative – which has already reached out to 170,000 secondary school children 2008-2015

• at Your Request® meal service of hospital the Gelderse Vallei, which helps patients to make a healthy choice, and which is now set as model by the national media and is discussed by the government

Societal relevance

Impact world leading (cont.).....

• the Taste Lessons programme is conducted in several 100’s of elementary schools to educate children on tasting food and create awareness on sustainability

• a software tool that optimizes concentrate feeding via a self-learning dynamic model at the level of the individual cow. This system is now used by over 1,000 dairy farmers. The use of the system significantly reduces the use of concentrates

• Animal Health Management blog (animal-health-management.blogspot.com) with more than 14,000 visits.

Societal relevance

Societal relevance

Remarks committees: three committees have serious comments about the increased focus to societal relevance

Remarks: Experimental Plant Sciences

The committee noticed that many research groups have put emphasis on applied aspects in recent years, to stress their relevance and increase funding options. While understandable, it is notable that industry specifically asks for fundamental research, to provide a basis for future activities. The research school could play a role here by identifying the fundamental issues for the future and lobby for funding in these areas.

Societal relevance

Remarks: Production Ecology & Resource Conservation

Societal relevance of the res`1earch done at the school is beyond doubt, with strong engagement within the Netherlands, and in Europe and particularly in Africa, Asia and Latin America. There are many different models for the chair groups to deliver applications of their research, and these differences unavoidably and harmlessly reflect individual preferences of the people involved as well as disciplinary constraints. The level of engagement with industry, private companies and stakeholders is high. All the groups in the ecology area deliver strongly to development and influence capacity building in developing countries. This is very positive and should not be changed, but it must be pointed out that a very strong focus on societal outcomes provides a risk that ‘fundamental’ science becomes ‘too applied’.

Societal relevance

Remarks: Food Sciences

These developments ensure that the university’s high societal relevance is matched with its major scientific relevance. In further developing these dual objectives, the continued and essential input of the basic physical and biological sciences must not be overlooked.

Societal relevance

Remarks committees: two committees questioned the structure of the information

Remarks: Animal Sciences

Especially for a heterogeneous, and rather descriptive assessment category such as “societal relevance”, evaluation of different factors may lead ultimately to one and the same numerical score.

Societal relevance

Remarks: Social Sciences

The Committee recommends that WASS and Wageningen University look for ways to make the societal impact of its research more tangible and explicit. In particular, the Committee suggests that there should be more clear and agreed guidelines on how societal impact is reported: for example, through a more structured presentation of activities or the inclusion of well-documented case studies. One particular possibility here could be to identify, on the one hand, specific clients or stakeholders (e.g., government, the private sector, civil society) and, on the other, the types of impacts involved (from briefings through to specific forms of implementation).

Peer Review and Societal relevanceConclusions and discussions

• Despite editorial comments and guidelines a rather diverse picture of group presentations both in size and focus

• Committees struggled with how to assess this assessment category, although most groups got a 1 or 2. Diversity and number of activities plus impact seems to be leading.

• It is up to the groups what they want to register in a CRIS but a group strategy on this is needed; use CRIS to compose CVs

• Assessing societal relevance would benefit from general applicable metrics; e.g. references found in media aimed at professionals or general public (e.g. LexisNexis)