1 [Slide 1] cover slide

[Slide 2] Introduction

-Purpose of presentation is to provide an overview of recent research about both the LS/CMI and its youth version YLS/CMI. We will look at what is being said about equality issues. What will also be discussed are issues to look out for in both instruments.

-To begin with, an overview will be provided of research studies about the LS/CMI and YLS/CMI.

-Then, we will look at research into the criminogenic factors (and protected characteristics as per equality legislation) of gender, ethnicity and mental disorders.

-The final section will highlight some additional items to look out for in both versions of the tool.

-To finish off, the main points will be summarised.

[Slide 3] Definitions

• LS/CMI – developed from personality psychology and social learning theories.

• Strengths of the LS/CMI – it is a measure of risk and needs with a case management component; takes into consideration the protective factors that may desist from further offending; levels of risk may be overridden by assessors in cases where the strengths or risks may suggest the total score does not accurately reflect the risk level.

• YLS/CMI – version of the LS/CMI for 12-18 year olds. Since adolescence is a fluctuating period for individual development, it has been recommended that this is updated every six months to capture development.

• Strengths of the YLS/CMI – protective factors are documented; space to record narratives for any additional information like special circumstances that are not captured in risk and needs items; case management component to aid planning and analysis.

• Which one to use? Overlap in ages since the LS/CMI can be used with ages sixteen and over; whilst the YLS/CMI has been used with 12-18 year olds. Consider the maturity and circumstances of the individual. A 16 year old who lives independently and is employed may be more suited to the adult version than an 18 year old who lives with their parents, has mental disabilities and is financially dependent on them.

• Another point worth mentioning is the legislation in Scotland is defining youth up until age 24; however, existing literature tends to consider youth as those aged under 18. Research has tested using the YLS/CMI with up to age 21 and found that it was not very accurate. It has, therefore been recommended that the youth version is not used with those aged over 18 (Vaswani and Merone 2012; Vaswani 2013). With regards to those younger than age 12, Campbell et al. (2014) did include individuals aged between 9-11 in a study of the YLS/CMI; yet were unable to draw any meaningful conclusions about how useful it was given only 6% of the sample fell within this age range.

2 [Slide 4 Research Update: LS/CMI]

• Two of the main ways to measure the strength of a risk assessment tool are assessing its inter-rater reliability and its predictive accuracy. Inter-rater reliability means the extent to which its items are scored in a consistent manner, i.e. different assessors will score items in the same way. A way to find this out would be to instruct a group of assessors (preferably trained and untrained ones) to score the instrument based on case studies of offenders and determining whether items are being scored in the same way. Predictive accuracy demonstrates the extent to which a tool’s scores can predict recidivism. This tends to be estimated by looking at a sample of offenders over a long period of time and calculating how many of those at different risk levels did go on to reoffend. So, those are the two key terms I will use throughout this presentation when discussing the strength of the LS/CMI and the YLS/CMI: inter-rater reliability and predictive accuracy.

• Studies have found LS/CMI items achieve moderate to high levels of inter-rater reliability (Labrecque et al. 2017; Rettinger and Andrews 2010). [IRR range of .65 to .91. Follow up periods of 1 month and 57 months respectively] A recent study, however, did find that seven items within the tool yielded inadequate levels of consistency, suggesting that training around these items should be strengthened (Labrecque et al. 2017) [ICC of below .50]. [For interested persons, these are the following items: criminal attitude, few criminal acquaintances, non-rewarding – parental, non-rewarding – other relatives, some criminal friends, unfavourable toward convention, poor toward supervision/treatment.]

• Bhutta &Wormith (2016) carried out a study in Pakistan. The LS/CMI was translated into Urdu and modified to address social factors: for instance, the education subscale was modified to align with the lower literacy rate than that of Western countries. After being administered to 55 probationers, the LS/CMI was found to be a useful indicator of recidivism. [follow-up period of 10-11 months]

• Gordon and colleagues (2015) tested the tool in Australia. It yielded significant but weak predictive utility. [AUC of .62. Follow up period of 12 months]. They concluded that it might not be the most suitable tool for measuring risk in Australia.

• The LS/CMI was translated into French and used in a Canadian study of 3446 individuals with a sentence of less than two years (Guay & Parent 2017). Good predictive accuracy was shown for new arrests and new convictions. [AUC .range of .70-.77. Follow up period of 24 months]

• Research in United States found the risk scores were strongly predictive of recidivism (Spence and Haas 2015). The authors found that it was possible to predict recidivism in 60-70% of cases using only LS/CMI scores. [Follow up period of 24 months]

[Slide 5 Research update YLS/CMI]

Good to excellent inter-rater reliability was reported for the YLS/CMI by a number of researchers in both UK and international studies (Onifade et al. 2008a; Rennie and Dolan 2010; Schmidt et al. 2005; Vieria et al. 2009; Welsh et al. 2008). [ICC range of .72 to .95; subscales ranged from .61 to .85 for Schmidt et al. 2005 study. Follow up periods of 12 months, 12 months, 42 months, 14-66 months, 7-61 months respectively.]

3

• A number of researchers in the UK and a study in Singapore have found the YLS/CMI accurately predicts recidivism of varying types (any, serious violence, non-violent) (Chu et al. 2015; Marshall et al. 2006; Olver et al. 2012; Rennie and Dolan 2010; Vaswani and Merone 2012). [AUC ranged from .61 to 85; although it was non-significant at 0.54 for Vaswani and Merone’s (2012) study. Follow up period of 37.5-97 months, 12 months, 84 months, 12 months, 12 months respectively.]

• A Scottish study found that the tool accurately predicted recidivism, future serious violence, the volume of future offending and the speed of future offending (Vaswani 2013). [follow up period of 12 months.]

[Slide 6: Equality issues, why review?]

• Social characteristics, such as gender, learning disabilities and so forth, can influence offending behaviours and also how an individual responds to treatment.

• The 2010 UK Equality Act prescribes a list of ‘protected characteristics.’ These are age, gender, ethnicity, mental disorders, gender realignment, religion or lack thereof, marriage/civil partnership and sexual orientation.

• Working within the public sector, we have a duty to advance equality of opportunity. In that sense, equality is not about treating everyone in the same way; rather, it is about recognising differences and treating people accordingly so the outcome is the same for everyone.

• With risk assessment tools, we have to ensure that we do not indirectly discrimination against people with protected characteristics. Characteristics particularly relevant to offending are thought to be gender, ethnicity, mental disorders – those are the ones that will be covered here. The remaining characteristics within the UK Equality Act (2010) are highly unlikely to factor into considerations of criminality; although I will look out for such studies. Later on in this presentation, I will briefly touch on a study involving religion in the LS/CMI assessment process.

[Slide 7: Equality issues gender]

Age will be covered throughout the next three slides with the comparison of both the adult and youth versions of the LS/CMI.

Firstly, looking at gender. The developers of the LS/CMI claimed it provides a ‘gender-informed’ assessment for risk, needs and responsivity issues.

Studies have found LS/CMI accurately predicts recidivism for female offenders (Andrews et al. 2012; Hogg 2011; Rettinger and Andrews 2010). [AUC range of .69 through to 87. Follow up periods of 12-50 months, 48 months, 57 months respectively.] For instance, a study of female prisoners found those marked as high risk were responsible for 74% of all new offences (Rettinger and Andrews 2010).

• An interesting finding of a study carried out in Pakistan was that females scored higher on the general risks/needs scores. The authors suggested this could be the result of a cultural or gender bias towards female offenders (Bhutta & Wormith 2016).

4

• Previous empirical research on the YLS/CMI has indicated that it is able to predict recidivism across genders (Anderson et al. 2016; Bechtel et al. 2007; Campbell et al. 2014; Jara et al. 2016; Marshall et al. 2006; Olver et al. 2012; Onifade et al. 2008; Vaswani and Merone 2012) [AUC range of 0.565 to 0.75. Follow-up periods of 24 months, 41 months, 24 months, 24 months, 12 months, 84 months, 12 months and 12 months respectively.] [for a meta-analysis of studies, interested readers are directed to Pusch and Holtfreter 2018]. Anderson et al. (2016), however, found that only family and personality subscales significantly predicted for girls, whereas all the subscales did for boys. [AUC of .565 for girls and .623 for boys. Follow-up period of 24 months.]

• It was found that gender played a significant role in affecting recidivism risk levels: females with a low risk level had a higher risk of recidivism than boys (Jara et al. 2016).

[Slide 8 – equality issues: ethnicity]

The LS/CMI demonstrated predictive accuracy across various ethnic groups in a range of international studies (Gutierrez et al. 2013; Olver et al. 2012; Wormith et al. 2015). [Follow up periods of not available, 84 months, 60 months respectively.]

For instance, a study in Australia found it accurately predicted for Aboriginal and non-Aboriginal individuals (Wormith et al. 2015). [AUC of .64 for Aboriginals and .74 for non-Aboriginals. Follow-up period of 60 months.] It was recommended that assessors adapt interviews to the needs of individuals, e.g. being mindful of cultural heritage, jargon and communication styles.

• Researchers in Spain, Australia and the United States have found the YLS/CMI to have predictive accuracy across different ethnic groups (Baird et al. 2013; Cuervo and Villanueva 2017; Olver et al. 2012; Onifade et al 2008b, 2009). [AUC of .62 to .67. Follow-up periods of 12 months, 24 months, 84 months, 12 months, 24 months respectively.]

• A handful of studies have highlighted issues: the YLS/CMI did not perform as well for African Americans and Latinos as Caucasians in a U.S. study (Baird et al. 2013); in a Spanish study, ethnic minorities were less likely to indicate a higher level of reoffending (Cuervo & Villanueva 2017).

[Slide 9: mental disorders]

This refers to learning disabilities and issues that could cause mental incapacity.

General risk/need scores showed low to moderate correlation for any convictions, violent convictions and non-violent convictions (Girard and Wormith 2004). [AUC range of .18 to .36. Follow up period of 31 months.]

Andrews (1995) found that individuals with mental disorders categorised as high/very high risk reoffended at a rate of 73% compared to 17% for those marked as low risk. [Follow-up period not available.]

Good predictive accuracy was achieved in YLS/CMI for individuals with mental disorders for all offending (any, violent and non-violent) (Rennie & Dolan 2010). [AUC range of .60 to .67. Follow-up period of 12 months.]

5

Not many studies in this area.

[Slide 10 – equality: other factors]

What about the other protected characteristics in the Equality Act? As mentioned earlier, other protected characteristics are unlikely to be relevant to offending.

Having said that, the transgender policy expert in the Scottish Government will visit the RMA soon to update us on how this characteristic could influence offending, so it may be the case that this is something that becomes relevant when considering risk assessment in future. At the moment, however, there is a gap in the research, with studies focusing on the factors I have covered here today: age, gender, ethnicity and mental disorders.

There was also a recent study examining the impact of religion on the predictive accuracy of the LS/CMI. A religious measure was added to the tool in a study carried out in Pakistan, an Eastern, highly devout Muslim country, since the LS/CMI currently does not measure religiosity or spirituality. The researchers concluded that adding a religiosity or spirituality measure to the LS/CMI is unlikely to improve its predictive abilities (Bhutta and Wormith 2016).

[Slide 11 – override feature]

• Override feature in both the LS/CMI and its youth version allows assessors to override the initial risk level based on strengths and specific risk factors they hold about the offender. In this case, professional judgment can surpass the scoring of the tool.

• A study of 3646 offenders in Quebec found the override was used in 4% of ‘upward’ (i.e. increased level of threat) LS/CMI cases and 2% of downward ones (i.e. lowering the risk level). Of the two, upward overrides had greater accuracy over downward ones; however, using the feature in either case was found to reduce predictive power (Guay and Parent 2017) .

• Studies about the YLS/CMI discovered the feature reduced the accuracy of the tool to predict recidivism, particularly in serious violence cases (Schmidt et al. 2016; Vaswani 2013).

[slide 12 – potentially problematic items]

Gordon and his colleagues (2015) highlighted that five of the items within the LS/CMI could be considered ‘double-barrelled’ questions: for instance, asking about both youth and adult criminal history ion the same item. It is, thus, suggested that these items are separated to allow them to be adequately measured.

The ‘could make better use of time’ factor within both tools was criticised for being a subjective items that is difficult to reliably score (Baird et al. 2013).

• Within the YLS/CMI, there is the options to check for ‘occasional drug use’ and ‘chronic drug use’ in the substance abuse item. Although these would appear to be mutually exclusive, both of these are to be selected when the ‘chronic drug use’ is checked. This is something

6

which happens automatically in automated versions of the YLS/CMI; yet, it is not consistently applied when the scoring is carried out manually, leading to errors.

[Slide 13 – concluding thoughts]

As discussed here today, research into the LS/CMI and YLS/CMI is on-going and international in nature.

• The LS/CMI can be used with ages 16 and over; whilst the YLS/CMI can be used up to age 18. Consider the maturity of the individual before deciding which tool to use when there is the overlap of between 16-18.

• Equality issues. Gender, ethnicity and mental disorders are the factors most likely to have an impact. Although the tools are generally still predictive of recidivism, this is something to consider when conducting risk assessments.

• Research indicates that the override feature should be used cautiously and be supported by strong evidence (Schmidt et al. 2016; Vaswani 2013).

• Studies have also highlighted potential issues: there are problematic items in the LS/CMI (double-barrelled items); both tools have the subjective item of ‘could make better use of time’; the dual option selection of chronic and occasional drug use in YLS/CMI.

[Slide 14 – interested?]

• Copies of presentation notes are included in today’s packs.

• A reference list is included for those wishing to pursue further reading.

• RATED is available here: http://rated.rmascotland.gov.uk/ Watch out for the update on the RMA’s new website!

• Any questions about research relating to the LS/CMI and the YLS/CMI please email me.

7 Reference List

-Anderson, V. R., W. S. Davidson II, A. R. Barnes, C. A. Campbell, J. L. Peterson and E. Onifade.

(2016) ‘The differential predictive validity of the Youth Level of Service Case Management

Inventory: the role of gender.’ Psychology, Crime and Law 22(7), 666-677.

-Andrews, D. A. and J. Bonta. (1995). LSI-R: The Level of Service Inventory-Revised. Toronto,

Ontario: Multi-Health Systems.

-Andrews, D. A, L. Guzzo, P. Raynor, R. C. Rowe, L. J. Rettinger, A. Brews and J. S. Wormith.

(2012) ‘Are the Major Risk/Need Factors Predictive of Both Female and Male Reoffending? A Test

With the Eight Domains of the Level of Service/Case Management Inventory.’ International Journal

of Offender Therapy and Comparative Criminology 56(1), 113-133.

-Baird, C., T. Healy, K. Johnson, A. Bogie, E. W. Dankert and C. Scharenbroch. (2013) A

comparison of risk assessment instruments in juvenile justice. U.S. Department of Justice, National

Council on Crime and Delinquency.

-Bhutta, M. H. and J. S. Wormith. (2016) ‘An examination of a risk/needs assessment instrument

and its relation to religiosity and recidivism among probationers in a Muslim culture.’ Criminal

Justice and Behavior 43(2), 204-229.

-Chu, C. M., H. Yu, Y. Lee and G. Zeng. (2015) ‘The utility of the YLS/CMI-SV for Assessing Youth

Offenders in Singapore.’ Criminal Justice and Behavior 41(12), 1437-1457.

-Cuervo, K. and L. Villanueva. (2017) ‘Prediction of recidivism with the Youth Level of Service/Case

Management Inventory (Reduced Version) in a sample of young Spanish offenders.’ International

Journal of Offender Therapy and Comparative Criminology

-Girard, L. and J. S. Wormith. (2004). ‘The predictive validity of the Level of Service Inventory–

Ontario Revision on general and violent recidivism among various offender groups.’ Criminal Justice

and Behavior 31, 150-181.

-Heidi Gordon, Sally F. Kelly and Roberta Julian. (2015) ‘Psychometric evaluation of the level of

service/case management inventory among Australian offenders completing community-based

sentences.’ Criminal Justice and Behavior 42(11), 1089-1109.

-Jean-Pierre Guay and Genevieve Parent. (2017) ‘Broken Legs, Clinical Overrides and Recidivism

Risk: An Analysis of Decisions to Adjust Risk Levels with the LS/CMI.’ Criminal Justice and

Behavior 45(1), 82-100.

-Gutierrez, L., H. Wilson, T. Rugge and J. Bonta. (2013). ‘The prediction of recidivism with

Aboriginal offenders: A theoretically-informed meta-analysis.’ Canadian Journal of Criminology and

Criminal Justice, 55, 55-99.

Hogg, S. M. (2011). The Level of Service Inventory–Ontario Revision Scale validation for gender

and ethnicity: Addressing reliability and predictive validity (Unpublished master’s thesis). University

of Saskatchewan, Saskatoon, Canada.

Jara, P., A. Garcia-Gomis and L. Villanueva. (2016) ‘Impact of type of intervention on youth

reoffending: are gender and risk level involved?’ Psychiatry, Psychology, Law 23(2), 215-223.

8 Labrecque, R. M. C.r M. Campbell, J. Elliott, M. King, M. Christmann, K. Page, J. McVay and K.

Roller. (2017) ‘An examination of the inter-rater reliability and rater accuracy of the level of

service/case management inventory.’ Corrections: Policy, Practice and Research, 1-14.

Marshall, J., V. Egan, M. English and R. Jones. (2006). ‘The relative validity of psychopathy versus

risk/needs based assessments in the prediction of adolescent offending behaviour.’ Legal and

Criminological Psychology, 11, 197–210.

Olver, M., k. Stockdale and S. Wong. (2012). ‘Short and Long-Term Prediction of Recidivism Using

the Youth Level of Service/Case Management Inventory in a Sample of Serious Young Offenders.’

Law and Human Behavior, 36(4), 331-344.

Onifade, E., W. Davidson, S. Livsey, G. Turke, C. Horton, J. Malinowski, D. Atkinson, D. Wimberly.

(2008a) ‘Risk assessment: Identifying patterns of risk in young offenders with the Youth Level of

Service/Case Management Inventory.’ Journal of Criminal Justice 36 (2), 165-173.

Onifade, E., W. Davidson, C. Campbell, G. TurkeIngham, J. Malinowski, K. Turner. (2008b)

‘Predicting Recidivism in Probationers With the Youth Level of Service Case Management Inventory

(YLS/CMI).’ Criminal Justice and Behavior 35(4), 474-483.

Onifade, E., D. William and C. Campbell. (2009). ‘Risk Assessment: The Predictive Validity of the

Youth Level of Service Case Management Inventory with African Americans and Girls.’ Journal of

Ethnicity in Criminal Justice 7, 205-221.

Pusch, N. and K. Holtfreter. (2018) ‘Gender and Risk Assessment in Juvenile Offenders: A Meta-

Analysis.’ Criminal Justice and Behavior 45(1), 56-81.

Rennie, C. E. and M. Dolan. (2010). ‘Predictive validity of the Youth Level of Service/Case

Management Inventory in a custody sample in England.’ Journal of Forensic Psychiatry and

Psychology 21, 407-425.

Rettinger, L. J. and D. A. Andrews. (2010). ‘General risk and need, gender specificity, and the

recidivism of female offenders.’ Criminal Justice and Behavior, 37, 29-46.

Schmidt, F. S. M. Sinclair and S. Thomasdottir. (2016) ‘Predictive validity of the youth level of

service/case management inventory with youth who have committed sexual and non-sexual

offences.: the utility of professional override.’ Justice and Behavior 43(3), 413-430.

Schmidt, F., R. D. Hoge and L. Gomes. (2005) ‘Reliability and validity analysis of the Youth Level of

Service/Case Management Inventory.’ Criminal Justice and Behavior 32(3), 329-344.

Spence, D. and S. Haas. (2015) The Predictive Utility of Risk and Needs Assessment. West

Virginia, Department of Military Affairs and Public Safety.

Vaswani, N. (2013) The use of the Youth Level of Service/Case Management Inventory (YLS/CMI)

in Scotland. Glasgow, Scotland: Centre for Youth and Criminal Justice.

Vaswani, N. and L. Merone (2012) ‘Are there risks with risk assessment? A study of the predictive

accuracy of the youth level of service case management inventory with young offenders in

Scotland.’ The British Journal of Social Work 44(8), 2163-2187.

9 Vieira, T., T. Skilling and M. Peterson-Badali, (2009) ‘Matching Court-Ordered Devices with

Treatment Needs: Predicting Treatment Success with Young Offenders.’ Criminal Justice and

Behaviour 36, 385-401.

Welsh, J., F. Schmidt, L. McKinnon, H. Chattha and R. Meyers. (2008). ‘A comparative study of

adolescent risk assessment instruments predictive and incremental validity.’ Assessment,

15(1),104–15.

Wormith, J. S., S. M. Hogg and L. Guzzo. (2015) ‘The Predictive Validity of the LS/CMI with

Aboriginal Offenders in Canada.’ Criminal Justice and Behaviour 42 (5), 481-508.