Evaluation of faMily SMilES - NSPCC€¦ · Children on Family SMILES go to a group with other...

May 2014

Impact and Evidence series

Evaluation of faMily SMilES

intEriM rEportRachel Cass and Prakash Fernandes

NSPCC Evaluation department

Impact and Evidence seriesThis report is part of the NSPCC’s Impact and Evidence series, which presents the findings of the Society’s research into its services and interventions. Many of the reports are produced by the NSPCC’s Evaluation department, but some are written by other organisations commissioned by the Society to carry out research on its behalf. The aim of the series is to contribute to the evidence base of what works in preventing cruelty to children and in reducing the harm it causes when abuse does happen.

3Impact and Evidence series

Contents

ACKNOWLEDGEMENTS 5

KEY FINDINGS: YOUNG PEOPLE’S VERSION 6

KEY FINDINGS 8

ExECUTIVE SUMMARY 10

MAIN REPORT 16

Chapter 1: Introduction 161.1 Background 161.2 Method 181.3 Evaluation measures 191.4 Sample of children and parents for

qualitative interviews 201.5 Ethics 221.6 Analysis 231.7 Limitations of the Research 23

Chapter 2: Increasing self-esteem 252.1 Overall increase in children’s self-esteem:

Adapted Rosenberg scale 252.2 Factors that contributed to improving

self-esteem amongst children 26

Chapter 3: Decreasing children’s emotional and behavioural problems 33

3.1 Overall decrease in emotional and behavioural difficulties: children’s and parents’ perspectives 33

3.2 Change in level of difficulties experienced by children 34

Evaluation of Family Smiles4

3.3 Change in children’s emotional and behavioural difficulties: practitioners’ perspectives 35

3.4 Examples of decreased emotional and behavioural problems 36

3.5 Examples of increased levels of behavioural and emotional difficulties 37

Chapter 4: Enhancing protective parenting 394.1 Overall reduction in behaviours that

contribute to risk: CAPI data 394.2 Factors affecting a change in parents’

protective behaviours 41

Chapter 5: Improving family communication about mental health 52

5.1 Changes in family communication 525.2 The role of the programme in

increasing communication 54

Chapter 6: Conclusion 606.1 Suggestions for improvement 616.2 Next steps 62

Bibliography 64

Appendices 66Appendix 1: Overview of sessions 66Appendix 2: Inclusion criteria 70Appendix 3: Family SMILES with standardised measures 72Appendix 4: Interview schedule for children 74Appendix 5: Interview schedule for parents 76Appendix 6: Ethics overview 78Appendix 7: Statistical analysis of standardised measures 81Appendix 8: Framework themes and sub-themes and snapshot from NVivo 83


aCKnoWlEDGEMEntSThe authors would like to thank:

• The children and parents who consented to take part in the evaluation, for giving their time to complete questionnaires through the programme.

• Children and parents who consented to take part in the interviews and share their perspectives and insights about the programme.

• Team managers, practitioners and administrative staff at the Family SMILES service centres in Belfast, Bristol, Coventry, Grimsby, Ipswich, Jersey, Manchester, Middlesbrough, Prestatyn and Southampton, who support the continuing implementation of the evaluation by explaining its importance to children and parents and ensuring that data is entered onto the system for analysis.

• Members of the Family SMILES Commission Delivery Group for their ongoing advice, suggestions, encouragement, and for promoting the evaluation within the service.

• Matt Barnard and Richard Cotmore, for reviewing the evaluation plans and reports.


KEy finDinGS: younG pEoplE’S vErSionFamily SMILES (Simplifying Mental Illness plus Life Enhancement Skills) is a service for parents and children from families where the parent has a mental illness. There are lots of different types of mental illnesses. Some might make the person feel scared and others might make it hard for the person to tell what is real and what is not.

Children on Family SMILES go to a group with other children who have a mum or dad with a mental health problem. Mums and dads get help from an NSPCC worker. At the end children and parents tell each other about what they learnt. They also make a safety plan that tells them how to keep safe at home.

The NSPCC did some research to find out what families who used the service thought of it. They asked children and parents to answer questionnaires. They also did interviews with children and parents. These are the main things that they found:

• Children said that they felt more confident and happier about themselves after going to the group. They thought that they had fewer problems at home, at school and with their friends.

• Some children said that they found it helpful to listen to other children in the group. They thought that it felt good to share their feelings with the other children. Some also said that making new friends in the group and taking part in fun activities helped them to feel better. They learnt that if your mum or dad has a mental illness it is not your fault.


• Mums and dads said that they felt happier, more relaxed and less worried at the end of Family SMILES. They thought that Family SMILES had helped them to find ways to talk to their child about their mental illness. They also thought that it had helped them to get on better with their children. Parents said that the workers from the NSPCC were very helpful.

• Some mums and dads said that nothing much had got better for them. This might have been because they were too unwell to take part. Some thought it was because not all the family had wanted to take part in the first place. Others thought that if everyone from their family had taken part it would have made things better at home.

The NSPCC is going to ask more people to answer questionnaires and take part in interviews about Family SMILES. A final report will be written with all the new information later in 2014.


KEy finDinGSFamily SMILES (Simplifying Mental Illness plus Life Enhancement Skills) is a group work intervention for children (8 to 13 years) with a parent who has a serious mental health problem, where the child is at risk of harm. This interim evaluation report assesses changes for children and parents based on the standardised measures they completed before the start of the programme and then again at the end. The report also highlights aspects of the programme that families found either helpful or not, through analysis of the qualitative interviews with children and parents who completed the programme.

The key findings from this report are:

• Children and young people reported an increase in their self-esteem and a decrease in their emotional and behavioural problems at the end of the programme. At the start of the programme half of all children reported clinical level of difficulties, but by the end of the programme only a third still reported clinical level of difficulties.

• Parents reported a decrease in their level of distress and unhappiness and increase in their self-esteem. Research indicates that the positive change in these factors will contribute towards enhancing parents’ protective behaviours.

• The factors that families identified as contributing towards achieving positive outcomes included: a positive group work experience for children; the positive relationship with the NSPCC staff; the ability of the parent to accept the impact of their behaviour on their child and gaining an insight into how their behaviour impacts on their child; the ability of children to express empathy with the


parent, and the perceived positive understanding of mental health by both children and parents.

• The factors that families highlighted as barriers towards them achieving the change they desired included: parent struggling to cope with the mental health problem; no perceived change in the child’s behaviour at home; lack of engagement from the entire family; inability of the programme to address family dynamics.

• The evaluation does not currently include a comparison group, so further research is required to be confident that the improvements in outcomes are a direct result of the Family SMILES programme.

The continuing evaluation study will aim to resolve some of the limitations of this interim report by having more robust quantitative data from parents, providing data from the comparison group and interview data from NSPCC practitioners and referrers.


ExECutivE SuMMary

Background

Family SMILES (Simplifying Mental Illness plus Life Enhancement Skills) is a group work intervention for children (8 to 13 years) with a parent who has a serious mental health problem, where the child is at potential risk of harm. Family SMILES aims to reduce the risk of harm to these children by improving children’s self-esteem, resilience and life skills; helping parents understand the impact of their mental health problem on their child, and improving protective parenting skills. The model involves eight group work sessions with children and six individual sessions with parents. A personal safety plan is drawn up with the family to ensure that, in the event of parental relapse, the child will be adequately protected and cared for, and that their wishes will be considered. The NSPCC runs the Family SMILES programme in Belfast, Bristol, Grimsby, Ipswich, Jersey, Manchester, Middlesbrough, Prestatyn and Southampton.1

This interim evaluation report is based on evaluation data collected since the project began in September 2011, up to August 2013.

Methodology and aims

The evaluation of Family SMILES sought to evidence whether the following key outcomes were being achieved: increased self-esteem among children/young people; reduced emotional and behavioural difficulties for

1 The Family SMILES programme was implemented in Coventry from September 2011 to April 2013. It began in Ipswich and Jersey in early 2014 and hence this report does not include any evaluation data from these two sites.


children/young people, and enhanced protective parenting behaviour. It was expected that these changes would strengthen the parent-child relationship by improving communication about mental health within the family, and contribute towards keeping children safer. The key elements of the evaluation design include:

• Pre- and post-design, where a series of measures are completed at the start of the work, at the end of the work and then again six months later, in order to understand longer-term changes. These measures gather quantitative data from the perspective of children, parents and practitioners (full details of all measures can be found in Appendix 3). This report includes data from the start and end of the programme, but not six months post-completion.

• Qualitative interviews with a sample of children and parents at the end of the work to understand the factors that helped or hindered them in achieving outcomes (included in this report).

• Qualitative interviews with a sample of practitioners to understand the processes that may have helped children and parents achieve outcomes (not included in this report).

• Qualitative interviews with a sample of referrers to understand their perceptions of the programme (not included in this report).

Key findings

Increase in children/young people’s self esteem

Children and young people reported an increase in their self-esteem by the end of the programme. The factors that acted as facilitators or barriers towards increasing self-esteem included: listening to their


peers; sharing their own experience; learning about mental health and building friendships among the group.

Decrease in young people’s emotional and behavioural difficulties

Findings on changes in difficulties for children indicate an overall improvement in this area by the end of the programme as measured by the Strengths and Difficulties Questionnaire (SDQ) (children’s and parents’ perspectives). There is a statistically significant reduction in the proportion of children in the ‘clinical’ level of difficulty as they move into the ‘normal’ range at the end of the Family SMILES programme.

Enhancing parent’s protective behaviours

Parents perceived a positive change in their parenting at the end of the programme. The data from the Child Abuse Potential Inventory (CAPI) suggests that at the end of the programme, parents reported that their level of distress and unhappiness had reduced and that they were feeling better about themselves. The key facilitators for these changes were: gaining an insight into the impact of their mental health problems on the children; better understanding about their children’s needs; the positive relationship with the NSPCC practitioner, and effective safety/support plans developed during the joint sessions. Change in protective parenting was less evident to parents where they felt they were still not coping with their mental health problems and where they perceived their child(ren) to be entirely responsible for the ‘problematic behaviour’, which suggests that they did not fully accept the impact of their own behaviour on the children.


Improving communication about mental health

The qualitative data indicate that children and parents valued the role of the Family SMILES programme in strengthening communication within the family. They reported that the programme had facilitated this by jointly working with the child and the parent, enabling both to reflect on perceptions of mental health problems and work on overcoming resistance to talk about it. The barriers families experienced in achieving this related to the importance of getting all family members at a similar level of communication.

Implications

Relevance of Family SMILES

This report contains promising evidence of positive change for children and parents at the end of the Family SMILES programme. The aspects of the programme that parents and children valued particularly was that it increased their understanding about the impact of parental mental health on the family and facilitated communication within the family. This suggests that the Family SMILES programme may be bridging a gap between adult mental health and children’s services by creating a safe space for parents and children to explore the impact of mental health on the family. Further data through the comparison group will enable us, with greater confidence, to understand the role of the programme in achieving these changes.


Structure of the programme

Participants reflected that the current structure of the intervention – giving children their own space to discuss issues in a group work setting; working with the parent individually for them to process their thinking, and working jointly with possibly the entire family – were all important elements of the programme.

Participants suggested that work with the entire family (not just the parent with the mental health problem and children aged between 8 and 13 years) would also support more sustainable, positive outcomes. They emphasised the importance of matching children of similar ages and having a good gender balance within the group. It was also highlighted that, since participants might have differing expectations of what joint working would involve and achieve, there may be a need for discussions to take place beforehand in order to clarify what participants hope to get out of the joint work.

Delivery of the programme

Parents, children and practitioners commented on the flexibility of the programme in terms of children having the opportunity to do one-to-one sessions alongside the group work; parents having the space to talk about issues currently affecting them, and the additional support provided by NSPCC practitioners in supporting families – especially those where children were on a child protection plan. This degree of flexibility and stepping outside the programme may need to be acknowledged and supported when equipping newer teams to deliver the programme. Additionally, any transfer of the Family SMILES model needs to take into


account the engagement skills and approach of NSPCC workers that was highly valued by parents and children.

Next Steps

The NSPCC is exploring how it can apply the learning from the interim evaluation to enable the service to have a greater impact on children and their families. The scope of this work includes engaging with local authority commissioners and other organisations working with this group of families. The aim is to identify changes to the content of both the programme and also the settings in which the programme might be delivered to make it more effective.

The ongoing evaluation will aim to resolve some of the limitations of this interim report by collecting more robust quantitative data especially from parents; data from the comparison group; more qualitative data from parents and children – especially those who may have dropped out – as well as interview data from NSPCC practitioners and referrers.


Main rEport

Chapter 1: IntroductionThis interim evaluation report is based on the evaluation data collected since the project began in September 2011 to August 2013. This introductory chapter summarises the Family SMILES (Simplifying Mental Illness plus Life Enhancement Skills) programme and describes the evaluation design.

1.1 Background

It is estimated that one in six adults experience significant mental health problems at any one time (ONS, 2001). Although parental mental health problems do not necessarily indicate poor parent-child relations, they are a fairly common element in child protection investigations (Prokop et al, 2010). Negative effects of parental mental health problems on children may include low self-esteem, stress, missing school, behavioural disturbances, inappropriate sexual behaviour, and the development of mental health problems such as eating disorders or depression (Aldridge, 2006; Stallard et al, 2004; Tunnard, 2004).

A systematic review of the clinical effectiveness, cost-effectiveness and acceptability of community-based interventions aimed at improving or maintaining quality of life for children of parents with serious mental health problems highlighted the lack of evidence that could provide an adequate rationale on which to base UK service development or delivery (Bee et al, 2014). The context of limited evidence from other interventions for this cohort of service users highlights a wider challenge


in understanding the scale of change for Family SMILES service users.

Family SMILES is a group work intervention for children with a parent who has a serious mental health problem, where the child is at risk of potential harm. It aims to reduce the risk of harm to children living with parents who have mental health problems and improve their self-esteem, resilience and life skills. It aims to help parents understand the impact of their mental health problems on their child and improve protective parenting skills.

Family SMILES is based on the Australian SMILES programme, developed in 1997 by Erica Pitman (Pitman, 2010). The Australian SMILES originally involved group work with young carers who have a parent or sibling with a mental health problem. It has been used in other countries, including Canada, and positively evaluated in a number of reports (Pitman and Matthey, 2002; Baldwin and Glogovic, 2010). Family SMILES is the NSPCC’s adaptation of the original SMILES intervention, taking a more family-based approach (working with both the child and parent) and focusing on child protection issues. It is targeted at children who are vulnerable and have been identified as being at risk due to their parents’ mental health problems. Appendix 1 provides an overview of the sessions with children and parents. These children may be living at home or placed elsewhere, but will have ongoing contact with the ill parent. After receiving a referral, the NSPCC carries out a four-week assessment with the family to check their eligibility for the service. Appendix 2 includes the inclusion criteria for the service.


The Family SMILES model involves group work with children and individual sessions with parents. There are eight group sessions, held over an eight-week period, with a maximum of eight children (8-13 years old) per group, combining play, creative activities, discussions and age-appropriate education about mental health and life skills (eg, what to do in an emergency) to help them cope more effectively. Family SMILES practitioners will also run up to six individual sessions with parents to help them understand the impact of their mental health problems on their child, and to strengthen the parent-child relationship. If this is not possible, (eg, if the parent is too ill) they will work with the child’s ‘well’ parent or carer. A personal safety plan will be drawn up with the family to ensure that, in the event of parental relapse, the child will be adequately protected and cared for, and that their wishes will be considered.

The NSPCC runs the Family SMILES programme in Belfast, Bristol, Grimsby, Ipswich, Jersey, Manchester, Middlesbrough, Prestatyn and Southampton.2

1.2 Method

Family SMILES is evaluated using a mixed-method design that includes a pre-test and a post-test element to examine the extent to which the programme’s intended outcomes for children and parents are achieved. In addition, this report also analyses the qualitative data from interviews with children and parents who completed the programme. The aim of the qualitative interviews

2 The Family SMILES programme was implemented in Coventry from September 2011 to April 2013. It began in Ipswich and Jersey in early 2014 and hence this report does not include any evaluation data from these two sites.


was to identify the facilitators and barriers to service users achieving positive outcomes, and to understand their experience of the programme. A total of 15 interviews with children and 16 interviews with parents/partners/carers were undertaken.

1.3 Evaluation measures

Children and parents were asked to complete evaluation measures for Time 1, at the end of the four-week assessment period (prior to the children starting their group work and the parents beginning individual work). The evaluation measures at Time 2 were completed at the end of the joint session with the child and their parent. Where it was not possible to undertake the joint work with parent and child, the Time 2 measure was completed either at the end of the final peer group session for children, or the final individual session with the parent. The outcomes of the programme and the evaluation measures used are outlined in Table 1. Details of each of these measures are discussed in Appendix 3.

Table 1: Overview of pre- and post-measures and number of completed questionnaires Outcome Tool Perspective Number of

Time 1 and Time 2

To enable children and young people to feel better about themselves

Adapted Rosenberg Self Esteem Scale

Children’s evaluation wheel (not included in this report)

Child

Child

61

63

To reduce children and young people’s emotional and behavioural issues

Goodman’s Strengths and Difficulties Questionnaire (SDQ)

Health of the Nation Outcome Scales (HoNOSCA)

Child/Parent

Practitioner

72

47

To enhance parents’ protective parenting/to improve safeguarding of children

Child Abuse Potential Inventory (CAPI)

Parents evaluation wheel (not included in this report)

Parent 22


1.4 Sample of children and parents for qualitative interviews

Children and parents were selected purposively. Purposive sampling is used to enable the evaluation to describe and understand the full range of views and experiences within the study population. It involves setting quotas using criteria based on dimensions that reflect key differences in the study population that are relevant to the study’s objectives, rather than trying to ensure that the sample is statistically representative.

Our key criterion in choosing children or parents to be interviewed was whether they perceived an improvement (on one or more of the standardised measures) or not. In addition we included: children and parents who had refused consent to complete evaluation measures, and partners or carers, to understand their experience of the programme. While we attempted to get equal numbers across all categories, this was not possible due to difficulties in contacting children for whom there had not been an improvement (according to one or more standardised measures), or parents who had not completed the evaluation measures.

Table 2: Numbers of children and parents interviewed based on pre-/post- changePre-/Post- change reported on measures Children Parents

Improved 9 3

Same/Got worse 3 2

Refused consent to complete measures 3 7

Partner/Carer NA 4

Total 15 16


Children were categorised by age (either younger or older than 10) and by gender; parents by the type and severity of mental health problem: either severe, which included schizophrenia and schizo-affective disorder, psychosis, borderline personality and personality disorder, severe depression; or non-severe, being defined as (non-severe) depression/anxiety – though it should be emphasised that these are still recognised as significant mental health problems. These two categories of mental health problems were used for sampling purposes, as it was hypothesized that they may have a bearing on service users’ experiences of the programme.

Table 3: Age and gender of interviews with children

8 years to 10 years 11 years to 13 years Total

Girls 3 6 9

Boys 1 5 6

Total 4 11 15

The range of conditions experienced by the parents is outlined in Table 4, which indicates that 7 of 14 families who were interviewed had parents with a serious mental health problem.

Table 4: Nature of parental mental ill healthDiagnosis Number

Bipolar 3

Personality disorder 3

Severe depression 1

Depression/Anxiety 6

No specific diagnosis 1

Finally, the sample was selected across different locations across the UK, as outlined in Table 5


Table 5: Distribution of interviews according to location Location Number of

childrenNumber of Parents/Carers

Number of families

Belfast 4 4 3

Bristol 1 1 1

Middlesbrough 3 3 3

Prestatyn 4 3 2

Southampton 3 5 5

Total 15 16 14

The interviews with each child and parent were done individually (except for one interview in Southampton that was done jointly by the child and mother) and lasted approximately 45 minutes. The interview schedules are attached in Appendix 4 (children) and Appendix 5 (parents).

1.5 Ethics

The key ethical considerations that influenced the evaluation included ensuring that service users could give informed consent; that confidentiality and its limits were understood by participants; that they were aware of their option to withdraw from the evaluation or any aspect; that the principal of no harm to participants as a result of the evaluation was kept in mind while explaining measures or conducting the interviews; that participants had access to advice or support related to the evaluation and participants, and that practitioners and the evaluation officers had access to debrief sessions to process any concerns raised through the evaluation. A note on the ethical considerations is provided in Appendix 6.

Prior to the study commencing, it was approved by the NSPCC’s Research Ethics Committee (REC). The REC includes members from senior NSPCC staff and external professional experts. This ethics governance procedure is in line with


the requirements of the Economic and Social Research Council (ESRC, 2005) and Government Social Research Unit (GSRU, 2005) Research Ethics Frameworks3.

1.6 Analysis

The responses to the evaluation measures were analysed using a range of statistical tests to determine whether or not the changes were due to chance. It is conventional to use the level of 5 per cent for statistical significance: a change is considered statistically significant if there is less than a 5 per cent chance of it happening randomly. This level is used in this report, and the changes that are statistically significant are reported. Further information about the analysis of measures is outlined in Appendix 7. The validity indicators of the CAPI were used to exclude questionnaires completed by parents that suggested they were answering in a socially desirable or inconsistent manner. The qualitative data from the child and parent interviews and the focus groups were analysed using a framework ‘case and theme’ approach. The list of themes used is attached in Appendix 8.

1.7 Limitations of the Research

This interim study currently does not include any comparison group data and is therefore unable to directly attribute change to the Family SMILES programme. The ongoing evaluation will aim to address this gap with the introduction of the naturally occurring ’waiting to start’ comparison group. The numbers of evaluation returns from parents (CAPI or parents evaluation wheel) are limited and hence need to be interpreted with

3 www.esrc.ac.uk/about-esrc/information/research-ethics.aspx

http://www.esrc.ac.uk/about-esrc/information/research-ethics.aspx


caution. The final report, in addition to including greater numbers of completed measures, will include the perspective of practitioners and referrers to understand their perspective of change for families as well as the role of the Family SMILES service.


Chapter 2: Increasing self-esteem A key objective of Family SMILES was to improve levels of self-esteem among children. This chapter explores how levels of self-esteem among children changed between the beginning and the end of the programme. It examines the data from the standardised measure and explores the key factors that contributed to changes in children’s self-esteem.

2.1 Overall increase in children’s self-esteem: Adapted Rosenberg scale

The evaluation used the Adapted Rosenberg Self-Esteem Questionnaire to measure changes to children’s self-esteem. It has been adapted for children, and there are no cut-off points defining high or low levels of self-esteem. This measure, however, enables trends of increasing or decreasing self-esteem to be identified. The average score from the measure before the programme was 19.2 and this increased by a small but statistically significant amount to an average of 21.4 at the end of the programme. Scores were out of a possible total of 30.


Figure 1: Change in mean scores pre- and post-programme for the Adapted Rosenberg Self-Esteem Questionnaire data (n=61)

The analysis of this data indicated that children’s self-esteem had improved by a small but statistically significant amount.

2.2 Factors that contributed to improving self-esteem amongst children

The key factors identified as having a role in either enabling children’s self-esteem to improve or acting as barriers are outlined below.

Listening to other children facing similar challenges

The peer groups offered an opportunity for children to listen to the experiences of other children living with a parent who had a mental health problem. For some children, this emerged as a particularly helpful function of the group experience as it enabled them to acknowledge that they were not the only child dealing with issues related to having a parent with mental health problems.

Self-esteem score Self-esteem score (pre-programme) (post-programme)

Time point

25

20

15

10

5

0

Score


Hearing their peers talk about their family’s experiences helped children to understand better they were not to blame for their parent’s mental health problems.

“it was very good because it was nice to know you were in a group and other people had the same experience as you.”(Boy, 13 years)

Hearing their peers talk about their family’s experiences helped children to understand better they were not to blame for their parent’s mental health problems. Parents explained that while they may have tried explaining this on previous occasions, this message was reinforced once children were part of a peer group where other children spoke about issues that they could identify with.

“i always told her it wasn’t her fault when i was poorly and i locked myself away, that it isn’t because i didn’t love her. But i think she just thought ‘you’re just saying that’. So i think being in a group with lots of other people she believes that now, and her confidence has grown each week throughout SMilES.”(Parent, non-severe mental health problem)

Some children, however, reported that they did not feel that they had gained anything from listening to other children in the group, because they could not relate to the wide range of mental health situations talked about in the group. These children found the group discussions confusing and did not understand the relevance to them.


Sharing personal experiences with the group

Through talking in the group, children reported that they developed positive feelings about themselves as they stopped bottling up difficult feelings and thoughts, and shared and reflected on their experiences and emotions. Children reported that the factors that helped them share their experiences within the group included: knowing that the group and practitioners would listen and understand what they were saying; knowing that there was a shared respect for confidentiality within the group, and that it was a safe and supportive group environment. Children also felt able to speak in the group since it was a space away from their family and those whom they were worried about upsetting by speaking about their feelings.

“you knew you weren’t hurting them by saying all this because obviously they’re not part of your family or nothing to do with you.”(Girl, 14 years)

However, there were a number of issues that children said made the group less effective. The first was when there was too great a mix of ages within the group, which children felt made it harder to participate in group activities. In situations where younger children did not take the group seriously, became overexcited or were unable to keep up with the content of the session, older children complained about feeling unable to speak out or say everything they wanted to.


“if the [younger] boys weren’t … mucking about then it was fine, but then if they was like really hyper they would just laugh at everything everyone was saying, or just used to mess about and you just wanted to like sit there and be like, ‘yeah i’m not going to say nothing’. So it was like more uncomfortable when they were really hyper or excited.”(Girl, 15 years)

“When you’re with little ones you don’t get to say as much as you want because they’re little and they keep going, ‘What’s that? What’s that?’, and you want to keep on talking but you can’t because of the little ones.” (Girl, 12 years)

The issue of having a good gender balance within the group also stood out as a key factor for enabling children to participate in the group and feel comfortable during the sessions.

“the only thing that i would have changed, really, was that i was the only girl and it was like all the boys they just wanted to muck about and it was like awkward at times as well.”(Girl, 15 years)


One further barrier to talking in the peer group was identified by children who indicated a preference for talking to a practitioner in private. While the group provided an important space for children to speak out, for some children the idea of talking in a group remained a daunting prospect. Practitioners incorporated ways for children to speak to them one-on-one about any problems they were facing if they did not feel comfortable sharing their thoughts with the whole group. Some children found this to be a preferred option and were happier to take the opportunity to speak to the practitioner alone “to explain myself a bit better” (Boy, 12 years).

Learning about mental health

Children reported that the group work had helped them to understand that having someone within the family with a mental health problem was not a reason to undervalue or feel bad about themselves. They identified that the group work achieved this by helping them to understand how commonplace mental health problems are throughout society; that many people are affected by mental health problems, and that it is not something that is obvious from a person’s external appearance. In particular, children found it useful to explore the notion of mental health problems among well-known celebrities.

“it helped me understand that if people look young and healthy, it doesn’t mean that they don’t have a mental illness, anybody can have a mental illness.”(Girl, 10 years)


the sense of being able to offer and receive support from their peers helped children to feel positive about themselves and valued within the group.

Children described a range of activities they took part in during the peer group that supported their learning around the prevalence of mental health problems. From writing in their journals, playing games and taking part in group discussions, children reported that they were able to find a medium that suited them best for learning and expressing themselves.

Building new friendships

Children perceived that the combination of listening, talking, and participating in thought-provoking activities within the group enabled them to build friendships with their peers. The sense of being able to offer and receive support from their peers helped children to feel positive about themselves and valued within the group.

“i think me and [another] girl really bonded. like on my first day... i had the pink chair and [she] was the purple chair right next to me. and if we got stuck we’d ask each other and i think that really helped me, to know that i’d made a new friend.”(Girl, 10 years)

One difficulty with this experience was children’s disappointment in realising that they were unlikely to be able to sustain their new friendships outside of the group. This raises some questions regarding children’s perceptions around their long-term ability to maintain friendships and engage in enjoyable activities once the peer group has finished. As the group ended, children contemplated whether they would be able to continue enjoying care-free time to


themselves, and the friendships that they had built in the peer group, in their normal day-to-day life.

“i felt a bit sad…‘cos it’s a shame that i might not bump into them again, you know. it’s tricky saying goodbye, isn’t it, really?”(Girl, 10 years)

Leaving the group and new friendships raised difficult emotions for children who felt a sense of loss in the ending of the time spent with their peers. It suggests potential difficulties in maintaining new levels of self-esteem and self-confidence associated with the enjoyment of having new friends, beyond the end of the programme. Time 3 data will provide a valuable insight into the extent to which improved levels of self-esteem are sustained once Family SMILES has ended. This data will be explored in the final evaluation report of the programme.


Chapter 3: Decreasing children’s emotional and behavioural problemsClosely linked to the objective of raising self-esteem among children was the aim to decrease children’s emotional and behavioural difficulties. The evaluation explored the way in which children’s behaviour and their ability to deal with difficult emotions changed over the course of the programme.

3.1 Overall decrease in emotional and behavioural difficulties: children’s and parents’ perspectives

The key measure used to measure change in children’s emotional and behavioural difficulties was the SDQ. The SDQ captured the perspectives of children and parents on how children were dealing with these issues. Data from the SDQ (n=72) show a decrease in mean scores between pre- and post-programme.

Figure 2: Average pre-/post-programme SDQ scores

Total difficulty score Total difficulty score (pre-programme) (post-programme)

16

14

12

10

8

6

4

2

0

Score


The average difficulty score pre-programme of 15.3 points decreased to 13.6 at the end of the programme. This change is statistically significant and indicates a decrease in children’s emotional and behavioural difficulties at the end of Family SMILES. This suggests children being more able to cope with difficulties in their lives, feeling less anxious, showing an improvement in behavioural difficulties such as fighting with peers, and being able to think things through before acting. It is important to remember, however, that these changes in difficulty cannot be directly attributed to Family SMILES without the availability of comparison group data.

3.2 Change in level of difficulties experienced by children

The SDQ has been normed and has defined cut-off points for scores that identify varying levels of difficulty for children. Cut-off points define normal, borderline or abnormal levels of difficulty. Scoring criteria that determine the band of difficulty which children fall into differ for child-completed and parent-completed SDQs. Details regarding cut-off points for each band of difficulty can be found in Appendix 2 of this report.

A clinical level of difficulty was deemed to be a total score in the SDQ that fell within either the abnormal or borderline scoring band. Table 6 indicates that nearly half of the young people fell within the clinical range of difficulty at the start of the programme, but at the end of the programme only a third had a clinical level of difficulty. An exact McNemar’s test determined that this difference in the proportion of children experiencing a clinical


the data indicates not only a decreasing trend in the level of difficulties among children post-family SMilES, but also that they shift from a level of difficulty that is of clinical concern to that of a normal level.

level of difficulty pre- and post-involvement in Family SMILES was clinically significant.

Table 6: Percentage of children moving between the clinical and normal ranges of the SDQ when comparing pre- and post-programme scores (n=72)Level of difficulties Pre- programme

(per cent)Post- programme (per cent)

Normal range 49 65

Clinical range (borderline or abnormal) 51 35

The data indicates not only a decreasing trend in the level of difficulties among children post-Family SMILES, but also that they shift from a level of difficulty that is of clinical concern to that of a normal level.

3.3 Change in children’s emotional and behavioural difficulties: practitioners’ perspectives

The HoNOSCA (Health of the Nation Outcome Scales) enabled practitioners to rate the level of emotional and behavioural difficulty they felt was being exhibited by the children with whom they were working. Practitioners provided a score between 0 and 4 against 13 criteria set out in the HoNOSCA, which cover four broad categories of behavioural problems, impairment, symptomatic problems and social problems. There are no clinical cut-off points for HoNOSCA: rather it allows for a change in average scoring to be detected over the course of the programme. Data from HoNOSCA completed pre- and post-programme (n=47) shows a decrease in an average score of 8.91 pre-programme to a score of 7.96 at the end of the programme. This change is not statistically significant and so does not indicate a clear pattern in scores reducing at the end


of the programme. The mean scores also remain relatively low at the beginning and end of the programme, although again, in the absence of any cut-off points for this measure it is not possible to interpret these scores any further. However, the fact that the mean scores changed in the same direction as the SDQ data does reinforce the evidence that the programme is having a positive effect in this area.

3.4 Examples of decreased emotional and behavioural problems

Interviews with children and parents included discussions around how they felt their behaviour and emotional problems had changed over the course of Family SMILES.

In terms of children’s conduct, parents reflected on changes they had seen in how cooperative children were within the home, and how argumentative they were with other members of the household. Children gave examples of how they were more able to listen and understand their parents’ needs, and to respond appropriately without getting agitated about their parent’s behaviour. One young person commented how they used to try to “wind my mum up and now I try not to” (Girl, 11 years).

Some parents spoke about how their child’s behaviour had changed from being angry and argumentative to being more helpful and caring.

“if i was down or i’d say to her, ‘oh, would you make me a coffee please?’ We’d have like, ‘argh!’ and her start shouting and that, but after she’s been to quite a few meetings i’d say to her, ‘oh, would you make me a coffee please?’ ‘yeah


sure’, and it was like, hang on, is this my daughter?” (Parent, non-severe mental health problem)

Parents and children also reflected on how the emotional difficulties experienced by the child had changed. Reduced levels of anxiety among children were reported in terms of them experiencing fewer complaints of stomach aches and headaches related to stress. Children were also seen to be more able to cope with what was happening at home without displaying signs of anxiety that they would have shown previously.

“if i was feeling down or if i was crying, he would be ‘are you alright mum? Make you a wee cup of tea?’ – doing all those wee things, and he’d come over and give me a hug more and… he didn’t look worried and that was really great.” (Parent, severe mental health problem)

Children’s ability to not only deal with their own emotions but also to feel less worried about their parent and to confidently offer them support, provided reassurance to parents at the end of Family SMILES that children’s emotional and behavioural difficulties may have changed for the better.

3.5 Examples of increased levels of behavioural and emotional difficulties

During interviews, some parents also reflected on where they had noticed a change for the worse in children’s conduct at home. Parents gave examples of how some children were using their improved knowledge of mental health to their advantage in


order to play up to their parents’ vulnerabilities, and felt that the programme had not had a positive effect on their behaviour. In other cases, children themselves commented on increasing levels of hyperactivity within the groups they had been part of, where the mix of age or gender had not been balanced (as discussed in the previous chapter).

With regard to children’s emotional difficulties getting worse, both parents and children referred to life events external to Family SMILES, which were causing high levels of emotional stress during the period of the child’s engagement with the programme. In particular, issues around contact with parents and long-term living arrangements emerged as factors closely linked to high levels of stress.

“it’s just a matter of getting through it and getting back to live with my mum now…that’s all we want.” (Girl, 14 years)

Those children who were interviewed and spoke about the emotional difficulties they were facing with regard to being able to live with their parent again, scored within the clinical band for emotional difficulties at the end of the programme. This appears to reflect the level of stress that ongoing care proceedings were having on the lives of a number of children engaged in Family SMILES.


Chapter 4: Enhancing protective parentingIn addition to the group work with children, an integral part of the Family SMILES model is the individual work with the parent who has a mental health problem. The aim of this individual work is to reduce parenting behaviours that contribute to the child’s vulnerability and risk, and to enhance the parent-child relationship. The Family SMILES programme also aims to culminate in the development of a bespoke safety/support plan for the child during the joint sessions with the parent and child. This is based on the premise that – in the event of a parental relapse – the child will be adequately protected; responsibility for the child’s safety will rest with adults, and the wishes of the child will be considered.

This chapter evaluates the changes for parents/carers relating to their protective parenting. It seeks to outline the changes reported by parents through the standardised measure of the Child Abuse Potential Inventory (CAPI); analyse the qualitative data from parents and children to illustrate the changes in parenting behaviours, and to identify facilitators and barriers to achieving these changes.

4.1 Overall reduction in behaviours that contribute to risk: CAPI data

The key standardised measure that was used to evaluate change in protective parenting behaviour was the CAPI. This questionnaire captures parents’ perspectives across six domains related to their protective parenting. Three of these domains relate to ‘psychological difficulties’ in terms of distress,


rigidity and unhappiness. The other three domains relate to ‘interactional’ problems stemming from problems with child and self, problems with family, and problems from others. Table 7 outlines the changes reported by parents who completed the CAPI at the beginning of the work and then at the end of the work, and indicates whether this difference is statistically significant or not.

Table 7: Average scores reported by parents who completed the CAPI before and after the programme (n=22)Subscale Pre-programme Post-programme Statistically

significant

Distress 185.7 168.5 Yes

Rigidity 20.9 18.1 No

Unhappiness 40.0 31.6 Yes

Problems with child and self

7.5 6.1 No

Problems with family 17.7 14.8 No

Problems with others 17.5 17.0 No

Total score 289.2 256.2 Yes

Ego-strength scale 9.7 12.5 Yes

Loneliness scale 11.5 11.3 No

The average overall total CAPI score of parents decreased, and this is a statistically significant change. This indicates that parents were reporting an overall improvement in their protective parenting behaviour. The ego-strength scale of the CAPI indicates that parents are reporting increased confidence and self-esteem; this change is statistically significant. The loneliness scale indicates a reduced feeling of isolation at the end of the programme, but this change is not statistically significant. The measure also includes cut-off points: a total score of 216 or above is in the clinical range. It is important to note that the average total score at the end of the programme (256.2) is still above the scale’s cut-off point, which indicates that, on average,


the nSpCC work often challenged parents to rethink their perceptions of how their behaviour impacted on their child.

parents are still in the clinical range, experiencing high levels of distress and unhappiness at the end of the programme.

4.2 Factors affecting a change in parents’ protective behaviours

Insight into behaviour

The NSPCC work often challenged parents to rethink their perceptions of how their behaviour impacted on their child. This was the first step to accepting the impact of their mental health on their child.

“We did go off the programme a couple of times because of my way of thinking but once i realised, i was willing to accept that maybe i don’t do this and i don’t do that because of my health, then we got back on. [nSpCC worker] asked me a question and it stuck in my head: ‘How do the boys feel about you?’ i’ve never asked that because you take it for granted and that was the turning point for us as a family. Since the programme i’ve realised that i’m there in body but obviously not in mind, and it does impact on everyday life for the children. So it’s definitely helped me understand myself as a mum as well as the children, and it is hard for them when Mum’s not well. So i think it’s been really informative and helpful.”(Parent, severe mental health problem)


Parents who were able not only to accept that their behaviour impacted on their child, but also to think about emotions and behaviours their child may be experiencing as a result seemed to benefit most from the programme.

“i think the main thing was it [nSpCC work] kind of made me think, like about how things are for them. So, if i’m having a bad day then it kind of made me think about how they might be feeling, they might be feeling scared or worried, or that they might have to kind of make up for that and stuff like that.” (Parent, non-severe mental health problem)

However, parents who still viewed their child in a ‘negative’ way, for example suggesting that the child’s problematic behaviour had no link to their mental health, did not seem to benefit as much from the programme.

“i’ve tried speaking to them [children] several times, even before SMilES or anything like that but it’s all like they didn’t seem to understand or anything and especially with [child who attended family SMilES] now he does seem to understand a little bit more but it’s like he sees it as an opportunity to play up. So if i tell him i’m having a bad day or i’m stressed and just give me a bit of time then they’re constantly there playing up


the programme helped parents to accept that mental health problems were not uncommon or something that needed to be hidden from their children.

and that doesn’t help. i think it [family SMilES programme] probably helped other families probably better than it has here because of the lack of respect that goes on in this house.”(Parent, severe mental health problem)

Understanding about mental health

The programme helped parents to accept that mental health problems were not uncommon or something that needed to be hidden from their children. This understanding helped them feel more confident about themselves as parents.

“Because initially i used to say … ’psycho moments’ and things like that, whereas she’s [nSpCC worker] actually taught us not to say that, it’s actually oK. Because initially i was really, really embarrassed by it, that i was different … but i’ve actually learnt that … i’m not that different, and there’s a lot of other people with the same thing as you.”(Parent, non-severe mental health problem)

Some parents reported that the programme increased the child’s understanding about mental health, strengthening their relationship and helping them to be more responsive to their child’s needs.

“Without it [family SMilES] we’d still be struggling and still be different, and i wouldn’t have as close a relationship i’ve


got with her, and that’s such an important thing. the programme helped us to understand each other better instead of us butting heads and just ignoring each other.”(Parent, non-severe mental health problem)

Conversely, when parents did not perceive a change in their children understanding their mental health, they still experienced high levels of distress and unhappiness in their parenting.

“it’s sad really because this elder one says to me ‘you’re a biological Mum but not a real Mum.’ that is so sad … i need to go there [mental health project] because i knew i needed support and if don’t go there i know i will get unwell and i need my medication, everything, because it’s important, but they [children] still don’t understand that we are in the same situation.” (Parent, Severe mental health problem)

Supportive approach to change behaviour

Parents emphasised that for them to achieve any change in their parenting it was important that they could trust the worker first. Parents spoke about their initial concerns about going onto a programme run by the NSPCC. They expressed some anxieties about it, thinking that their children might be removed from their care and also the implication that they had been physically harming their child.


“initially i was petrified of nSpCC. i mean i didn’t know SMilES was the nSpCC at first. i probably would have run a mile because initially you think that’s for children that are being abused. it is a big stigma.”(Parent, non-severe mental health problem)

Once parents met practitioners and had a better understanding of the service and what was being offered, their anxieties were eased. Parents said that the NSPCC practitioners were able to pace the work based on their (the parents’) needs and what had happened to them the previous week. They felt that the NSPCC workers’ approach was respectful and supportive.

“She was friendly and down to earth, she wasn’t sort of lording it over anyone, or, ‘i’m from the nSpCC and i’m here to …’ We were just talking, talking like you would with a friend or somebody else like that. She knew her stuff and it didn’t come out as she was trying to tell you things, it was like she was trying to suggest things to help and she wanted to help.” (Parent, severe mental health problem)

Extent of parent’s stress

Some parents felt that their mental health problem was an ongoing source of distress; they had experienced fluctuations in their recovery process and at times struggled to keep up with taking their medication, which had impacted on their parenting.


“i didn’t know that i had depression. i wished nobody had that depression. it’s horrible. and it is awful. it goes in your body and it is like worse than dying, honestly. it’s horrible, so we just have to keep on going actually.”(Parent, severe mental health problem)

Parents also noted that they were more susceptible to not coping well with stressors in their lives, compared to other parents without mental health problems. Two of the key stressors that parents identified in the qualitative interviews since starting their work with Family SMILES were breakdowns in long-term relationships, and uncertainty about the future of their children’s situations with the involvement of social services.

“obviously i was already going on the slope of going down myself, living with the problems that were going on at home and… my own personal problems, and you find it difficult like to cope with everyday things. you just like cut off from everything.” (Parent, non-severe mental health problem)

Safety plans developed during the joint sessions

Some parents found the safety plan developed during the joint sessions with their child useful in supporting their parenting. This was because it contained helpful information as well as providing guidance to children, on what the key messages are that need to be communicated to a professional


in an emergency, and how those messages should be delivered (eg, be clear, be calm, be confident, explain why you are calling and what the symptoms are).

“there is nothing like this [safety/support plan] we had in place before, and that gives me a massive relief because i know that they’re safe, i know they’ve got contacts. i would never have thought of an action plan before until SMilES. it gives me a lot of confidence to know that they’ve got this to refer to. it stays up here [above fireplace], it’s access to all of them [children]. Because we want the other kids to be able to access it as well, so we have it up available, because it’s got contact numbers in, who they can talk to.”(Parent, non-severe mental health problem)

The process of putting together a safety plan had been helpful for some parents, as it had involved input from others who were not part of the Family SMILES programme. This helped to create a joint family ownership of the plan.

“i think it was useful because [daughter] got to put her input into it and then i got to put mine, and [older sibling] even had a little bit. i think the plan was useful for [older sibling] because i think sometimes he thinks maybe he’s not as involved,


the joint sessions had not resolved underlying family tensions in some families that were not specially related to the parental mental health problems.

where basically if i’m ill he takes over, he does the washing and the cooking.

[Daughter] put in the plan that the person she would talk to most would be him, and we said to her what would be helpful is can you say to [older sibling] i need a cuddle, and she was like, ‘no’. So we spoke to [older sibling] and basically said ‘if i’m poorly and that, i think she would like you to sit down with her every now and again and say “are you all right, do you need to talk to me, do you need a cuddle, i’m here if you need me”’ type of thing.

then it gives her a chance to open up to him. i think before, she would just try to get along, where now they both understand and he understands more that she might need it off him.”(Parent, non-severe mental health problem)

The joint sessions had not resolved underlying family tensions in some families that were not specially related to the parental mental health problems. These included issues as coping with a bereavement within the family, or dealing with children’s misbehaviour. Parents reflected that maybe the joint sessions needed to be longer, as they would like the opportunity not only to share what had happened but also to have the NSPCC workers facilitate a discussion with the family on how things could change.


“Well i suppose in an ideal world, because SMilES is going to be limited by time and resources and stuff… i suppose it would have been nice to have like a consolidation session where we could all sit down to discuss what our understanding, coming out of the programme…, the care plan and then perhaps… sharing our hopes for our family…, just like something with a nice positive direction. i think like that would have been really handy.”(Parent, severe mental health problem)

Parent’s ongoing relationship with children’s services

The aim of increasing protective parenting was particularly important as approximately half of the families on the programme had at least one child on a child protection plan. This also highlights the level of additional support that the NSPCC needed to provide to the families so that they could sustain their involvement in the programme. This involved:

• suggesting advocates for children

• providing information to families about benefits

• talking to children about professionals’ involvement in the family

• talking to parents about professional involvements

• talking to children’s services about the family.

The input from the NSPCC was highly valued, as one parent whose children were on a care order with the children’s maternal grandmother commented:


“for example, i’ve had concerns over [daughter] in particular. So i’ve spoken to them [nSpCC worker] and they’ve gone into school to see her. […] her social worker don’t go into school to see her and it’s been like a listening ear for her, she’s felt she can talk to somebody. So that in itself is enough for me. But they’ve also… supported me by ringing the social worker saying ‘hold on a minute, this isn’t right, in our opinion, this’ and they’ve not just been the children’s voice, they’ve been my voice as well. they didn’t have to do that, all my children are on a care order at the moment, social services have got like all this power, they aren’t interested; but the nSpCC, without them i don’t think my mum [maternal grandmother] would have got through the last six months if i’m honest.”(Parent, severe mental health problem)

It was suggested that work carried out with families on Family SMILES should be used to help inform decisions made by children’s services regarding care and contact arrangements for those children involved. One child commented how they had hoped that social services would recognise the commitment that her family had shown to taking part in Family SMILES and that this would be taken into account in decisions made about contact with her parent.


“and then also like the nSpCC… they’d be able to put together a conclusion. and a lot of nSpCC speak to social services so they’d be able to…put the conclusion to social services, and then social services will see the change.” (Girl, 14 years)

A number of families commented how they would find it helpful to know that the NSPCC was formally contributing to decisions about the family, based on their knowledge of their needs and capabilities as a result of their work with them through Family SMILES.


Some parents and children said that they were communicating better as a family since the programme – discussing mental health in a more honest way with each other.

Chapter 5: Improving family communication about mental health The previous chapters have outlined changes experienced by children in their self-esteem, confidence and behaviour, as well as by parents in their protective behaviours. An outcome that the evaluation had not specifically measured but that was reflected in the qualitative interviews related to a change in family communication post-Family SMILES. This chapter focuses on examining how family communication changed, why it was important for them, and the role the programme played in bringing about these changes. It also examines the barriers to improving communication experienced by some families.

5.1 Changes in family communication

Some parents and children said that they were communicating better as a family since the programme – discussing mental health in a more honest way with each other.

“But i was able to talk to him then more about it and i learnt that from the SMilES from my own sessions. it’s oK to discuss things with your child and it’s oK to say ‘i’m not feeling well’, which before i would have said nothing. Just ‘oh, mummy hurt her back today’ or ‘mummy tripped and fell’ or something, i’d come up with these reasons for being in bed or if i’d been


crying or whatever. Definitely, definitely made a big difference.”(Parent, Severe mental health problem)

Children said that they found the parent’s ability to discuss their mental health problems with them very valuable. While the eligibility criteria for the programme require the child to be aware of the parents’ mental ill health, often it had not been acknowledged or discussed with the child within the family. Feeling that there are no longer secrets within the home environment was important for parents and children.

“i found things helpful, like what sort of illnesses my mum had, and it’s just that i’ve been wondering what sort of things she has got. now i know… at least my mum has told me instead of keeping it a secret for her life.”(Girl, 10 years)

The changes some families had noticed were that they were talking more, spending more time together doing fun things, and the atmosphere at home was more relaxed and positive.

“But i do know that if [Mum’s] mood is not that great, the kids’ talking to her does bring her round... that’s the one thing i have noticed [after the programme], is that she’s a wee bit more relaxed because they know and…she’s not trying to hide it. But she does try, she struggles upstairs when she’s not feeling too well or her


mood’s down. Before [son] would maybe think to himself, ‘is Mum in bed again?’ But now, since he’s been [to family SMilES] it’s, ‘Mum’s just not feeling too well at the minute, she’s tired.’ it’s just not all doom and gloom and it’s not all negative and it’s not all bad.”(Partner of parent with severe mental health problem)

The conversations within the family also felt easier for some children as they felt the language used in conversations was more appropriate to their age and understanding of mental health.

“When she talks about it… she doesn’t feel all like she has to explain it and get it into normal language…. She can talk to me at my level now. She doesn’t have to simplify it anymore because i know the basis of what’s going on. yeah, it’s a lot easier.” (Boy, 13 years)

5.2 The role of the programme in increasing communication

Parents and children reflected that prior to engaging with the Family SMILES programme they were embarrassed about talking about mental health as family. As referred to in the previous chapters, the group work with the children and the individual work with the parent helped them to process their thoughts and feelings about mental health. This helped them understand each other better; respond


Even though parents may have spoken to their children about mental health prior to the programme, they felt family SMilES had enabled children to process that information more effectively.

to each other with more empathy, and emphasise the importance of talking about feelings as a family. This strengthened the relationship between the child and the parent, and consequently improved communication within the family.

Better understanding

Children said that they had not realised what their parent was experiencing and that the programme gave them better insight into their parents’ lives. Parents observed that the programme had enabled children to have a much better understanding of what happens when their parent’s mental health deteriorates. Even though parents may have spoken to their children about mental health prior to the programme, they felt that Family SMILES had enabled children to process that information more effectively.

“i’ve always been able to talk to her. i think the difference was that she didn’t believe me and now she does. now she knows that i don’t go into myself because she’s getting on my nerves, i go into myself because that’s what i do, that’s part of my illness. So… it’s more that she understands and believes it, more importantly.” (Parent, non-severe mental health problem)

Parents explained that they were able to prioritise listening to their children’s needs as they understood the impact of their mental health problem on their child. This helped them to be more proactive in creating opportunities to discuss issues with their children.


“i’ve learnt so much more. We used to just think she [daughter] used to go up and that was her way with dealing with it and we’d leave her to it, whereas actually it was going around in her head, and it was things i never knew until SMilES managed to get us communicating. it’s brilliant; it really is nice, because now we’ve got a bigger bond than i thought we had initially. i thought i was doing right by letting her deal with her own thing, when actually now she’ll ask me for five minutes and we sit and talk and we will have our own one-to-ones now as well.” (Parent, non-severe mental health problem)

More appropriate language and responses

Families said that the programme had given them the language to talk about mental health more appropriately with each other. This included a greater knowledge about terminology (medication, treatments and conditions) and their meanings, which enabled more appropriate communication. In addition to language, children said that having a better understanding of their parent had helped them to respond confidently and appropriately to their parent, with more care and respect.

“i never really felt comfortable talking to my mum about anything. Well, i can cope with information now. My mum is starting to tell me stuff, and usually before i would probably go: ‘why’s that; what’s going on;


for some children this was the first programme that had made them realise the benefits of talking about feelings instead of just withdrawing from family communication and ignoring them.

i don’t really care’. But i do care now. Well, we’re not all secretive now.” (Boy, 13 years)

Caregivers also emphasised the benefit of the programme in working with them to support conversations about mental health with the family and valued the reassurance that had been provided to them.

“Because it’s been very, very traumatic for the children, it has for us, you know, we’re grandparents but we’re not child psychologists and at times i used to say to them [nSpCC worker], ‘i’m fearful of what to say in case i say the wrong thing and make it worse,’ but when you’ve got people you can turn to and who can give you that advice it makes you reassured.” (Carer 1, daughter has a severe mental health problem)

Believing that talking does help

Children and parents said that the programme had enabled them to better appreciate the value of talking about feelings. For some children this was the first programme that had made them realise the benefits of talking about feelings instead of just withdrawing from family communication and ignoring them.

“… it’s not that i didn’t trust my family, it was that i didn’t want to hurt them by the things i would have come out with. So i


used to bottle it up and… […] just kind of sit alone, listen to music and just be on my own…[…]. i’d spoken to loads of people, they just don’t tend to do much, and i just decided to give up sooner or later because… i’d just waste my time talking and i just feel like they’re being nosy… because they don’t take anything into consideration.”

But then when i had been to that group for eight weeks, somehow i just couldn’t end up holding it in and maybe that was a better way of me showing that i’m hurting[…]. now i’ve learnt that telling someone is the best thing and bottling up is the worst[…]… and that’s my general reaction, to tell someone.” (Girl, 14 years)

Some families, however, did not experience the change they desired in their communication. In some situations not all family members were ready to engage and this created a barrier to increasing communication among them.

“But i think it would take quite a while to get everyone talking. i think my sister, my nana and my mum and me can all talk, and it’s just hard to talk to the person with the mental health because they don’t understand as good as you do. So i think


that’s quite hard and that needs to be taken quite a lot of care of.”(Girl, 12 years)

There was also hesitation among parents about providing information regarding mental health to younger children; some expressed discomfort with their child being exposed to such information at a young age. This acted as a barrier to further conversation about mental health within the home.

“Well, to be honest i didn’t agree with it at first about her even going and finding out what’s wrong with her mother, she’s 10 – well, she was 9 at the time – and i think it was maybe too much too soon.”(Carer, ex-partner of parent with non-severe mental health problems)

Some parents reported that their children wanted to speak to more external people rather than family members, but acknowledged that building trust with the child was a process, and that the Family SMILES programme had supported them.


Chapter 6: Conclusion This interim report indicates that the wellbeing of children improves by the end of the programme, and that parents are in a better position to protect their children. While some of these changes appear small (for example, in relation to children’s self-esteem and protective parenting) they do show the value of a short eight-week intervention in enabling family members to gain positive insights into themselves and their interactions with each other. The evaluation has pointed towards the legacy of Family SMILES being the improved communication within the family, which provides the building blocks for parents to continue discussions with their children about their mental health problems, and for children to confidently communicate their needs and concerns within the family environment.

The evaluation highlights the value of an approach to service delivery that works with the whole family, bringing together adults and children who may previously have been engaged with multiple agencies. One aspect of the programme particularly valued by both parents and children was the way in which it improved their understanding of the impact of parental mental health problems on the family and facilitated communication within the family. This suggests that the Family SMILES programme bridges a gap between adult mental health and children’s services by creating a safe space for parents and children to explore the impact of mental health on the family.


This linking of adult mental health and child protection support draws attention to the value of a service such as Family SMILES for all families where there are parental mental health problems, regardless of whether there have been any prior child welfare concerns. Providing parents with support to understand the potential implications of their mental health problems on their children’s wellbeing, and enabling children to develop their understanding of mental health and how to keep safe at home, presents a valuable model for improving communication and relationships within the family environment.

6.1 Suggestions for improvement

Despite the positive findings, there were suggestions from participants on how the programme could be improved. These included a review of the assessment process to ensure that children are appropriately matched to groups, and that all children taking part in the programme should understand that their parent has a mental health problem. In terms of improving the group structure, parents and children spoke about the importance of ensuring that groups are well-balanced in terms of age and gender. They also felt that it was important for the location of group sessions to be accessible and for transport to be provided to those families who might struggle to attend sessions without such support. There were also suggestions around the content of group sessions, for example, including discussions on domestic violence. Parents stressed the importance of scheduling the group sessions to avoid clashes with other similar services in the local area, such as Young Carers, a group that some children who attended Family SMILES had also been involved with.


Parents in particular felt that a more positive image of the NSPCC would have made them feel more comfortable about attending the programme from the outset. Linked with this was the suggestion of promoting Family SMILES more widely, within an NHS setting. Parents felt that improved publicity of Family SMILES would enable more families to access valuable support and appreciate the range of services provided by the NSPCC to help families, breaking down preconceptions around the NSPCC as an agency that removes children from families.

Both children and parents spoke about how they would have valued the provision of longer- term or drop-in support once the programme had ended, or signposting to other services that could provide such support on a more ongoing or ad hoc basis.

It is possible that, if implemented, these changes may improve the effectiveness of the intervention and result in improved outcomes.

6.2 Next steps

This evaluation has found that the outcomes for both parents and children moved in the right direction, even if some of the changes appear to be relatively small. Without a comparison group, however, it is not possible to directly attribute the changes to the intervention. The planned addition of comparison group data will help to strengthen evaluation findings and should enable greater confidence that Family SMILES has directly led to the positive changes for children and parents. The findings from this additional element to the evaluation will be included in the final evaluation report, along with more qualitative data from parents and children – especially those who may have dropped out – as


well as interview data from NSPCC practitioners and referrers.

The NSPCC is exploring how it can apply the learning from the interim evaluation to enable the service to have a greater impact on children and their families. The scope of this work includes engaging with local authority commissioners and other organisations working with this group of families. The aim is to identify changes to the content of both the programme and also the settings in which the programme might be delivered to make it more effective.


BibliographyAldridge, J. (2006) The experiences of children living with and caring for parents with mental illness. Child Abuse Review 15: 79-88.

Baldwin, P. and Glogovic, C. (2010) Providing S.M.I.L.E.S to children of the Waterloo region whose caregiver has a mental health diagnosis in Ontario Association of Children’s Aid Societies Journal, 55(1).

Bee, P., Bower, P., Byford, S., Churchill, R., Calam, R., Stallard, P., Pryjmachuk, S., Berzins, K., Cary, M., Wan, M., Abel, K. (2014) The clinical effectiveness, cost-effectiveness and acceptability of community-based interventions aimed at improving or maintaining quality of life in children of parents with serious mental illness: a systematic review. Health Technology Assessment 18 (8). Available from: www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0019/106453/FullReport-hta18080.pdf [accessed 10 May 2014]

Office of National Statistics. (2001) Psychiatric Morbidity Survey. London: ONS.

Pitman, E. (1997) Simplifying Mental Illness plus Life Enhancement Skills (SMILES). Information Sheet.

Pitman, E. and Matthey, S. (2002) Evaluation of the SMILES program for children with mentally ill parents: Bankstown, Sydney. South West Sydney Area Health Service, Sydney, Australia.

Pitman, E. (2010) SMILES program: simplifying mental illness plus life enhancement skills (PDF). Adelaide S. Aust.: Children Of Parents with a Mental Illness (COPMI).

http://www.oacas.org/pubs/oacas/journal/2010SpringSummer/smiles.html



http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0019/106453/FullReport-hta18080.pdf

http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0019/106453/FullReport-hta18080.pdf

http://www.copmi.net.au/jsp/resources/files/smiles_program/app/program_info_p97-101.pdf

http://www.copmi.net.au/jsp/resources/files/smiles_program/app/program_info_p97-101.pdf


Prokop, J., Bundred, K. and Green, J. [2010] Background paper on London serious case reviews completed April 2006 – September 2009. [London]: London Safeguarding Children Board.

Sahin N. (2010) Childhood Trauma, Types of marriage and self esteem as correlates of Domestic Violence in Married Women in Turkey. Journal of Family Violence 25 (7): 661-668.

Spade, Julie A. “The Relationship Between Student Bullying Behaviors and Self-Esteem.” Electronic Thesis or Dissertation. Bowling Green State University, 2007. https://etd.ohiolink.edu/

Stallard, P., Norman, P., Huline-Dickens, S., Salter, E. and Cribb, J. (2004) The effects of parental mental illness upon children: A descriptive study of the views of parents and children. Clinical Child Psychology and Psychiatry 9: 39-52.

Tunnard, J. (2004) Parental Mental health Problems: Key Messages from Research, Policy and Practice. Dartington: Research in Practice.

http://www.londonscb.gov.uk/files/2010/resources/serious_case_reviews/scrs_in_london_for_web.doc

http://www.londonscb.gov.uk/files/2010/resources/serious_case_reviews/scrs_in_london_for_web.doc

https://etd.ohiolink.edu/


appEnDiCES

Appendix 1: Overview of sessions

Overview of sessions for children:

Session 1: All about me

Aims: The child feels settled in the group and gains an understanding of the programme.

Learning outcomes: The child identifies strengths and resilience factors in their life, such as people they like and things they enjoy.

Session 2: Mental illness

Aims: The child begins to understand mental illness, and works with the group to dispel some myths.

Learning outcomes: The child has a better understanding of mental illness and the terminology around mental illness.

Session 3: My Story

Aims: The child begins to understand how mental illness affects their family.

Learning outcomes: The child sees how their parent’s mental illness leads to them assuming a particular role within the family. The child identifies positive and negative aspects of this role, relating to their own health, wellbeing and development.


Session 4: My feelings

Aims: The child explores and understands the emotional impact mental illness has on them and their family.

Learning outcomes: The child identifies strengths and resilience factors in their life, such as people they like and things they enjoy.

Session 5: Looking after myself

Aims: The child increases their ability to maintain and improve self-esteem.

Learning outcomes: The child learns, and can employ, self-esteem and assertion techniques, to help if they are feeling self-critical or have been criticised or victimised by others.

Session 6: People I can trust

Aims: The child identifies people who can support them if they are feeling upset, lonely, sad or confused.

Learning outcomes: The child reflects on the people who can provide support when needed and thinks about how those people can offer support. The child collates information that will contribute to the safety plan component of Family SMILES.

Session 7: Identifying sources of support/ safety planning

Aims: To assist participants in identifying support and in thinking through what sources of support they might use in different situations.


Learning outcomes: Children understand that it is the responsibility of adults to ensure their safety and are able to identify the people within their own support system whom they can rely on if needed.

Session 8: Feedback, evaluation and graduation

Aims: To reflect on feelings about group ending and formally evaluate the programme.

Learning outcomes: The graduation enables children to take pride in completing the group and to bring together any reflections on being part of the group.

Overview of parent sessions:

Session 1: My story

Aim of session: To help the parent gain an insight into their illness, how it affects them, their relationship with their child and other significant family members.

Session 2: Managing anxiety

Aim of session: To help the parent gain an understanding of how anxiety and stress impact upon them and their child.

Session 3: My parenting

Aim of session: To assist the parent in setting boundaries, examining what helps and what hinders boundary-setting.

Session 4: My strengths as a parent

Aim of session: To discuss and reinforce what the parent does well.


Session 5: Understanding my child’s feelings

Aim of session: To increase the parent’s understanding of how their child experiences family life.

Session 6: Communicating with my child

Aim of session: To increase the parent’s understanding of how to effectively communicate with their child.

Outcome: The parent will have knowledge of practical strategies to aid effective communication with children.


Appendix 2: Inclusion criteria

Inclusion:

• The child has a parent (or significant carer) with a mental illness, which has given rise to, or contributed to, concerns regarding their ability to safely care for the child.

• The parent has been diagnosed or is being assessed for diagnosis by health professionals as having a mental illness or has been receiving professional support for a significant period of time for a mental illness, but has not received a diagnosis.

• The child is aware of the parent’s illness.

• The parent is informed of the purpose and remit of the group and consents to their child’s attendance.

• The child falls within the age limit for the group and has been assessed as cognitively able to participate in the group.

• The child presents as being emotionally able to engage in the group. Children with emotional/behavioural difficulties who are receiving a service from CAMHS can be considered for participation in the programme. However, advice should be sought from the relevant CAMHS team as to the child’s suitability for and ability to engage with the programme and it should be clarified that CAMHS remains the lead service with responsibility for the mental health of the child.


Exclusion:

• The child is not in a reasonably secure family/living position – i.e. in the process of being or very recently admitted into care.

• The child during the course of the assessment presents with emotional or behavioural distress on a level that requires assessment by a suitably qualified healthcare professional. NSPCC assessment should be suspended, pending outcome and recommendations regarding the child’s ability to engage in the programme.

• The child or young person has a diagnosis of a major mental illness that significantly impedes their ability to participate safely in a group.

• If the parent is extremely unwell at the time of the group, attendance may not be appropriate due to corresponding distress levels in the child and inability of the parent to participate in individual work. This should be assessed on a case-by-case basis.


Appendix 3: Family SMILES with standardised measuresOutcome Tool Summary of measure

Protective parenting(Parent perspective)

Child Abuse Potential Inventory (CAPI)

• Instrument to help towards distinguishing between parents who may be at risk of physically abusing their child and those that are not. Subsequently used as an assessment tool and for evaluation purposes.

• Developed in the 1970s by Joel Milner to assist social services in the US to screen parents who’ potentially’ may be at risk of physically abusing their child.

• Used for Family SMILES as an evaluation tool only and not for predicting abuse. It is a reliable tool for measuring change in protective parenting.

• The author of the scale identified six aspects through literature and existing research that would distinguish parents who were a risk compared with those who were not. Three relate to the attitude of the parent and three relate to relationships.

• Validated to determine six key aspects: – Attitude: Rigidity, Distress, Unhappiness – Interpersonal: Problems with child and self, problems with family, problems from others.

• Also includes scales to measure ego strength and loneliness.

• Inbuilt validity scales to filter out parents who may be faking good responses or being inconsistent or random in their responses.

• More than 1,000 journal articles, chapters, books, dissertations, theses, convention papers, and unpublished reports describing the psychometric characteristics and/or applications and/or limitations of the Child Abuse Potential (CAP) Inventory. (as at 2/2012)

• Abuse scale cut-off scores: 166 and 215• We would be looking for a statistically significant

decrease in the mean abuse scores of parents engaged in Family SMILES between the beginning and end of the programme.

Children’s self esteem(Child’s perspective)

Self Esteem Scale (based on Rosenberg)

• The self-esteem questionnaire is based on the Rosenberg self-esteem questionnaire (most widely used self-esteem measure) but since it has been adapted for use by children it is not a standardised measure, therefore no normative data is available.

• Research shows that children who care for parents with mental illnesses are more likely to have lower self-esteem. Therefore, this is a valuable tool for measuring changes in children’s self-esteem over the course of the programme.

• Relatively short (10 statements); includes reversed scoring.

• Total scores range from 0 – 30 with a higher score indicating a higher level of self-esteem.

• Rosenberg has been used in studies of domestic abuse (Shahin et al, 2010), substance abusers (Eastman and Bunch, 2009) and bullying (Spade, 2007).

• We would be looking for a statistically significant increase in the mean scores of children’s self-esteem questionnaires between the beginning and end of the programme.


Outcome Tool Summary of measure

Children’s emotional wellbeing(child’s perspective)

Goodman’s Strengths and Difficulties Questionnaire (SDQ)

• Measures the emotional and behavioural problems of children and young people – measures strengths as well as difficulties.

• It has been used in the Framework for the Assessment of Children in Need and their Families

• It is completed by the young person if aged 11 years or over, or where the practitioner feels that the child is able to complete it alone. It is completed by the parent if the child is younger than 11 or unable to complete it themself.

• It contains four difficulty subscales: hyperactivity, conduct problems, emotional symptoms and peer problems, plus a strength subscale of pro-social items.

• The higher the total score, the higher the level of difficulty. Where parents complete the SDQ, high scores are deemed to be 17 or over; where self-completed, 20 or over. Borderline scores range between 14 and 16 for parent-completed SDQs and between 16 and 19 for those completed by the child. Anything below these thresholds falls into the normal category.

• We look for a statistically significant decrease in the mean score of SDQs between the beginning and the end of the programme.

Children’s social functioning

HoNOSCA (Health of the Nation Outcome Scales Child and Adolescent Mental Health)

• Measures the range of physical, personal and social problems associated with mental health.

• Contains four subscales: behavioural problems, impairment, symptomatic problems and social problems.

• Rating for each sub-scale (between 0 and 4) is based on the judgement of the practitioner.

• There is no cut-off point for a high level of mental health problems.

• It is easy to use and sensitive to change, making it a useful tool for measuring change in practitioners’ perspectives between the beginning and end of the programme.

• We look for a statistically significant decrease in mean scores of HoNOSCA between the beginning and end of the programme.

Children’s safety skills

Evaluation wheel

• Evaluation wheels are completed by parents and children at the beginning and end of the programme to measure softer outcomes; it is not a standardised measure.

• Respondents rate themselves between 1 and 5 (1 being low, 5 being high) against 5 criteria (parents) and 6 criteria (children), such as confidence in being able to do the best for their children (Parents’ wheel) and being able to talk to parents about their mental health (Children’s wheel).

• We expect to see an increase in the way in which parents and children rate themselves against the criteria on the evaluation wheels between the beginning and end of the programme.


Appendix 4: Interview schedule for children

1 Genogram

To understand more about you and your family.

2 About the programme

To understand how much children remember about the programme and activities .

3 Specifically about the group experience

• How did you feel about working with the other children in your group?Did you enjoy this most of the time or not? (explore: group dynamics)(If yes) What did you like about this?(If no) What didn’t you like about this?

• How did you feel about talking about things which have happened to you and your parent?(If yes) Did you find it any easier to talk about things after you had been at the groups a bit longer?Did you feel any better or worse after talking about these things?

4 Changes before and after:

4.1 How you feel life is going for you overall?

Did they make you feel differently about yourself?

4.2 Change in understanding about your parent’s mental health

Do you feel you learnt anything about your parents’ situation?


4.3 Change in talking about your parent’s behaviour with others

Did they make you feel differently about your parent?

4.4 Change in your own behaviour

Have you noticed any changes in yourself at home, with friends or in school?

4.5 Change in your parents’ behaviour

Do you think they feel better after this work or not?

4.6 Change in your relationships

Have you noticed any changes in how you get on with others and your parents?

5 Comments on how the safety plan worked

How did you feel about doing the joint work with your parent?Did you enjoy this most of the time or not?(If yes) What did you like about this?(If no) What didn’t you like about it?

Is there anything that could be done to make the Family SMILES programme better?If yes, please explain


Appendix 5: Interview schedule for parents

1) On the whole, how would you describe your experience on this programme?

For example, has it been a mostly positive or negative experience?

2) Was there anything you particularly liked or disliked about the individual work?

Prompts used if necessary: For example, any particular sessions, anything in general about the sessions?

3) Do you think this experience has helped you in any way, or not?

Prompts used if necessary: For example, coping with your past experiences, relating to your child, understanding your child better, feeling better about yourself?

3a) (If yes) Please give details.

3b) (If no) Are there any reasons you feel this work has not helped you?

4) Do you think the groups/work have helped your child in any way, or not?

Prompts used if necessary: For example, coping with their past experiences, affecting their behaviour in any way, dealing with their emotions differently?

4a) (If yes) Please give details

4b) (If no) Are there any reasons you think they have not helped?


5) Do you feel that your relationship with your child has been affected in any way by the Family SMILES program?

5a) If yes, how has it been affected?

For example: positively or negatively?

6) How have you felt about talking about things that may have happened in your past to the worker?

For example: have you found it hard, easy. What has been hard/easy?

7) How have you felt about talking about your experiences with your child in the joint work?

For example: have you found it hard, easy. What has been hard/easy?

8) Has this programme made you think differently about anything which has been discussed?

8a) (If yes) What do you think you have learnt?

9) Do you feel you would know where to go for further support if you needed it?

Where do you think you would go to get this support?

10) What did you think of the worker on the programme?

For example: in terms of how well you related to them, how easy it was to talk to them, how they dealt with the group, how they dealt with sensitive issues?

11) Do you have any suggestions of how this service could be improved?


Appendix 6: Ethics overviewAll NSPCC practitioners delivering Family SMILES attended evaluation training in order to gain an understanding of how the evaluation worked and the key ethical considerations in carrying out the evaluation. The ethical issues central to this evaluation are listed below:

• Gaining informed consent from service users Practitioners explained the purpose of the evaluation to service users, their role within the evaluation and the way in which information they shared with the evaluation team would be used. Service users consented to the evaluation with a clear understanding of these issues. Consent forms were signed pre- and post-programme by parents who were happy to take part, and/or for their child to take part, in the evaluation. Verbal consent was obtained from children regarding their participation in the evaluation. The same process is also being used for those parents and children who are part of the comparison group.

• Confidentiality It was explained to service users that all information shared with the evaluation team would be treated in the strictest confidence unless they shared something with the team that raised concerns about a child’s safety.

• Option to withdraw from study Service users understood at the time that they consented to be part of the evaluation; that they could change their mind at any time, and withdraw their involvement or contribution to the evaluation before the final report is produced.


• Protection of participants Practitioners were requested to use their judgement when asking service users to take part in the evaluation, particularly regarding their capacity to cope with completing certain measures, to ensure that doing so did not cause them any harm. This was especially crucial when administering the CAPI, which is a lengthy measure containing some difficult and personal questions that may cause parents to reflect on their lives in such a way as to cause them a level of distress. Practitioners were able to use their judgement regarding where the CAPI should not be completed; this was particularly important when working with parents with mental illness. Similarly, practitioners were able to use their judgement to ensure that service users who may have felt vulnerable in an interview situation were not included in the sample for qualitative interviewing. In this way, participants were protected from any potential harm. All interviews were gently wound down at the end, to ensure that participants were left feeling as upbeat as possible at the end of the interview.

• Advice/support for participants Should any evaluation participant be left feeling in need of extra support following an interview, researchers were able to suggest services that might be able to help them. While practitioners administered measures with service users, they were able to discuss any feelings that arose from completing the measures, and obtain advice and support directly from their NSPCC worker.


• De-briefing All interviews and completion of measures ended with a discussion about the experience of taking part in the evaluation for the service user, to ensure that they felt comfortable with what they had just done, and to provide them with the opportunity to ask any questions that they may have had.


Appendix 7: Statistical analysis of standardised measures

I. Strengths and Difficulties Questionnaire data: Change in mean score, pre- and post-Family SMILES; analysis based on the Wilcoxon Ranked Sign Test (n=72)SDQ subscale Mean at

T1Standard Dev. at T1

Mean at T2

Standard Dev. at T2

P-Value

Emotional difficulties 4.08 2.73 3.87 2.65 0.39

Conduct problems 3.04 2.26 2.52 2.06 0.01*

Hyperactivity 5.50 2.51 4.82 2.49 0.02*

Peer problems 2.83 1.97 2.38 1.83 0.045*

Pro-social 8.35 2.06 8.38 8.38 0.50

Total score 15.33 6.54 13.56 6.18 0.003*

* Statistically significant

II. Strengths and Difficulties Questionnaire data: Proportional shift in children from a clinical level of difficulty (abnormal or borderline) to a normal level between the beginning and end of Family SMILES; analysis based on an Exact McNemar’s Test (n=72)Level of difficulties Pre-

programme (per cent)

Post- programme (per cent)

P-value

Normal range 49 65 0.01*

Clinical range (borderline or abnormal) 51 35



III. Adapted Rosenberg Self-Esteem Questionnaire for Children: Change in mean score, pre- and post-Family SMILES; analysis based on the Wilcoxon Ranked Sign Test (n= 61)

Mean at T1 Standard Dev. at T1


P-Value

Total score 19.16 5.10 21.44 5.82 0.003*


IV. HoNOSCA: Change in mean score, pre- and post-Family SMILES; analysis based on the Wilcoxon Ranked Sign Test (n=47)



P-Value

Total score 8.91 4.81 7.96 4.89 0.73

V. CAPI: Change in mean scores, pre- and post-Family SMILES; analysis based on the Wilcoxon Ranked Sign Test (n= 22)

Mean at T1

Standard Dev. at T1

Mean at T2

Standard Dev. at T2

P Value

Distress 185.73 62.86 168.55 70.88 0.02*

Rigidity 20.91 18.89 18.09 16.71 0.19

Unhappiness 39.95 20.25 31.59 17.39 0.001*

Problems with child

7.45 7.60 6.14 6.81 0.33

Problems with family

17.68 13.13 14.82 13.92 0.21

Problems with others

17.50 6.13 17.00 6.17 0.48

Total score 289.23 99.62 256.18 100.12 0.03*

Ego strength 9.73 8.93 12.50 9.69 0.01*

Loneliness 11.55 3.08 11.27 3.38 0.70



Appendix 8: Framework themes and sub-themes and snapshot from NVivo1.0 Analysis of standardised measures

2.0 Practitioner case closure summary

3.0 Family environment

4.0 Changes after or during the programme

4.1 Talking about the issue and impact

4.2 Knowledge or confidence in dealing with crisis situations

4.3 Change in own behaviour and impact

4.4 Change in feelings about self, others, situations

4.5 Change in other behaviour and impact

4.6 Change in relationship and impact

4.7 Change in family situation and impact

4.8 Change in aspiration, motivation and impact

4.9 Change in understanding about issue

4.10 Change in using other services

5.0 Role of the programme in bringing about change or not

5.1 Activities that helped or not

5.2 Practitioner support

5.3 Peer support (for children)

5.4 Safety plan

5.5 External factors


5.6 Image of the NSPCC

6.0 Expectations met or not, overall

7.0 Suggestions

7.1 Suggestions about timings; logistics

7.2 Suggestions about programme content

7.3 Suggestions about separate or joint working with child and parent.


Figure 3: Screenshot of analysis in NVIVO

www.nspcc.org.ukregistered charity numbers 216401and SC037717. photography by Jon Challicom, posed by models.

Evaluation of faMily SMilES - NSPCC€¦ · Children on Family SMILES go to a group with other...

Documents

Transcript of Evaluation of faMily SMilES - NSPCC€¦ · Children on Family SMILES go to a group with other...