2010 Annual State of Hawaii Forensic Mental Health Examiner Training Conference, Kaneohe, Hawaii...

2010 Annual State of Hawaii Forensic Mental 2010 Annual State of Hawaii Forensic Mental Health Examiner Training Conference, Kaneohe, Health Examiner Training Conference, Kaneohe,

HawaiiHawaii

Quality of Conditional Release Quality of Conditional Release Reports Submitted to the Reports Submitted to the

Hawaii JudiciaryHawaii Judiciary

2010 Annual State of Hawaii Forensic Mental Health 2010 Annual State of Hawaii Forensic Mental Health Examiner Training Conference, Kaneohe, HawaiiExaminer Training Conference, Kaneohe, Hawaii

• This project examined quality of Conditional This project examined quality of Conditional Release (CR) reports submitted to the Hawaii Release (CR) reports submitted to the Hawaii Judiciary. Hawaii utilizes a three panel system for Judiciary. Hawaii utilizes a three panel system for assessing trial competency, criminal responsibility, assessing trial competency, criminal responsibility, and conditional release, where two psychologists and conditional release, where two psychologists (one Department of Health and one community-(one Department of Health and one community-based) and one community-based psychiatrist based) and one community-based psychiatrist submit independent reports to the Court. submit independent reports to the Court.

• HRS 704-411 provides criteria for post-acquittal HRS 704-411 provides criteria for post-acquittal CR, with a focus on “danger to self or the person of CR, with a focus on “danger to self or the person of property of others” and whether the defendant property of others” and whether the defendant may be safely placed in a less restricted setting. may be safely placed in a less restricted setting.


• Utilizing a survey instrument based on nationally-Utilizing a survey instrument based on nationally-derived quality standards, 150 conditional release derived quality standards, 150 conditional release (CR) reports submitted to the First Circuit Court (CR) reports submitted to the First Circuit Court were examined. The research replicated a previous were examined. The research replicated a previous quality study of Competency to Stand Trial (CST) quality study of Competency to Stand Trial (CST) reports (Robinson & Acklin, 2010). reports (Robinson & Acklin, 2010).

• The current study examined six report quality The current study examined six report quality elements: 1) Data; 2) Legal/Ethical; 3) Historical; 4) elements: 1) Data; 2) Legal/Ethical; 3) Historical; 4) Practical; 5) Clinical Assessment/Diagnostic; and 6) Practical; 5) Clinical Assessment/Diagnostic; and 6) Opinion Rationale. We also examined usage of Opinion Rationale. We also examined usage of psychological testing and forensic assessment psychological testing and forensic assessment instruments, and agreement between examiners instruments, and agreement between examiners and between examiners and judges. and between examiners and judges.


• The survey instrument was comprised of 44 The survey instrument was comprised of 44 items that assessed report comprehensiveness items that assessed report comprehensiveness and quality. The items were equally weighted and quality. The items were equally weighted and summed to produce a quality score (QC). and summed to produce a quality score (QC). Each item was coded 0, 1, or 2. The QC as Each item was coded 0, 1, or 2. The QC as calculated by dividing the total score for each calculated by dividing the total score for each evaluation by the maximum possible score for evaluation by the maximum possible score for a report in that category and converting that a report in that category and converting that number into a percentage score. The QC score number into a percentage score. The QC score ranged from 0 to 100. Report quality criterion ranged from 0 to 100. Report quality criterion was set at 80%.was set at 80%.


• It was hypothesized that It was hypothesized that • 1) CR reports would fail to achieve 80% total 1) CR reports would fail to achieve 80% total

possible QC score, possible QC score, • 2) Department of Health examiners would 2) Department of Health examiners would

produce higher quality reports than produce higher quality reports than community-based evaluators, community-based evaluators,

• 3) Community-based psychiatrists and 3) Community-based psychiatrists and community-based psychologists would not community-based psychologists would not differ in report quality, differ in report quality,

• 4) Reports written by board-certified 4) Reports written by board-certified examiners would be significantly higher in QC examiners would be significantly higher in QC than non-board certified evaluators, than non-board certified evaluators,


• 5) Psychologists would more frequently utilize 5) Psychologists would more frequently utilize empirically-validated assessment instruments than empirically-validated assessment instruments than psychiatrists, psychiatrists,

• 6) Concordance of agreement for CR decisions 6) Concordance of agreement for CR decisions between pairs of examiners would reach between pairs of examiners would reach kappakappa greater than or equal to .60), and greater than or equal to .60), and

• 7) Concordance of agreement between at least two 7) Concordance of agreement between at least two examiners and judicial conditional release examiners and judicial conditional release determination would reach levels (determination would reach levels (kappakappa greater than greater than or equal to .60).or equal to .60).


• Findings revealed that CR report quality Findings revealed that CR report quality was mediocre across the board. None of was mediocre across the board. None of the 150 reports achieved the 80% quality the 150 reports achieved the 80% quality criterion. QC scores ranged from 32.35 to criterion. QC scores ranged from 32.35 to 75.86 (MQC=53.22, SD = 8.35, median = 75.86 (MQC=53.22, SD = 8.35, median = 52.94). 52.94).


• Data and Practical elements had the highest QC Data and Practical elements had the highest QC scores. On the Ethical element, informed consent scores. On the Ethical element, informed consent was documented in only 52% of the reports. was documented in only 52% of the reports. Historical elements were often entirely missing Historical elements were often entirely missing from reports. For example, social history was from reports. For example, social history was absent from 93.3% of the reports and family history absent from 93.3% of the reports and family history was absent in 66% of reports. Reports commonly was absent in 66% of reports. Reports commonly included the defendant’s psychiatric diagnosis included the defendant’s psychiatric diagnosis (96%). Over half of the reports did not explain how (96%). Over half of the reports did not explain how the psychiatric condition was a factor in the the psychiatric condition was a factor in the defendant’s capacity for release (56.7%) or how defendant’s capacity for release (56.7%) or how the defendant’s diagnosis might impact risk for the defendant’s diagnosis might impact risk for violence or recidivism (52%). violence or recidivism (52%).


• In 97 of 150 reports (64.7%) examiners indicated that In 97 of 150 reports (64.7%) examiners indicated that the defendant might pose a danger to himself or others, the defendant might pose a danger to himself or others, but did not provide information about what factors or but did not provide information about what factors or circumstances might heighten or mitigate risk. circumstances might heighten or mitigate risk.

• In only 52 cases (34.7%), did the FMHE provide a In only 52 cases (34.7%), did the FMHE provide a complete rationale for the opinion on the defendant’s complete rationale for the opinion on the defendant’s dangerousness. In 90 cases (60%), examiners provided dangerousness. In 90 cases (60%), examiners provided a complete rationale (Grisso, 2003) for the CR a complete rationale (Grisso, 2003) for the CR recommendation. In 80 cases (53.3%), reports recommendation. In 80 cases (53.3%), reports identified conditions under which the defendant could identified conditions under which the defendant could be safely released into the community. However, only be safely released into the community. However, only 11 (7.3%) reports included a comprehensive 11 (7.3%) reports included a comprehensive justification for the conditions of release.justification for the conditions of release.


• Overall report quality did not differ between Overall report quality did not differ between community-based psychiatrists and community-based psychiatrists and psychologists. Reports prepared by psychologists. Reports prepared by community-based psychologists had QC scores community-based psychologists had QC scores that ranged from 35.14 to 75.68 (M = 53.12, that ranged from 35.14 to 75.68 (M = 53.12, SD = 7.86). Reports prepared by community-SD = 7.86). Reports prepared by community-based psychiatrists had QC scores that ranged based psychiatrists had QC scores that ranged from 36.76 to 63.51 (M = 53.05, SD = 6.65). from 36.76 to 63.51 (M = 53.05, SD = 6.65). Reports submitted by community-based Reports submitted by community-based psychologists and psychiatrists did not differ in psychologists and psychiatrists did not differ in quality, t (98) = .05, p = .96.quality, t (98) = .05, p = .96.


• Report quality was compared between Report quality was compared between community-based and DOH psychologists. community-based and DOH psychologists. None of the reports by either group scored None of the reports by either group scored above the 80% quality criterion. Reports above the 80% quality criterion. Reports prepared by community-based psychologists prepared by community-based psychologists had QC scores that ranged from 35.14 to had QC scores that ranged from 35.14 to 75.68 (M = 53.12, SD = 7.86). Reports 75.68 (M = 53.12, SD = 7.86). Reports prepared DOH psychologists had QC scores prepared DOH psychologists had QC scores that ranged from 32.35 to 71.62 (M = 53.50, that ranged from 32.35 to 71.62 (M = 53.50, SD = 10.29). Report quality between the two SD = 10.29). Report quality between the two groups did not differ, t (91.67) = .21, p = .83. groups did not differ, t (91.67) = .21, p = .83.


• Report quality was compared between board-Report quality was compared between board-certified psychologists (American Board of certified psychologists (American Board of Professional Psychology) and non-board-Professional Psychology) and non-board-certified psychologists. QC scores for board-certified psychologists. QC scores for board-certified psychologists (N=16) ranged from certified psychologists (N=16) ranged from 32.35 to 65.63 (M = 46.84, SD = 11.24). QC 32.35 to 65.63 (M = 46.84, SD = 11.24). QC scores for non-board-certified psychologists (N scores for non-board-certified psychologists (N = 84) ranged from 35.14 to 75.68 (M = 54.54, = 84) ranged from 35.14 to 75.68 (M = 54.54, SD = 8.16). Contrary to expectation, reports SD = 8.16). Contrary to expectation, reports for non-board certified psychologists obtained for non-board certified psychologists obtained higher QC scores, t(18.13) = 2.61, p<. 05.higher QC scores, t(18.13) = 2.61, p<. 05.


• When examining psychological test usage, When examining psychological test usage, some type of assessment instrument was used some type of assessment instrument was used in 23.3% of the reports (N=35). Cognitive in 23.3% of the reports (N=35). Cognitive testing was administered in 22 reports (14.7%) testing was administered in 22 reports (14.7%) and personality testing was administered in 1 and personality testing was administered in 1 case (0.7%). Forensic Assessment Instruments case (0.7%). Forensic Assessment Instruments (FAIs) were completed in only 13 reports (FAIs) were completed in only 13 reports (8.6%). There was no significant difference in (8.6%). There was no significant difference in test usage between psychologists (N=100, test usage between psychologists (N=100, M= .28, SD= .53) and psychiatrists (N=50, M= .28, SD= .53) and psychiatrists (N=50, M=.20, SD=.40), t(148) =-.93, p=.35. M=.20, SD=.40), t(148) =-.93, p=.35.


• CR was granted in 99 cases (CR+)CR was granted in 99 cases (CR+)

• CR was denied in 48 cases (CR-)CR was denied in 48 cases (CR-)

• QC total and elements scores did not differ QC total and elements scores did not differ between CR+ and CR-) Hotellings T= between CR+ and CR-) Hotellings T= 100.02, F= 1986.15, df=7, p = .055).100.02, F= 1986.15, df=7, p = .055).

• Only the Rationale Quality Score Only the Rationale Quality Score approached significance (p = .08).approached significance (p = .08).


• Conditional Release agreement between Conditional Release agreement between examiners was poor. In 60 of the 150 reports examiners was poor. In 60 of the 150 reports (40%) all three-panel members reached CR (40%) all three-panel members reached CR consensus. In 69 of the reports, 46% of consensus. In 69 of the reports, 46% of reports (N = 23 cases), two of the three reports (N = 23 cases), two of the three examiners reached CR agreement. Thus, in examiners reached CR agreement. Thus, in 85% of the cases (N = 43, 86 reports) at 85% of the cases (N = 43, 86 reports) at least two of the three evaluators made least two of the three evaluators made similar CR recommendations. In 14% (N=7) similar CR recommendations. In 14% (N=7) of the cases, there was complete of the cases, there was complete disagreement among examiners. disagreement among examiners.


• DOH psychologists and community-based DOH psychologists and community-based psychiatrists agreed 60% (N=30 cases) of the time. psychiatrists agreed 60% (N=30 cases) of the time. Level of agreement was “poor” (Level of agreement was “poor” (k k = .27, p < .01). = .27, p < .01).

• Community-based and DOH psychologists agreed Community-based and DOH psychologists agreed 46% of the time (N = 23 cases). Level of agreement 46% of the time (N = 23 cases). Level of agreement in their CR recommendations was “poor” (in their CR recommendations was “poor” (k k = .10, p = .10, p = .34). = .34).

• Community-based psychiatrists and psychologists Community-based psychiatrists and psychologists agreed 60% (N=30) of the time. Their level of agreed 60% (N=30) of the time. Their level of agreement was “poor” (agreement was “poor” (k k = .21, p = .04). Level of = .21, p = .04). Level of agreement between all FMHE was very low, ICC agreement between all FMHE was very low, ICC = .068, sig. = .19, F= 1.23, = .068, sig. = .19, F= 1.23, df df = (49, 98), 95% CI: -.08 = (49, 98), 95% CI: -.08 to .25, for a “poor” level of agreement. to .25, for a “poor” level of agreement.


• Concerning CR judicial determination, the court Concerning CR judicial determination, the court and all three examiners reached consensus in and all three examiners reached consensus in 38.8% (N = 19) of the cases. In an additional 38.8% (N = 19) of the cases. In an additional 28.57% (N = 14) of cases, judicial 28.57% (N = 14) of cases, judicial determination was concordant with at least two determination was concordant with at least two of the examiners. Thus, for 67.3% (N = 33) of of the examiners. Thus, for 67.3% (N = 33) of cases, the court agreed with at least two of the cases, the court agreed with at least two of the evaluators on release decisions. Agreement evaluators on release decisions. Agreement level was “poor” between the court and level was “poor” between the court and majority recommendation of examiners (majority recommendation of examiners (kk = .26, p = .07, ICC = .30, p < .05; 95% CI: -.02 = .26, p = .07, ICC = .30, p < .05; 95% CI: -.02 to .53). to .53).


• Judges and community-based psychiatrists Judges and community-based psychiatrists reached agreement on release decisions in reached agreement on release decisions in 42 cases (85.7%) for a “good” level of 42 cases (85.7%) for a “good” level of agreement (agreement (kk=.66, p <.001). Judges and =.66, p <.001). Judges and community-based psychologists reached community-based psychologists reached consensus in 28 cases (57%) for “poor” level consensus in 28 cases (57%) for “poor” level of agreement (of agreement (kk= .16, p = .11). Judges and = .16, p = .11). Judges and DOH examiners made the same release DOH examiners made the same release opinion in 30 cases (61%) for “poor” level of opinion in 30 cases (61%) for “poor” level of agreement (agreement (kk= .30, p < .01). = .30, p < .01).


• Overall agreement for CR between all Overall agreement for CR between all mental examiners and judicial mental examiners and judicial determination was very “poor”, ICC= .11, determination was very “poor”, ICC= .11, p < .05; 95% CI: .00 to .26. In a high p < .05; 95% CI: .00 to .26. In a high number of cases there is complete number of cases there is complete disagreement among evaluators and in an disagreement among evaluators and in an equally high proportion of cases, judges equally high proportion of cases, judges disagreed with at least two of the disagreed with at least two of the evaluators.evaluators.


• The study suggests that quality of CR The study suggests that quality of CR reports submitted to the Hawaii judiciary is reports submitted to the Hawaii judiciary is poor. This has serious consequences for poor. This has serious consequences for the quality of CR jurisprudence, rights of the quality of CR jurisprudence, rights of defendants, and public safety. defendants, and public safety.


• A great deal of variability is found in A great deal of variability is found in examiner methods, report formats, and examiner methods, report formats, and findings. This variability appears to be findings. This variability appears to be reflected in the disarray found in reflected in the disarray found in agreement statistics between examiners agreement statistics between examiners and judges. The low utilization of FAIs for and judges. The low utilization of FAIs for dangerousness risk assessment suggests dangerousness risk assessment suggests that examiners are using unstructured that examiners are using unstructured clinical judgment in dangerousness clinical judgment in dangerousness opinions (Douglas, 2009). opinions (Douglas, 2009).


• Suggestions for improving report quality Suggestions for improving report quality include formal training in the conduct of include formal training in the conduct of forensic evaluations and report preparation forensic evaluations and report preparation (Robinson & Acklin, 2010). Report quality (Robinson & Acklin, 2010). Report quality may be specifically improved by may be specifically improved by standardizing methods, report formats to standardizing methods, report formats to insure coverage of elements, clearly insure coverage of elements, clearly described rationales integrating clinical described rationales integrating clinical findings, functional legal impairments, and findings, functional legal impairments, and forensic opinions (Grisso, 2003). forensic opinions (Grisso, 2003).


• The application of FAIs using structured The application of FAIs using structured professional judgment for assessing risk for professional judgment for assessing risk for violence (Douglas, 2009) is an emerging trend violence (Douglas, 2009) is an emerging trend nationally. nationally.

• Implementation of risk assessment Implementation of risk assessment methodologies is a powerful means to improve methodologies is a powerful means to improve the quality of CR reports and release decision-the quality of CR reports and release decision-making, especially since the latter is mandated making, especially since the latter is mandated specifically in the statutes and court orders. specifically in the statutes and court orders.

• The use of checklists may also be helpful in The use of checklists may also be helpful in improving report quality (Witt, 2010). improving report quality (Witt, 2010).

2010 Annual State of Hawaii Forensic Mental Health Examiner Training Conference, Kaneohe, Hawaii...

Documents

Transcript of 2010 Annual State of Hawaii Forensic Mental Health Examiner Training Conference, Kaneohe, Hawaii...