Skip Navigation

Institution: CLOCKSS Sign In as Personal Subscriber

Brief Treatment and Crisis Intervention Advance Access originally published online on June 29, 2005
Brief Treatment and Crisis Intervention 2005 5(3):261-278; doi:10.1093/brief-treatment/mhi019
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
5/3/261    most recent
mhi019v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Malcolm, A. S.
Right arrow Articles by Van Hasselt, V. B.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Malcolm, A. S.
Right arrow Articles by Van Hasselt, V. B.
Social Bookmarking
 Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2005. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oupjournals.org.

Original Article

Critical Incident Stress Debriefing and Law Enforcement: An Evaluative Review

   Abigail S. Malcolm, MS
   Jessica Seaton, MS
   Aimee Perera, PsyD
   Donald C. Sheehan, MA
   Vincent B. Van Hasselt, PhD

From the Center for Psychological Studies, Nova Southeastern University (Malcolm, Seaton, Perera, Van Hasselt) and the Law Enforcement Communication Unit, Federal Bureau of Investigation Academy (Sheehan)

Contact author: Vincent B. Van Hasselt, Professor, Center for Psychological Studies, Nova Southeastern University, 3301 College Avenue, Fort Lauderdale, FL 33314-7796. E-mail: vanhasse{at}nova.edu.

Emergency and disaster mental health may have century-old foundations, but its development as a field is far from complete (Everly, 1999). One of the more popular tactical interventions within the field is the Critical Incident Stress Debriefing (CISD) model of small-group crisis intervention, developed by Jeffery T. Mitchell (sometimes referred to as the "Mitchell Model"). CISD is but one intervention that falls within the strategic array of crisis interventions collectively referred to as Critical Incident Stress Management (Everly & Mitchell, 1999; Sheehan, Everly, & Langlieb, 2004). With the advent of CISD came a burgeoning number of case studies, personal accounts, and clinical research reports all focused on the efficacy of CISD. The purpose of this paper is to examine and critique the literature specifically addressing the Mitchell Model of CISD with law enforcement. Suggestions for directions that future research on CISD with police officers might take are discussed.

KEY WORDS: CISD, CISM, law enforcement, debriefing, group crisis intervention

High impact events, such as the Oklahoma City Bombing, the 1993 World Trade Center Bombing, the Pan Am Disaster, and the September 11, 2001, attacks, tax the resources of civilians and emergency responders alike (National Institute of Mental Health [NIMH], 2002). The response to these events and the resultant research revived an interest in the field of emergency mental health. A former Baltimore County firefighter developed the most influential intervention, to date, for emergency responders in the wake of critical incidents. Mitchell (1983) created Critical Incident Stress Debriefing (CISD) in the 1970s to help emergency responders quickly recover from a traumatic incident. CISD (sometimes referred to as the "Mitchell Model") is a formalized seven-phase group discussion pertaining to a critical incident, disaster, or traumatic experience. Later, Mitchell collaborated with Everly and developed the strategic Critical Incident Stress Management (CISM) system, "a comprehensive, integrated multi-component crisis intervention system" (Mitchell & Everly, 2001). CISM represents an effective, strategic continuum of care approach to crisis intervention (Sheehan, Everly, & Langlieb, 2004; Wagner, 2005).

As CISM developed during the 1990s, over 350 crisis response teams were created. The International Critical Incident Stress Foundation (ICISF) is composed of these teams. ICISF members are trained to understand the basic physiology and psychology of stress, stress management, and traumatic stress. Knowledge about traumatic stress allows CISM members to facilitate the appropriate CISM component in the acute crisis phase. Table 1 delineates the core components in the functional integration that CISM uses to adapt interventions for the appropriate incident.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Core Components of CISM

 
CISM continues to expand on its original model by incorporating new research and strategies from the fields of emergency mental health, psychology, faith interventions, military procedures, and individual CISM team experience (Everly, 2003a, 2003b; Everly & Langlieb, 2003). It is imperative to understand that the comprehensive, multifaceted, integrated nature of CISM was designed for the needs of emergency responders, their families, and their organizations (Everly & Mitchell, 1999). CISM was not developed to replace psychotherapy or to interfere with the natural recovery of human resiliency. Rather, CISM was designed to facilitate a triagelike process, for a nonpsychiatric population, in the face of a critical incident as a means of returning the individual and/or the organization to levels of prior functioning (Everly & Mitchell, 1999; Wessely & Deahl, 2003). Moreover, CISD was designed as a small-group crisis intervention within the overarching strategic CISM continuum of care (Everly & Mitchell, 1999).


    Critical Incidents and Critical Incident Stress Debriefings
 TOP
 Critical Incidents and Critical...
 Review of Research
 Research Findings
 Discussion: Practical Scientific...
 Conclusions and Considerations...
 References
 
A "critical incident" has been defined as a stressful event that is so consuming it overwhelms existing coping skills (Kureczka, 1996). A more functional definition describes a critical incident as an event that has the potential to interfere with a person's normal management of everyday stress. The Mitchell Model carefully differentiates between a critical incident and a crisis response. Again, a critical incident is the event itself, which is "critical" due to its potential to engender dysfunction. A crisis response is defined as the actual presentation of an individual whose coping resources have been overwhelmed by the incident and there is evidence of impairment (Caplan, 1964).

One approach to address reactions of emergency responders to critical incidents is debriefing. Unfortunately, many researchers have used the term debriefing to encompass a variety of incident–response interventions. This paper uses the terms CISD to refer to the Mitchell Model and debriefing to refer to all other descriptions of crisis intervention in the literature.

CISD consists of a group of emergency responders, all of whom were involved in the same critical incident, and the CISD intervention team members deployed for service. In law enforcement, the CISD team consists of at least one peer, who is a law enforcement officer, and at least one mental health professional. The CISD is composed of seven stages. The first stage, Introduction, takes the time to describe the process, rules of CISD (i.e., confidentiality), and expectations. During the Fact Phase, the second stage, officers are asked to say who they are and what their role was in the incident. The next stage (Thought Phase) asks the officer to share his or her first thoughts after the incident. The Reaction Phase, the fourth stage, explores the personal reactions surrounding the event. In the fifth stage, the Symptom Phase, signs and symptoms of critical incident stress are discussed and normalized. In the next stage, the officers are taught different ways of dealing with critical incident stress in their lives (Teaching Phase). Finally, the Reentry Phase encourages officers to discuss any other issues and ask questions. Most importantly, this phase focuses on returning the officer to duty. Each stage represents a gradual step designed to return the emergency responder to his/her precritical incident level of functioning (Mitchell, 1991).

Case Example: Oklahoma City
On April 19, 1995, at 9:02 a.m. a bomb exploded and destroyed the Alfred P. Murrah federal building in Oklahoma City, OK. One hundred and sixty-eight people were killed and more than 500 were wounded in the blast (Ottley, 2003). Although the immediate and long-term effects on emergency responders included anxiety, depression, suicide, increased alcohol consumption, sexual misconduct, increased divorce rates, and Posttraumatic Stress Disorder (PTSD), levels of these anticipated problems were lower than predicted (Owen, 1999; Plumberg, 2000; Raymond, 1999).

In the wake of this tragedy, researchers have dubbed the aftermath of Oklahoma City a "good" disaster, indicating that the general outcome surpassed what might be expected following a terrorist attack (Owen, 1999). The implementation of the strategic CISM system (and more specifically, CISD via Project Heartland and the Oklahoma Critical Incident Stress Management Network) helped to manage the harmful aspects of traumatic stress (Klinka, 1996, 1999). Contributing to the success of the outcome was the cooperation and collaboration among the multidisciplinary team of emergency response workers (firefighters, law enforcement officers, emergency medical support, clergy, and volunteers). According to City Manager Brown (1995), the team effort of the workers set a new standard in the face of disaster, named, "The Oklahoma Standard." The Oklahoma City response followed the CISM model, including CISD for police, and proved that if followed properly, it is an effective way to minimize the impact of a critical incident.


    Review of Research
 TOP
 Critical Incidents and Critical...
 Review of Research
 Research Findings
 Discussion: Practical Scientific...
 Conclusions and Considerations...
 References
 
Subsequent research from the Oklahoma City Bombing provided support for the CISD model of crisis intervention. However, CISD is not without its detractors, and there is existent research contradicting the case study support for the Mitchell Model (see reviews by Bledsoe, 2004; Rose, Bisson, & Wessely, 2002; Wessely, Rose, & Bisson, 2000). Controversy has always been associated with new interventions, and further empirical research is required to ascertain the effects of the CISD on reducing harm from critical incidents. However, although there is a modicum of research concerning the specialized area of emergency mental health, several studies offer evidence of the utility of CISD for law enforcement personnel. Many of these articles are personal accounts, of police officers themselves, and are primarily subjective and anecdotal reports. The studies discussed below were selected based on their (a) empirically oriented focus and (b) attempt to ascertain the heuristic value of CISD in law enforcement populations.


    Research Findings
 TOP
 Critical Incidents and Critical...
 Review of Research
 Research Findings
 Discussion: Practical Scientific...
 Conclusions and Considerations...
 References
 

  1. Robinson and Mitchell (1993) evaluated the impact of 31 debriefings on two groups: emergency services personnel (including 13 police officers) who participated in 18 of the 31 debriefings and hospital and welfare staff who participated in 13 of the 31 debriefings. The investigators developed their own evaluation questionnaire, asking participants to rate the impact of the critical incident and the value of the debriefing using a 5-point scale (1 = none; 5 = great) and their perceived impact of the event on other personnel and family. Additional open-ended questions (i.e., reasons this critical incident had an impact on them, other aspects of their lives that were affected by their recall of the critical incident including family and signs of stress) were included to collect subjective data (e.g., 40% of emergency service personnel indicated that their families were impacted by their own critical incident). Then, following any of the 31 debriefings, they mailed the questionnaire to persons who consented to being a participant in their study 2 weeks postdebriefing. Sixty percent of all the individuals who participated in the 31 debriefings consented and responded to the questionnaire.
    The hospital and welfare staff rated the impact of the critical incident, at the time of the event, an average of 3.3 compared to 2.2 for the emergency service personnel. When the evaluation questionnaire was administered, both groups showed a significant reduction in their average rated impact of the event, although the emergency service personnel showed a larger decrease. CISD was not only rated as valuable but also viewed as responsible, at least, in part, for a reduction in stress symptoms (e.g., fatigue, tearing, sleep disturbances) for both groups.
    Additionally, both groups rated the debriefing as considerably valuable to themselves and other personnel (a range of 3.8–4.5). Subjective data also were compiled about the broader effects of the critical incidents and the debriefings. For example, the evaluation showed that (a) the type of critical incident impacts the prevalence of reported stress symptoms, (b) over 50% of the participants stated that the event caused them to recall prior critical incidents, (c) the most common sign of stress in emergency service personnel was cognitive (e.g. sleep disturbance, preoccupation with the incident), and (d) the 96% of emergency responders who noted decrease in stress symptoms attributed part of their improvement to the debriefing, particularly because the debriefing provided an opportunity to talk about the incident.
  2. Nurmi (1999) assessed the impact of CISD on police, firefighters, and nurses involved in the 1994 rescue of the ferry Estonia off the coast of Finland. Nurses did not receive CISD because it was not hospital policy, whereas the police and firefighters did take part in debriefings. All three groups were assessed using the Impact of Event Scale-Revised, the Penn Inventory, the SCL-90-R, and a specially developed Perceived Satisfaction with CISD questionnaire. Nurmi found the nurses to be significantly more distressed than police officers and firefighters based on the differences in outcome using the abovementioned measures. Responses of the latter two groups on the Perceived Satisfaction questionnaire revealed that a majority (81%) found the debriefing to be useful.
  3. Bohl (1991) examined police officers in Southern California using the State-Trait Anxiety Inventory, the Beck Depression Inventory, and the Novaco Provocation Scale. All officers had experienced a critical incident 3 months prior to the assessment. Also, 40 underwent a brief psychological intervention as per departmental policy. A second group of 31 officers did not receive any intervention, again, due to department policy. Bohl found that the officers who participated in the brief intervention showed a statistically significant decrease in their rates of stress symptoms (e.g., depression, flashbacks, and anger).
  4. Smith and de Chesnay (1994) evaluated a CISD program with a South Carolina police department. Ten officers, from an agency of 100 employees, were interviewed by the primary investigator. The primary investigator contacted each of the 10 officers following a debriefing that he himself had led. Each officer was offered the opportunity to participate in the study or decline participation; they all accepted the offer. A semistructured interview format allowed the participants to discuss any area of the critical incident, the debriefing, and any related stress. Results indicated that 9 of the 10 officers felt that CISD was beneficial in helping to reduce their stress.
  5. Leonard and Alison (1999) assessed groups of Australian police officers (N = 60) who did (n = 30) and did not (n = 30) receive CISD following a shooting incident. The evaluation packet included (a) a questionnaire asking for details about the incident (e.g., prior experiences regarding the shooting incident, appraisal of personal safety during the shooting incident, support received from the department regarding the shooting incident), (b) the Coping Scale (Carver et al., 1989), and (c) the State-Trait Anger Expression Inventory. They found that the CISD group had more shots fired and more people killed or injured in the incident, had scored higher on active coping, and were significantly less angry than the non-CISD group, on all measures. Leonard and Alison suggested that officers in the non-CISD group may have reported elevated anger and poor coping skills due to the fact that CISD was not offered to them; of the 30 officers who did not receive CISD, it was stated that they either refused CISD or were "overlooked by the Department."
  6. Carlier, van Uchelen, Lamberts, and Gersons (1998) studied trauma in Dutch officers who responded to a plane crash. The investigators selected their groups from the 200 police officers who arrived at the crash site, of whom 45% were debriefed and 55% were not. The control group of nondebriefed officers consisted of officers who, according to Carlier et al. (1998), did not attend due to "operational reasons" (e.g., length of service time on scene) or because officers chose not to (e.g., not on duty at time of debriefing, volunteered to continue with service duty uninterrupted). The investigators assessed 105 officer participants, 59 of whom had not been debriefed and therefore comprised their control group. The investigators completed the testing at the police departments. They used the structured interview for PTSD at both 8 and 18 months postdisaster.
    The 8-month assessment revealed that although some of the officers displayed PTSD symptoms postdisaster, only two met DSM-III-R criteria; there were no significant differences between the research groups. Results were similar for the 18-month postdisaster evaluation.
  7. Carlier, Voerman, and Gersons (2000) described their intervention as the Mitchell Model of CISD and administered debriefings immediately after an unspecified trauma. The experimental group of officers (n = 86) had been recruited for participation following their CISD. One control group of officers (n = 82) did not receive the debriefing due to operational reasons, which were not stated in the article. Another "external control group" (n = 75) consisted of officers who experienced a previous traumatic event prior to the institution of debriefing offerings within the department.
    The pretest and 24-hr postincident test phases involved administration of the Spielberger State-Trait Anxiety Inventory to debriefed and nondebriefed officers but not to the external control group. One week postincident, the investigators assessed the debriefed and nondebriefed groups using the Self-Rating Scale for PTSD and the Peritraumatic Dissociative Experiences Questionnaire. The external control group was assessed with the Impact of Events and the Peritraumatic Dissociative Experiences Questionnaire. The debriefed and non-debriefed groups were again evaluated 6 months posttrauma using the structured interview for PTSD, the Anxiety Disorders Schedule-Revised, additional DSM-IV-related issues (e.g., dissociation, history of psychiatric illnesses), and several open-ended questions regarding perceived support, satisfaction with the debriefing, and stress.
    The investigators found no significant differences between groups in psychological symptoms, number of sick days taken, or rate of returning to work. However, 98% of the officers reported satisfaction with the debriefing sessions.
  8. The NIMH (2002) reviewed 17 studies (see Table 4) that used self-described psychological debriefings to address groups and individuals following various disasters and traumatic events. The review evaluated articles for standards of research design that were met, were unmet, or raised other concerns. Reviewed articles included a wide range of populations (medical patients, bank tellers, law enforcement officers). The majority of the group crisis intervention "debriefings" applied to emergency services, and military personnel yielded positive outcomes. However, the debriefings applied to individual medical and surgical patients led to no significant change and, in some cases, indicated negative outcomes.
  9. Young (2003) focused on the effectiveness of debriefings with law enforcement populations. The overarching goal of this study was to assess the effect of periodic stress debriefings (debriefings were based on the Mitchell model of CISD) on officers' levels of depression and PTSD symptoms. The author suggested that the lack of statistically significant findings in this study reflected on the debriefings' lack of focus on internal police department stressors (e.g., supervisors, administrative duties). It was not suggested that the results indicated a negative effect from the debriefings; in fact, subjectively, police viewed the debriefings as helpful and positive (Young, 2003).
  10. Sheehan et al. (2004) utilized the application of best practices to provide law enforcement with practical, yet empirically based, information regarding the issue of law enforcement response to critical incidents and critical incident stress. Their study surveyed 11 organizations, including federal, state, and city agencies, and ascertained their response to major critical incidents (e.g., Waco, 9/11). Using both interviews and written descriptions, the authors cited how these agencies structurally responded to large-scale disasters and relayed their findings to offer tactical and stylistic response options for other law enforcement agencies. While this study supported the use of postincident crisis intervention, such as CISM, a resounding conclusion from all agencies reviewed in the article focused on the lack of preincident responses. The authors suggested increased attention to applying a comprehensive approach in response to major critical incidents, including (a) preincident education, (b) preincident training, and (c) early-warning screening.
  11. Meta-analysis findings of Roberts, Everly, & Camasso (2005) included the effect size of multicomponent CISM. The high-average, 2.11, effect size for multicomponent CISM provided statistical significance and practical comparability that most notably supports the use of debriefings within a comprehensive, multifaceted, integrated continuum of crisis intervention. In sharp contrast, the studies of one-shot relatively brief debriefings had low overall effect sizes and were not statististically significant. This study offers federal, state, and local agencies a backbone of strong research to develop crisis intervention programs, including law enforcement.
  12. Wagner (2005) approached the issue of CISD effectiveness within the field of emergency services, including law enforcement, with a review of literature. Wagner addresses the controversy of CISD in the field of emergency mental health and implied an increased need for critical evaluation of relevant literature, specifically in reference to the application of CISD to specialized populations (e.g., emergency service workers). The overall conclusion of this review supports the use of CISD; however, the author is rightfully careful to specify the effectiveness of CISD when applied with emergency service workers and as part of a comprehensive response program.


View this table:
[in this window]
[in a new window]
 
TABLE 4. Comparison of Debriefing Effectiveness According to Study Group Participants

 

    Discussion: Practical Scientific Suggestions
 TOP
 Critical Incidents and Critical...
 Review of Research
 Research Findings
 Discussion: Practical Scientific...
 Conclusions and Considerations...
 References
 
The pattern of disparate findings in the articles reviewed above mirrors CISD research in general (Wagner, 2005; Wessely & Deahl, 2003). Specifically, the reports above have many of the same terminological and research design shortcomings as other CISD investigations (Everly, Flannery, & Mitchell, 2000; Everly & Mitchell, 2000). Because in this paper we focuse on the variable of law enforcement culture, concerns we have identified regarding research on CISD with this population are illustrated in Table 2.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Concerns for CISD Research With Law Enforcement

 
Assessment
Overlapping Culture Concerns.
Although each area in Table 2 identifies specific issues in the research, the overlapping aspect of these observations deserves attention. For example, conducting research using police officers sustains its own inherent challenges due to shift work, unwillingness to disclose, and lack of validated measurements. An investigator knowledgeable about police culture would likely inquire about how methodology was adjusted for a 24-hr occupation or how researchers gained access and trust within any given department. However, before addressing law enforcement culture, psychosocial instruments used in the studies require scrutiny with regard to their potential threat toward the strength and validity of their findings.

Measurements.
One of the critical issues in assessment is balancing objective and subjective measurements. Police officers must appear as if they are in control, well-adjusted, and calm. Therefore, a reliance on objective measurements, particularly those which are face valid (e.g., Beck Depression Inventory), will not accurately assess police officer stress. Similarly, dependence on personal interviews to assess the effectiveness of CISD does not produce unequivocal results.

This lack of useful and valid assessment tools permeates CISD research in general. Specifically, there is an overreliance on self-reported measurements and a lack of objective assessment (NIMH, 2002). In addition, there is a paucity of evaluation tools validated on police officers: a notable exception is the Law Enforcement Officer Stress Survey (LEOSS; Van Hasselt, Sheehan, Sellers, Baker, & Feiner, 2003). However, a mixture of measurements provides a more well-rounded assessment to balance out the anticipated "cop culture" concerns. It is also important to note that most valid assessment devices that measure psychological symptoms will not answer questions about the perceived effectiveness of CISD. If investigators intend to examine how CISD affects particular psychological symptoms, they must employ instruments specific to the symptoms they are examining (e.g., State-Trait Anger Scale, Beck Depression Inventory). However, if they wish to examine the efficacy of CISD, researchers must be sure to (a) assess with psychometrically sound instruments and (b) generalize from symptom-specific evaluation tools (e.g., Robinson and Mitchell, 1993, used a Likert-type scale to measure perceived effectiveness; Nurmi, 1999, developed a Perceived Satisfaction questionnaire). In addition, several of the articles reviewed mentioned occupational records (e.g., absenteeism, sick and family leave, disciplinary history). However, none implied that these type of data reflect the impact of CISD. It is important to note that one of the primary goals of CISD is to return the officer to a premorbid (i.e., precritical incident) level of functioning, which includes getting back to work without mandates or discipline (Wessely & Deahl, 2003).

Assessment issues regarding law enforcement and CISD are not strictly police culture concerns. Indeed, they are relevant to research on CISD in general. However, assessment of law enforcement carries its own challenges and must be addressed when conducting CISD research with this population (Sheehan & Van Hasselt, 2003). Table 3 outlines suggestions specific to this problem pulled from previously reviewed studies.


View this table:
[in this window]
[in a new window]
 
TABLE 3. Suggestions for Clinical Assessment With Law Enforcement

 
Control Groups.
Robinson and Mitchell (1993) discussed the complexity, but underscored the importance, of creating adequate control groups for CISD investigations. Much CISD research does not have a control condition that includes police officers. The lack of such controls makes it difficult to generalize the favorable findings of Robinson and Mitchell and of Nurmi (1999) to police officers. In contrast, Smith and de Chesnay (1994) only used police officers for the in-depth personal interviews; however, they did not include a control group to confirm their subjective results.

Creating adequate control conditions that ethically control the variables of interest between groups is problematic. However, there are basic differences, which, if addressed, would make the experimental control in law enforcement/CISD research more reliable: (a) increased use of law enforcement officers in control groups, (b) improved awareness of cop culture in the selection of control groups within an individual law enforcement agency, and (c) improved attention to departmental differences in the selection of control groups between various law enforcement agencies. For example, many of the articles reviewed here describe the demographical differences within and between the groups; there was no mention of departmental differences (Bohl, 1991; Carlier et al., 2000). Departmental differences that could weaken the utility of control groups include (a) law enforcement agencies that do or do not employ CISD/CISM, (b) agencies that make CISD standard operation, and (c) agencies where the command staff may offer CISD but fail to support the CISM model.

Overlapping with departmental disparities, there may also be individual differences between officers who choose to attend versus those who decline or are not offered the CISD. Leonard and Alison (1999) created a control group from officers who were either "... overlooked by the Department or refused the debriefing." Similarly, Carlier et al. (1998) incorporated a control condition of officers who were not involved with the debriefing due to operational reasons. Without clarification, the utility of a control condition that includes officers who were disregarded by their own department, declined the intervention, or were not offered the opportunity is questionable.

Future research regarding CISD and law enforcement may not be able to completely address individual and departmental differences affecting control groups. However, awareness of how these issues could impact the validity of research findings lead to the following suggestions: (a) improve assessment batteries given to both groups (to rule out significant personality differences), (b) conduct comparative studies contrasting departments that mandate CISD/CISM versus departments that do not (to lower the significance of departmental disparities), and (c) continue to make efforts to include law enforcement in control groups if results are to be generalized to police personnel.

CISM/CISD: Specific Concerns with Law Enforcement Research
Definitions and Terminology.
A persistent issue in CISD research concerns terminology and application. The nonspecified application of various debriefing models and the misleading definitions of debriefing constitute the most consistent flaw in CISD research, in general, and work with police officers, in particular. Beginning with the event itself, an operational definition of a critical incident would undoubtedly enhance any study focusing on CISD (Everly & Mitchell, 2000). For example, Nurmi (1999) explained the sinking of the Estonia, and that description left no doubt about the capacity for it to have been labeled a critical incident. The NIMH (2002) review found that many studies in the area did not define the "traumatic exposure." Or, the event that was eliciting the debriefing was itself unclear. A formal definition of a critical incident initiates the process of designating the model of debriefing being analyzed in the study.

When CISD is the focus of an investigation, the researchers should credit the intervention as the Mitchell Model as Carlier et al. (2000) did in their report. There is an extant body of work that accuses debriefings of being ineffective and even harmful to participants (Bisson & Deahl, 1994; Bledsoe, 2004; NIMH, 2002; Wessely et al., 2000). Although controversy fuels empirical integrity, there seems to be a simple answer to this debate about the validity of CISD. Careful review of the articles that indicate a lack of efficacy reveals numerous debriefings conducted in one-on-one settings, with hospital patients, and with ill-defined methods of crisis intervention. These debriefings do not follow the CISD guidelines, the Mitchell Model, or the ISCIF standard of care. Although these articles appear to show a lack of effectiveness for the completed debriefings, they do not have external validity relative to CISD as designed by the ICISF (Bisson & Deahl, 1994; Everly & Mitchell, 1999; NIMH, 2002; Wessely et al., 2000).

Proper Application of CISD/CISM.
CISD team members ostensibly practice the standardized Mitchell Model small-group CISD. Therefore, it can be assumed that CISD performed by trained CISD team members would be more internally valid and reliable than debriefings performed by other individuals. Some of the articles presently reviewed describe their debriefing as CISD (Carlier et al., 1998, 2000; Leonard & Alison, 1999; Nurmi, 1999; Robinson & Mitchell, 1993; Smith & de Chesnay, 1994). However, given the frequent terminological confusion and the importance of both clinical and statistical significance, it is crucial that individuals performing the CISD are properly trained. Nurmi (1999) and Robinson and Mitchell (1993) are the only two investigations to note that their debriefers were trained according to the Mitchell Model. When training standards are clarified, there is a clearer link between intervention and outcome of the debriefing/CISD.

Most of the reports reviewed here justified their description of CISD and explained how they identified with the Mitchell Model (Carlier et al., 2000; Leonard & Alison, 1999; Nurmi, 1999; Smith & de Chesnay, 1994; Wagner, 2005). Bohl (1991) explained that she used her personal model of debriefing that was adapted from the Mitchell Model. CISD is a flexible intervention but not interchangeable with any strategy that bears a likeness to debriefing (Mitchell, 2003). Nurmi (1999) and Leonard and Alison (1999) consistently used the term CISD in their articles, avoiding confusion between CISD and other interventions described as debriefings. However, although Smith and de Chesnay (1994) described their group procedure as CISD, it was difficult to ascertain if they were actually using the Mitchell Model. Similarly, Young (2003) cited that the debriefing in the study was based on the Mitchell Model; however, the description of the debriefing and its outcome appeared disparate. Terminological clarification is critical in research to avoid the ongoing confusion concerning CISD and other interventions mistakenly referred to as debriefings.

CISD was originally created to address the emergency mental health needs of public health responders, such as firefighters, paramedics, law enforcement officers, teachers, and nurses (Mitchell & Everly, 2001). CISD has never been suggested for civilians, medical patients, and other nonemergency responders who are facing long-term stressors or severe mental health problems. The NIMH (2002) review offers a comparison model for other types of CISD research. It shows how this approach is ineffective when used with nonemergency responders. An examination of Table 4 reveals that debriefings are not helpful for medical patients compared to the near 70% positive outcome rate of debriefing efficacy when administered to law enforcement officers, emergency responders, and military personnel. In addition, the negative-outcome studies involving police and other emergency responders included the reports by Carlier et al. (1998, 2000), reviewed earlier.

The NIMH (2002) review also indicates a number of concerns with CISD research, including (a) inadequate random assignment, (b) lack of comparison groups, (c) overemphasis on self-report measures, (d) inattention to preintervention measurement, and (e) an undefined exposure/critical incident. More importantly, the NIMH (2002) review highlights other problems, such as control groups that opted out for operational reasons, debriefing interventions that were not defined, and intervention groups that contained more severe injury and symptoms rates than control conditions predebriefing. In addition, none of the studies reviewed in NIMH (2002) discussed the application of a comprehensive CISM that included debriefing as a necessary tool within a more comprehensive plan of stress management.

Finally, CISD was designed within the integrated system of CISM (Everly & Mitchell, 1999; Roberts et al., 2005; Sheehan et al., 2004). The literature on CISM clearly states that none of the CISM components are meant to be administered as a solitary means of addressing critical incidents stress (Wessely & Deahl, 2003). Kureczka (1996) purports that preincident stress education (a CISM component) is the most important element in addressing critical incident stress in police officers. Volkman (2001) discussed the evolution of CISM in the "blue culture" and the importance of training, stress management, and CISD in law enforcement populations. Unfortunately, there is no literature, to date, that addresses the heuristic value of CISD within the appropriate CISM model. Indeed, the only mention of the CISM model being followed in any of the studies we reviewed were from (a) the Roberts et al. (2005) presentation (who reported a high average effect size of multicomponent CISM from their meta-analysis), (b) the Sheehan et al. (2004) article (they reported best practices for response mass disasters and noted that most agencies failed to provide a continuum of care beyond one-dimensional crisis intervention), and (c) the Wagner (2005) review article (which made specific recommendation for the multicomponent usage of CISM with emergency responders). Fortunately, current literature has ascertained this weakness in CISD research and has begun the process of improvement with suggestions for research and practical applications for law enforcement agencies (Sheehan et al., 2004).

Potential for Randomized Studies.
Experimentally rigorous research requires randomized controls and assignment to interventions. Robinson and Mitchell (1993) noted the difficulty in obtaining a baseline (precritical incident) level of functioning; the current controversy concerning randomized studies in CISD highlights this problem. Although most investigations did not employ a randomized control trial, there appears to be a false consensus in the literature that randomized studies are not possible when using CISD (Carlier et al., 2000). Even opponents of CISD admit that such studies are possible and would improve extant work in the field (Wessely & Deahl, 2003).

Critical incidents are unpredictable, and withholding intervention is unethical. However, with increasing focus on officer assessment at hiring, there seems to be the possibility of collecting baseline data for a longitudinal study of CISD effectiveness within police personnel (Hibler & Kurke, 1995). Similarly, with the prevalence of disparate debriefing models (Bohl, 1995; Dyregov, 1998), there is the potential for comparing interventions by utilizing adequate research designs.

Law Enforcement Culture
Police Culture.
Volkman (2001) describes several cultural characteristics of police officers: (a) pride and perfectionism, (b) rigidity, (c) emphasis on bonding between officers (i.e., "us vs. them" mentality), (d) development of language unique to the profession (e.g., using dispatch codes), and (e) focus on safety and security, both at work and at home. Although police culture cannot be explained in a list of themes, writings addressing this closed culture concur that the same cultural forces that create the effective role of an officer often work to destroy the same officer: process and action oriented, structurally organized, cohesion and loyalty, rigidity, "code of silence," and a sense of duty (Kirschman, 1995; Paoline, 2001; Volkman, 2001). It is particularly challenging then to enter this closed culture as an outsider and ask officers to divulge personal information that reflects the human vulnerabilities their occupational role often works hard to repress or deny. In fact, some of the articles reviewed here were authored by individuals who were present at the time of the incident or were law enforcement officers themselves; these individuals were part of the cop culture they were investigating (Nurmi, 1999; Smith & de Chesnay, 1994). Although research methods may hinder investigators from being part of the CISD team, it may be useful to break the "outsider" role by having a CISD team member or police officer introduce the research team. Further, research teams would greatly benefit from a detailed prestudy debriefing to explain their presence and to allow officers to ask questions.

However, even if research teams are able to avoid having their presence serve as a cultural deterrent, investigators must address how the police culture affects their work. In addition, the presence of supervisors during CISD must also be addressed as an influencing factor. A heightened sensitivity to police culture may also impact decisions regarding the location of the CISD, media involvement in the critical incident, any political/bureaucratic issues that arise, the presence of a CISM peer who is also police officers, and the timing of the CISD relative to their shift work. All these variables play a salient role in routine police work and will impact evaluation and intervention.

Acceptance of CISD/CISM.
Volkman (2001) contends that police culture is on the verge of accepting CISM as an opportunity to break their code of silence and improve stress management practices. In addition, many of the articles discussed here reported a subjective acceptance of CISD by police officers, often in the face of nonsupporting data. Researchers in this area need to address the acceptance of CISD/CISM within the law enforcement community.

However, many efforts focus on proving the efficacy of CISD before providing evidence of its acceptability in the law enforcement profession. Investigators must be cognizant of the acceptance of CISD and its impact on the efficacy of CISD itself. Several of the articles reviewed above mentioned that officers did not participate in CISD due to operational reasons or because the department did not allow their use (Bohl, 1991; Carlier et al., 1998). Although individual differences in the acceptance of CISD are difficult to rule out, departmental policies influencing its acceptance are not, and could easily be avoided.

Overlapping Cultural Concerns.
As mentioned earlier, the assessment of police officers is greatly influenced by their culture and approach to testing. However, simple descriptions of the assessment tools, with reference to their relevant strengths and weaknesses when being used with emergency responders, address the issue. Only one of the articles we reviewed offered such a description (Nurmi, 1999). With the exception of the LEOSS, there are few evaluation methods validated specifically for police. Meanwhile, it would improve the quality of research to take the extra effort and address the challenge of clinical assessment with law enforcement personnel. Similarly, although research in general has struggled with the voluntary aspect of creating control groups, this is particularly problematic in work with law enforcement professionals due to the closed culture, discussed earlier.


    Conclusions and Considerations for Future Research
 TOP
 Critical Incidents and Critical...
 Review of Research
 Research Findings
 Discussion: Practical Scientific...
 Conclusions and Considerations...
 References
 
CISD and CISM have gained considerable recognition over the past two decades as stakeholders in the field of emergency mental health, and the research continues to grow. The focus here highlighted three overlapping concerns in the study of CISD in law enforcement. The culture of law enforcement continues to be a challenge for all research regarding police personnel; CISD investigation is no exception. Similarly, definitions and applications of CISD terminology continue to plague the majority of CISD research, regardless of the population of interest. Finally, the application of solid research and assessment methodology poses challenging problems when working with special populations and multifaceted interventions.

CISD with law enforcement clearly warrants increased investigative attention in the future. However, the existing literature, albeit limited, appears to support the use of group CISD with emergency personnel (Everly, Boyle, & Lating, 1999; NIMH, 2002; Sheehan et al., 2004; Wagner, 2005; Young, 2003). Although improved research would undoubtedly enhance confidence in CISD application, such work must be well defined, based on sound design principles, and generalizable to the population of interest. In addition to the three overlapping concerns mentioned above, there are other complex and challenging questions that warrant attention in CISD research with law enforcement. For example, are there occupationally specific criteria to consider when addressing the CISD Reentry phase, which impacts return to preincident levels of functioning? Does acceptance of CISD from departments and supervisors affect the impact of the procedure? Does application of CISM in a department strengthen the outcome of CISD?

We have examined basic guidelines and considerations based on extant research in an effort to instigate and encourage continued study of CISD in law enforcement. The evolving needs in the field of emergency mental health require an adherence to standardized, empirically driven procedures and careful delineation of variables that may ultimately affect the outcome in this important area.


    Acknowledgments
 
Preparation of this paper was supported, in part, through a Nova Southeastern University President's Faculty Scholarship Award (Grant 338339) to the last author. Portions of this paper were presented at the Council of Police Psychological Services Conference in Fort Lauderdale, Florida (2003), and the World Congress on Stress, Trauma, and Coping in Baltimore, Maryland (2005).


    References
 TOP
 Critical Incidents and Critical...
 Review of Research
 Research Findings
 Discussion: Practical Scientific...
 Conclusions and Considerations...
 References
 

    Amir, M., Weil, G., Kaplan, Z., Tocker, T., & Witztum, E. (1998). Debriefing with brief group psychotherapy in a homogenous group of noninjured victims of a terrorist attack: A prospective study. Acta Psychiatrica Scandivavica, 98, 237–242.

    Bisson, J. I., & Deahl, M. P. (1994). Psychological debriefing and prevention of posttraumatic stress. British Journal of Psychiatry, 165, 717–720.[Free Full Text]

    Bisson, J. I., Jenkins, P. L., Alexander, J., & Bannister, C. (1997). Randomized controlled trial of psychological debriefing for victims of acute burn trauma. British Journal of Psychiatry, 171, 78–81.[Abstract/Free Full Text]

    Bledsoe, B. (2004, January). CISM debunk. Retrieved May 26, 2005, from http://www.bryanbledsoe.com/cismdebunk.htm

    Bohl, N. (1991). The effectiveness of brief psychological interventions in police officers after critical incidents. In J. T. Reese, J. M. Horn, & C. Dunning (Eds.), Critical incidents in policing (pp. 31–38). Washington, DC: U.S. Department of Justice.

    Bohl, N. (1995). Professionally administered critical incident debriefing for police officers. In M. I. Kurke & E. M. Schriver (Eds.), Police psychology: Into the 21st century (pp. 169–188). Hillsdale, NJ: Erlbaum.

    Brown, D. (1995). Oklahoma City, April 19, 1995. Public Management, 77, 6–9.[find it @ Stanford]

    Caplan, G. (1964). Principles of preventative psychiatry. New York: Basic Books.

    Carlier, I. V. E., van Uchelen, J. J., Lamberts, R. D., & Gersons, B. P. R. (1998). Disaster-related posttraumatic stress in police officers: A field study of the impact of debriefing. Stress Medicine, 14, 143–148.[find it @ Stanford]

    Carlier, I. V. E., Voerman, A. E., & Gersons, B. P. R. (2000). The influence of occupational debriefing on post-traumatic stress symptomatology in traumatized police officers. British Journal of Medical Psychology, 73, 87–98.[Medline][find it @ Stanford]

    Carver, C. S., Scheier, M. F., & Weintraub, J. K. (1989). Assessing coping strategies: A theoretically based approach. Journal of Personality and Social Psychology, 56, 267–283.[CrossRef][ISI][Medline][find it @ Stanford]

    Chemtob, C. M., Tomas, S., Law, W., & Cremniter, D. (1997). Postdisaster psychosocial intervention: A field study of the impact of debriefing on psychological distress. American Journal of Psychiatry, 154, 415–417.[Abstract]

    Conlon, L., Fahy, T. J., & Conroy, R. (1999). PTSD in ambulant RTA victims: A randomized controlled trial of debriefing. Journal of Psychosomatic Research, 46, 37–44.[CrossRef][ISI][Medline][find it @ Stanford]

    Deahl, M. P., Srinivasan, M., Jones, N., Thomas, J., Neblett, C., & Jolly, A. (2000). Preventing psychological trauma in soldiers: The role of operational stress training and psychological debriefing. British Journal of Medical Psychology, 73, 77–85.[CrossRef][ISI][Medline][find it @ Stanford]

    Dyregov, A. (1998). Psychological debriefing: An effective method? Traumatologye, IV:2, Article 1. Retrieved June 12, 2003, from http://www.fsu.edu/~trauma/art3v4i2.htm

    Eid, J., Johnsen, B. H., & Weisaeth, L. (2001). The effects of group psychological debriefing on acute stress reactions following a traffic accident: A quasi-experimental approach. International Journal of Emergency Mental Health, 3, 145–154.[Medline][find it @ Stanford]

    Everly, G. S. (1999). Emergency mental health: An overview. International Journal of Emergency Mental Health, 1, 3–7.[Medline][find it @ Stanford]

    Everly, G. S., Jr. (2003a) Pastoral crisis intervention in response to terrorism. International Journal of Emergency Mental Health, 5, 1–2.[Medline][find it @ Stanford]

    Everly, G. S., Jr. (2003b) Psychological counterterrorism. International Journal of Emergency Mental Health, 5, 57–59.[Medline][find it @ Stanford]

    Everly, G. S., Boyle, S. H., & Lating, J. M. (1999). The effectiveness of psychological debriefing with vicarious trauma: A meta-analysis. Stress Medicine, 15, 229–233.[CrossRef][find it @ Stanford]

    Everly, G. S., Flannery, R. B., & Mitchell, J. T. (2000). Critical Incident Stress Management (CISM): A review of the literature. Aggression and Violent Behavior, 5, 23–40.[find it @ Stanford]

    Everly, G. S., & Mitchell, J. T. (1999). Critical Incident Stress Management (CISM): A new era and standard of care in crisis intervention. Ellicott City, MD: Chevron Publishing.

    Everly, G. S., & Mitchell, J. T. (2000). The debriefing "controversy" and crisis intervention: A review of the lexical and substantiate issues. International Journal of Emergency Mental Health, 2, 211–225.[Medline][find it @ Stanford]

    Hibler, N. S., & Kurke, M. I. (1995). Ensuring personal reliability through selection and training. In M. I. Kurke & E. M. Scrivener (Eds.), Police psychology into the 21st century. Hillsdale, NJ: Erlbaum.

    Hobbs, M., Mayou, R., Harrison, B., & Worlock, P. (1996). A randomized control trial of psychological debriefing for victims of road traffic accidents. British Medical Journal, 131, 1438–1439.[find it @ Stanford]

    Jenkins, S. R. (1996). Social support and debriefing efficacy among emergency medical workers after a mass shooting incident. Journal of Social Behaviour and Personality, 11, 477–492.[find it @ Stanford]

    Kirschman, E. (1995). Organization consultation to law enforcement: An essay from the field. In M. I. Kurke & E. M. Scrivener (Eds.), Police psychology into the 21st century. Hillsdale, NJ: Erlbaum.

    Klinka, K. (1996, March 31). Many seek comfort at Project Heartland. Oklahoman. Retrieved June 12, 2003, from http://archives.oklahoman.com

    Klinka, K. (1999, April 11). Murrah bombing's effects long-term, study finds. Oklahoman. Retrieved June 12, 2003, from http://archives.oklahoman.com

    Kureczka, A. W. (1996). Critical incident stress in law enforcement. FBI Law Enforcement Bulletin, 65(213), 10–16.[find it @ Stanford]

    Lee, C., Slade, P., & Lygo, V. (1996). The influence of psychological debriefing on emotional adaptation in women following early miscarriage: A preliminary study. British Journal of Medical Psychology, 69, 47–58.[Medline][find it @ Stanford]

    Leonard, R., & Alison, L. (1999). Critical incident stress debriefing and its effects on coping strategies and anger in a sample of Australian police officers involved in shooting incidents. Work & Stress, 13, 144–161.[CrossRef][find it @ Stanford]

    Mayou, R. A., Ehlers, A., & Hobbs, M. (2000). Psychological debriefing for road traffic accident victims: Three-year follow-up of a randomized controlled trial. British Journal of Psychiatry, 176, 589–593.[Abstract/Free Full Text]

    Mitchell, J. T. (1983). When disaster strikes ... the critical incident stress debriefing process. Journal of Emergency Medical Services, 8(1), 36–39.[find it @ Stanford]

    Mitchell, J. T. (1991). Law enforcement applications of critical incident stress teams. In J. T. Reese, J. M. Horn, & C. Dunning (Eds.), Critical incidents in policing (pp. 201–211). Washington, DC: U.S. Department of Justice.

    Mitchell, J. T. (2003, February). Crisis intervention and CISM: A research summary. Retrieved May 26, 2005, from http://www.icisf.org/articles/cism_research_summary.pdf

    Mitchell, J. T., & Everly, G.S. (2001). The basic critical incident stress management course: Basic group crisis intervention (3rd ed.). Ellicott City, MD: International Critical Incident Stress Foundation.

    National Institute of Mental Health. (2002). Mental health and mass disasters: Evidenced-based early psychological intervention for victims/survivors of mass violence (NIH Publication No. 02-5138). Washington, DC: U.S. Government Printing Office.

    Nurmi, L. A. (1999). The sinking of the Estonia: The effects of critical incident stress debriefing on rescuers. International Journal of Emergency Mental Health, 1(1), 23–31.[Medline][find it @ Stanford]

    Ottley, T. (2003, July 16). Bad day dawning. In The Timothy McVeigh story: The Oklahoma bomber (chap. 1). Retrieved June 12, 2003, from http://www.crimelibrary.com/serial9/mcveigh/

    Owen, P. (1999, January 3). Recovery ongoing for Murrah rescuers. Oklahoman. Retrieved June 12, 2003, from http://archives.oklahoman.com

    Paoline, E. A., III. (2001). Rethinking police culture. New York: LFB Scholarly Publishing LLC.

    Plumberg, D. (2000, April 16). Rescuers live with memories—Many didn't seek help in dealing with stress. Oklahoman. Retrieved June 12, 2003, from http://archives.oklahoman.com

    Raymond, K. (1999, November 20). Officers' misdeeds blamed on stress. Oklahoman. Retrieved June 12, 2003, from http://archives.oklahoman.com

    Roberts, A. R., Everly, G. S., & Camasso, M. J. (2005, February). Meta-analysis of 36 crisis intervention studies: Plenary address. Paper presented at the 8th World Congress on Stress, Trauma, and Coping, Baltimore.

    Robinson, R. C., & Mitchell, J. T. (1993). Evaluation of psychological debriefings. Journal of Traumatic Stress, 6(3), 367–382.[CrossRef][find it @ Stanford]

    Rose, R., Bisson, J., & Wessely, S. (2002). Psychological debriefing for preventing post-traumatic stress disorder (PTSD). Cochrane Review. The Cochrane Library, 2002.

    Rose, S., Brewin, C. R., Andrews, B., & Kirk, M. (1999). A randomized controlled trial of individual psychological debriefing for victims of violent crime. Psychological Medicine, 29, 793–799.[CrossRef][ISI][Medline][find it @ Stanford]

    Shalev, A. Y., Peri, T., Rogel-Fuchs, Y., Ursano, R. J., & Marlowe, D. H. (1998). Historical group debriefing after combat exposure. Military Medicine, 163, 494–498.[Medline][find it @ Stanford]

    Sheehan, D. C., Everly, G. S., & Langlieb, A. (2004). Current best practices: coping with major critical incidents. FBI Law Enforcement Bulletin, 73(9), 1–13.[find it @ Stanford]

    Sheehan, D. C., & Van Hasselt, V. B. (2003). Identifying law enforcement stress reactions early. FBI Law Enforcement Bulletin, 72(9), 12–17.[find it @ Stanford]

    Smith, C. L., Jr., & de Chesnay, M. (1994). Critical incident stress debriefings for crisis management in post-traumatic stress disorders. Medicine and Law, 13, 185–191.[Medline][find it @ Stanford]

    Van Hasselt, V. B., Sheehan, D. C., Sellers, A. H., Baker, M. T., & Feiner, C. (2003). A behavioral-analytic model for assessing stress in police officers: Phase I. Development of the Law Enforcement Officer Stress Survey (LEOSS). International Journal of Emergency Mental Health, 5(2), 77–84.[Medline][find it @ Stanford]

    Volkman, P. (2001). Law enforcement perspectives for CISM enhancement. Ellicott City, MD: Chevron Publishing.

    Wagner, S. L. (2005). Emergency response service personnel and the critical incident stress debriefing debate. International Journal of Emergency Mental Health, 7(1), 33–41.[Medline][find it @ Stanford]

    Wessely, S., & Deahl, M. (2003). Psychological debriefing is a waste of time. British Journal of Psychiatry, 183, 12–14.[Free Full Text]

    Wessely, S., Rose, R., & Bisson, J. (2000). Brief psychological interventions ("debriefing") for trauma-related symptoms and the prevention of post traumatic stress disorder (Cochrane Review). In The Cochrane Library (Issue 3). Oxford, UK: Update Software.

    Young, A. T. (2003). An examination of the effectiveness of period stress debriefing with law enforcement personnel. Dissertation Abstracts International: Section B: The Sciences & Engineering, 64, 2902.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
Criminal Justice and BehaviorHome page
K. M. Newbold, J. M. Lohr, and R. Gist
Apprehended Without Warrant: Issues of Evidentiary Warrant for Critical Incident Services and Related Trauma Interventions in a Federal Law Enforcement Agency
Criminal Justice and Behavior, October 1, 2008; 35(10): 1337 - 1353.
[Abstract] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
5/3/261    most recent
mhi019v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Malcolm, A. S.
Right arrow Articles by Van Hasselt, V. B.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Malcolm, A. S.
Right arrow Articles by Van Hasselt, V. B.
Social Bookmarking
 Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?