HHS Public Access Author manuscript Author Manuscript Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Published in final edited form as: Issues Ment Health Nurs. 1994 ; 15(2): 149–168. EVALUATION OF A GERIATRIC MENTAL HEALTH TRAINING PROGRAM FOR NURSING PERSONNEL IN RURAL LONG-TERM CARE FACILITIES Marianne Smith, MS, RN, Abbe Center for Community Mental Health, Cedar Rapids, Iowa, USA Author Manuscript Kathleen C. Buckwalter, PhD, RN, FAAN, Abbe Center for Community Mental Health, Cedar Rapids, Iowa, USA Linda Garand, MS, RN, CS, Abbe Center for Community Mental Health, Cedar Rapids, Iowa, USA Susan Mitchell, MA, University of Iowa, Office of Consultation and Research in Medical Education, Iowa City, Iowa, USA Mark Albanese, PhD, University of Iowa, Office of Consultation and Research in Medical Education, Iowa City, Iowa, USA Author Manuscript Clare Kreiter, PhD University of Iowa, Office of Consultation and Research in Medical Education, Iowa City, Iowa, USA Abstract Author Manuscript Although mental illness among elderly living in nursing homes is a substantial and growing concern, the behavioral problems associated with mental illness or threats to mental health are not well understood, tolerated, or effectively managed by nursing home staff. As a result, resident care and quality of life, and staff morale often suffer. The need for geriatric mental health training in long-term care settings has become increasingly apparent. Psychiatric/mental health nurses are in an advantageous position to address this current need and future challenge through the development, implementation, and evaluation of geriatric mental health education and training programs in long-term care settings. This article describes one such innovative training effort, designed to improve the quality of psychosocial care provided by nursing personnel in rural longterm care settings, and highlights evaluation outcomes related to participant satisfaction, staff knowledge, and attitudes. The “graying of America” and the spiraling costs of long-term care make it essential that health care providers, administrators, and policy makers understand the characteristics of Address correspondence to Kathleen Buckwalter, PhD, RN, FAAN, T-136, CNO University of Iowa Hospital and Clinics, Iowa City, IA 52242. Smith et al. Page 2 Author Manuscript nursing home residents, especially with regard to their mental health status and management of their emotional problems. Although at present only about 5% of persons over the age of 65 reside in institutions, the prevalence of untreated mental illness and behavioral problems among these residents is both a current and growing concern. This concern has led gelopsychiatric experts to label nursing homes the modern “psychiatric ghettos” (Liptzin, 1986), and a substitute for public mental hospitals for the chronically mentally ill (Aiken, 1987). Author Manuscript Psychiatric/mental health nurses are in an advantageous position to address these issues through the development and provision of education and training programs, as well as consultation services to colleagues in long-term care. This article describes one such innovative training effort, designed to improve the quality of psychosocial care provided by nursing personnel practicing in rural long-term care settings, as well as outcomes related to participant satisfaction, staff knowledge, and attitudes. Its purpose is to acquaint mental health nurses with our training approach and to encourage them to undertake similar training efforts in long-term care settings. BACKGROUND AND SIGNIFICANCE: THE SOCIODEMOGRAPHIC IMPERATIVE Author Manuscript As geropsychiatric nurses are well aware, the rapidly growing segment of elderly in our country, particularly among the “oldest-old,” suggests that geriatric mental health issues and problems will only become more intense in the next decade and beyond. Current projections for the year 2000 are that there will be 2.1 million persons living in nursing homes (Bureau of the Census, 1989), and that more than 40% of those who turned 65 in 1990 will enter a nursing home in their lifetime (Kemper, Spillman, & Murtaugh, 1991). To effectively manage the needs of this growing population of long-term care residents, staff must be prepared to deal with both physical and psychosocial issues, including the provision of geropsychiatric care. Prevalence of Mental Health and Behavioral Problems in Nursing Homes Author Manuscript Although the prevalence of mental disorders and behavioral disturbances in nursing home residents is consistently reported to be quite high (Ouslander, 1989; Rovner, Kafonek, Filip, Lucas, & Folstein, 1986); most clinicians and researchers agree that mental illness is often under-diagnosed and frequently misdiagnosed (German, Rovner, Burton, Brant, & Clark, 1992). Estimates that 40 to 80% of nursing home residents suffer from some type of dementia can be found in the literature (Finkel & Denson, 1990). However, data from the 1987 National Medical Expenditure Survey (NMES) suggests that 20% of residents surveyed had dementia only and 14% had dementia in combination with at least one other mental disorder. One explanation for the discrepancy in these prevalence rates is set forth by Rovner and Rabins (1985) who suggest that a substantial proportion of elders with cognitive impairment (that is attributed to dementia) may actually have a reversible disorder that mimics dementia, such as delirium, major depression, physical illness, or a medication side effect. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 3 Author Manuscript Depression, which is highly treatable, is the second most common mental illness found in long-term care settings (Rabins, Storer, & Lawrence, 1992), affecting as many as 20 to 30% of all residents. Rovner et al. (1991) found that 12% of the nursing home residents they examined had a major depressive disorder, and another 18% experienced depressive symptoms. Importantly, a majority of these cases were unrecognized and untreated by the attending nursing home physician. Like dementia, which precipitates losses in multiple domains (i.e., affective, conative, cognitive, and the ability to tolerate stress), depression can also cause a number of management problems for staff, including apathy, withdrawal, lack of self-care, and suicidal behavior. Author Manuscript Author Manuscript Behavioral problems caused by various types of mental illness are also common in longterm care facilities (LTCFs). As Levenson (1987) noted that a growing number of nursing home residents have sufficient cognitive, emotional, or behavioral dysfunctions (e.g., agitation, aggression, wandering, depression, withdrawal) to cause some management problems. Burgio, Butler, & Engel (1988) found that the problems reported by staff were attributable to anxiety (29.8%), affective disorders (28.3%), personality and character disorders (23.0%), schizophrenia (7.5%), alcohol/drug abuse (6.5%), and other psychoses (5.0%). Interestingly, data from the NMES (1987) suggest that although patterns of mental disorders differed little between the sexes, 52% of male residents experienced behavioral problems compared with 46% of female residents; and significantly more men were likely to hurt others (16% vs 9% for women). According to this survey, just under half of the nursing home residents surveyed exhibited behavioral problems, the most prevalent of which was getting upset or yelling (31%), followed by wandering (11%) and physically hurting others (11%). Similarly, Rovner and Rabins (1985) found that irritability, yelling, explosiveness, and resistance (all of which can lead to staff avoidance and abuse) were the most commonly noted behavior problems in their study. Moreover, in addition to those already suffering from mental health problems, a significant percentage of unaffected residents may be at risk for developing behavioral problems or mental illness because of high physical comorbidity and the nonsupportive emotional milieu of many LTCFs (Smith, Buckwalter, & Albanese, 1990). The prevalence of mental illness in LTCFs is further substantiated by the fact that 29% of the total national expenditure for care of the mentally ill goes to nursing homes—more than state, county, and other public mental health hospitals combined (Frazier, 1986). Author Manuscript In summary, mental illness is a substantial problem among elderly living in nursing homes. Yet in far too many cases, mental problems are simply “written off” as a consequence of the aging process, a “normal” and unavoidable part of growing older (Harper, 1986). This misconception is reinforced by outdated reimbursement policies that direct attention away from psychosocial care (Still, 1985). Lack of Staff Knowledge and Adequately Trained Professionals The reversal of treatable changes in mental status, as well as the management of behavioral problems resulting from mental illness, require that nursing home personnel be skilled in psychiatric assessment and screening. Compelling evidence supports our observations that most staff in LTCFs are not knowledgeable about the aging process, and are especially illIssues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 4 Author Manuscript prepared to deal with the psychological symptoms and behavioral problems experienced by a majority of residents (Meunier & Holmes, 1987). Staff seldom seek appropriate mental health interuention once a problem is recognized (Caston, 1983). Guy & Morice (1985– 1986) found that less than 20% of nursing home staff had any formal training in the behavior management strategies that are known to be useful with this population. Research conducted by Baltes et al. (Baltes, Burgess, & Steward, 1980; Baltes, Honn, Barton, Orzech, & Lago, 1983; Barton, Baltes, & Orzech, 1980) indicates that staff-resident interactions may actually encourage dependent, passive behaviors. Author Manuscript These problems can be attributed to three main factors (Birren & Sloane, 1983): (a) an insufficient number of practitioners available in long-term care who are specifically trained to address the mental health needs of the elderly; (b) an insufficient supply of researchers with interest and expertise in issues related to mental health and aging; and (c) inadequate numbers of geriatric mental health teachers and trainers needed to train the current hands-on personnel. Until this situation is rectified and there are sufficient numbers of well-trained staff working in nursing homes, we cannot expect the long-term care service system to improve. Moreover, the tremendous deficit in geriatric mental health knowledge among nursing home personnel suggests that without increased psychiatric nursing consultation and education, staff will continue to be unprepared to deal with the complex emotional and behavioral problems exhibited by residents, perpetuating the overuse of physical and chemical restraints. Author Manuscript Nationally, as much as 80 to 90% of the care provided in nursing homes is given by nursing assistants, in many ways the least well-prepared provider (Reagan, 1986). Until recently, training of nursing home staff focused almost exclusively on the physical care of residents, or compliance with state and federal regulations. Although these aspects are important, the constantly changing psychosocial needs of residents are equally critical (Coons, 1991). Information on mental and social health service needs is now mandated as part of nurse aide training curricula; however, the amount of time devoted to these topics is still inadequate. Thus, lack of quality care in nursing homes is often attributed to the undertrained and devalued health care worker. Author Manuscript A number of inservice education efforts and training programs for nursing assistants in longterm care settings have been described in the literature (Almquist & Bates, 1980; Hameister, 1977, 1978). Very few, including earlier training efforts by the authors (Smith, Buckwalter, & Albanese, 1990) have rigorously evaluated the effects of the training on staff or residents. A notable recent exception is the Penn State Nursing Home Intervention Project (Smyer, Brannon, & Cohn, 1992), which was designed to affect nursing assistants’ performance by increasing their knowledge and motivation through skills training and job redesign. The training project described and evaluated in this article differs from previous training efforts in several ways. These include content exclusively focused on geriatric mental health problems common in nursing homes, program organization of each educational module, use of a “train-the trainers” model and two-way interactive telecommunications techniques, and a focus on staff in rural LTCFs. The training staff believed that the provision of training was one important form of respect and recognition for the nursing assistant, and that training, in Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 5 Author Manuscript combination with attitudes of appreciation and caring from others in the facility, has the potential to reduce staff stress by increasing their sense of competence, morale, and selfworth. Each educational module is organized in the same way. The first three sections of the module, which are standard in each program, provide practical information to assist the instructor to use the program materials. The first three sections are: Introduction to the Training Materials. reviews the various components of the module to encourage optimal and individualized use by the instructor. Getting Started. Suggestions on organizing the educational program are offered, including the composition and size of the audience, the physical environment of the program, and incentives to encourage implementation of the program concepts. Author Manuscript Presentation Skills. Communication strategies, use of audiovisual aids, speaking skills, and methods to lead and manage the discussion are offered for the nurse who is not an experience instructor. The rest of the module includes information on the specific mental health or illness topic, including the following: Statement of Purpose and Objectives. Facilitate the provision of continuing education units. Notes to the Instructor. Provides background information on the topic, including the rationale for/importance of teaching the topic to long-term care staff, and methods to personalize the content to the facility. Author Manuscript Checklist. A quick reference for all materials needed to teach the program. Lecturers Script. A word-for-word manuscript from which the nurse can teach, including notes on when to change slides, refer the audience to handouts, as well as discussion questions, illustrations, and other methods to personalize the topic to the facility. Slides. A set of 35-mm slides complement the lecturer’s script and other materials. Handouts. Summarize the important content for the participant. Exercises and Activities. Encourage understanding and application of the concepts taught in the module. Bibliography. References that support or expand on the concepts taught in the Author Manuscript module. Glossary of Terms. A quick reference outlining psychiatric terms, abbreviations, or medical jargon in the lecturer’s script that may need to be explained or simplified. In summary, the need for geriatric mental health training in long-term care settings has become increasingly apparent. Although an increasing number of residents are suffering from some type of mental disorder, the behavioral problems associated with mental illness Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 6 Author Manuscript or threats to mental health are not well understood, tolerated, or effectively managed by staff in nursing homes. As a result, both resident care and staff morale often suffer. PROJECT OVERVIEW Author Manuscript In response to the issues discussed above, geropsychiatric nurses at the Abbe Center for Community Mental Health developed a Geriatric Mental Health Care Training Project, funded by the Division of Nursing. The overall goal of the project was to increase the ability of nurses and nursing personnel in rural long-term care facilities (LTCFs) to provide quality care to their residents with psychiatric and behavioral problems, through the development and provision of six free-standing “core” educational training programs or “modules.” All of the training modules are on psychiatric and mental health topics, and are designed to help staff be more knowledgeable about the causes of problem behaviors (i.e., depression, dementia, or long-standing personality traits), techniques to manage those troubling behaviors (i.e., using the Antecedent, Behavior, Consequence, or “ABC” model of behavior modification and rules of reinforcement), and the influence of the care providers’ own feelings and behaviors in response to the problem behaviors (e.g., the role of values and beliefs, personal needs, gaining cognitive control of the situation, and stress in the workplace). The programs are targeted for use by professional and nonprofessional staff who are not specialists in psychiatric mental health care, but who provide direct patient care in nursing homes. Thus the modules are written in simple easy-to-understand language that eliminates nearly all medical or psychiatric jargon, as described in more detail below. Train-the-Trainers Model Author Manuscript The 3-year, three-phase training project used a train-the-trainers model first locally (Phase I), then regionally (Phase II), and finally statewide (Phase III) in Iowa. In an effort to reach nurses and nursing personnel in increasingly remote LTCFs, a two-way interactive telecommunication strategy was implemented and evaluated in Phases II and III. In this train-the-trainers model, geropsychiatric nurse specialists from the Abbe Center trained long-term care nurses during 2-day Intensive Training Sessions (ITSs). These nurse trainers then used the training materials (a six-part training manual and slide sets) to train staff in their own facilities. This model emphasized the importance of developing the competence of selected long-term care nurses who not only trained additional staff, but who became resource persons, served as liaisons with the geropsychiatric nurse specialists at the Abbe Center, and became leaders in the application of geriatric mental health care principles to residents in their own setting. The training staff also provided on-site technical assistance and case consultation to participant LTCFs. Author Manuscript Overview of Program Organization Each module is constructed in the same way, emphasizing a problem-solving approach to behavior management rather than personalizing the problem. The first part of each module describes the type of problem behavior seen by staff (e.g., verbal abuse). The underlying illness or emotional conflict is then outlined, emphasizing the resident’s experience of distress. Contributing and/or intervening variables are included as part of the discussion of methods to assess the resident. Finally, nursing interventions are offered to help staff manage Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 7 Author Manuscript the problem behavior, either by implementing behavior management or modification techniques or by changing their reaction to the resident. Advantages of this program design included good audience rapport, personalization of the content, ease of use, flexibility, and replicability. Project Objectives Author Manuscript All three phases of the training project shared the same objectives: (a) to train nurses (RNs) and directors of nursing (DONs) in LTCFs in Linn County (Phase I), the eight counties comprising the Heritage Area Agency on Aging catchment area (Phase 2), and statewide (Phase 3) in geriatric mental health issues and training techniques; (b) to have those RNs and DONs together training at least 10 direct care providers (RNs, LPNs, and nursing assistants) in their respective facilities in geriatric mental health issues related to long-term care; (c) to provide on-site technical assistance and case consultation to participant LTCFs; (d) to provide a forum for discussion of training issues and to disseminate new information in the field of geriatric mental health; and (e) to evaluate the effectiveness of the geriatric mental health training program in terms of number of people trained, programmatic impact, and attitude and knowledge change in trainees. The project was a collaborative effort between the Abbe Center for Community Mental Health, Kirkwood Community College, and the University of Iowa’s Office of Consultation and Research in Medical Education, which conducted the independent evaluation. Design of the Evaluation Three basic questions were addressed in the evaluation: Author Manuscript 1. To what extent did the train-the-trainer model provide training opportunities to the staff of the LTCFs? 2. How effective were the programs in terms of participant satisfaction, learning, and ability to apply the material to patient care situations? 3. How valuable was the consultation program? Author Manuscript The evaluation would determine the effectiveness of the program in changing staff’s knowledge, attitudes, and behaviors. The model used for this evaluation was based on Dixon’s (1978) hierarchy of outcome variables. Three of the four levels in the hierarchy were assessed: first level indicators reflecting participant perceptions, second level indicators reflecting knowledge and attitude changes, and third level indicators reflecting changes in participant behavior. The fourth level, assessing changes in the health of clients, was beyond the scope of this evaluation. Because of space limitations, this article presents data from only Phase II of the project, and focuses on outcomes related to evaluation of the effectiveness of the training program in terms of participant satisfaction and changes in knowledge and attitude of the trainers and trainees. Other manuscripts, currently under review, describe outcomes related to behavioral changes in the staff, perceived barriers to training. methodological issues in evaluation research, use of geropsychiatric consultation services, and changes in staff self-esteem, feelings of competence, and empowerment. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 8 Number of Staff Trained Author Manuscript Phase I served as an opportunity to pilot test the content of the training modules and the evaluation design with instruments under operational conditions. In Phase I, 33 staff members from 16 LTCFs in Linn County participated in a 2-day intensive training session using a train-the-trainers model. Subsequently, 24 nurse trainers taught 313 staff members in 15 facilities on geriatric mental health issues related to long-term care. Author Manuscript In Phase II, 43 RNs and DONs representing 22LTCFs in eight different counties were trained in an interactive teleconference. The 2-day ITS originated from Kirkwood Community College and was broadcast to counties that comprised the Area Agency on Aging catchment area. The six programs taught at the Phase I ITS were revised based on Phase I evaluations and taught again at this second ITS. Of the 22 facilities, 21 completed the six-part training series; 177 staff members attended all six training programs, and a total of 520 staff attended one or more training programs during Phase II. On-site technical assistance and consultation was again provided in Phase II. The data presented below are primarily from this phase. The largest number of trainers and trainees (43 and 520, respectively) were analyzed during this phase, and findings are consistent with those in the other phases. Author Manuscript Phase III reached a much larger population than either Phase I or Phase II. Instruction was again presented via an interactive teleconference, this time broadcast across the state to 30 counties in Iowa. Content presented at the ITS and training materials sent to the facilities were essentially the same as those utilized in Phase II. The evaluation procedures were performed for 12 of the 62 facilities randomly selected from the group of participating LTCFs, and the evaluation of these centers proceeded as in Phase II. A total of 124 trainers participated in the Phase III ITS, and they in turn trained another 800 staff members in their own facilities. PROJECT EVALUATION: PHASE II Evaluation of Participant Satisfaction Author Manuscript Forty-three staff members (trainers) from 22LTCFs were trained at six sites by interactive telecommunication in a 2-day ITS on March 20 and 21, 1991. These trainers completed a post-ITS evaluation form that assessed five factors (overall quality of the program, relevance, content, organization, and presentation), using a Likert-type format ranging from 1 = poor to 6 = excellent. Mean ratings of the ITS participants were quite high and ranged from 5.3 (organization and presentation) to 5.5 (for relevance). Overall quality of the training session and content received a mean rating of 5.4. Participants in the Phase II ITS generally had a very favorable response to the televised training sessions. Responses were quite similar to those of the Phase I respondents who received training directly from the ITS instructors. Trainer evaluations of the ITS were consistently high, ranging between 5.3 and 5.5. The participants’ free responses also demonstrated a general high regard for the quality of the presentation, content, and training staff. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 9 Author Manuscript Author Manuscript Program evaluation forms were completed by trainers after each program presented at the LTCFs. The primary goal of this tool was to determine if the program materials were constructed in a way that made them most useful to the trainers. Participants were asked to rate the program’s relevance, statement of purpose, objectives, outline, and notes to the instructor on a 6-point Liker-type scale ranging from 1 = poor to 6 = excellent. In addition, trainers were asked to rate the vocabulary in the program, the examples and illustrations used, the complexity of the materials (i.e., too simple, complex, just right), and the amount of information in the handouts for their setting. A series of open-ended questions asked trainers for any difficulties or comments they had about the slides, handouts, and exercises, and for suggestions on how the program could be improved. As in Phase I, average ratings for the Likert portion of the form were computed and reviewed by the training coordinator and instructional designer. A summary of trainer evaluations revealed that the percentages of trainers who selected a rating of either 5 or 6 on a 6-point scale ranged from 70% on “objectives” in program 3 (“Help, Hope, and Power”) to 88% (modal response) for all of the components of program 6 (“Assessment and Management of Aggressive and Acting Out Behaviors”). In general, as in Phase I, trainers found the program components highly useful for each of the six programs. Trainee forms were also compiled into one summary for each program and average ratings for the Likert portion of the form were computed. Table 1 displays the results of the trainee evaluation. Author Manuscript Problems encountered by trainers and trainees were noted and appropriate steps were taken to either resolve the problem immediately or make changes to program materials or training for Phase IIL In most cases, nurse trainers thought the program materials were relevant and constructed in an easy-to-use format, and trainees regarded the overall quality of the program, its relevance, and usefulness quite highly. Evaluation of Knowledge and Attitudes Author Manuscript The instruments designed by the investigators to measure knowledge and attitude changes consisted of a test and questionnaire that were administered before and after training. The 80-item knowledge test contained both multiple-choice and true/false items. The attitude questionnaire consisted of 16 items using a Likert-type scale. Prior to the training effort, both instruments v/ere reviewed by a statewide advisory council, which consisted of gerontologic, psychiatric, and long-term care experts, to establish content validity. In addition, a panel of geropsychiatric nursing experts at the University of Iowa College of Nursing reviewed every knowledge item to assure that they were derived from content presented in the training program. Phase I reliability testing of the knowledge instrument using Cronbach’s α yielded coefficients ranging from .71–.75 for trainers and .79–.86 for trainees. Reliability coefficients for the attitude measure were .72 for the pretest and .69 for the posttest. Pretests were administered in the care facilities to both trainers and trainees 2–3 weeks before the beginning of the ITS. Trainers were administered an identical posttest immediately after the ITS. Posttesting for the trainees was conducted at the facilities within 3 weeks of the presentation of the last training module. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 10 Knowledge Test Outcomes Author Manuscript Results of the knowledge test for Phase II trainers and trainees are noted in Table 2. Differences between pre- and posttest scores were calculated using dependent (paired) t-tests for both trainers and trainees. All trainee pre- and posttests could not be matched for a number of reasons. In some cases, the trainees neglected to complete the birthdate information on the front of the test used to match the examinee’s pre-and posttest. In other instances, trainees did not have complete sets of pre- and posttests due to either starting the training series late or not completing the series. Data for all trainees who took either the preor posttest are presented in Table 2, as is data for those trainees who took both the pre- and posttests—the “matched” trainees (N = 153). Author Manuscript For all three groups there were significant differences between pre-and posttest performance, with the posttest performance being significantly better in each case. The effect size for the trainers (.64) is between a medium and large difference according to the criteria set by Cohen (1969). The magnitude of the difference between the pre- and posttest means for the unmatched trainees (.45) is considered an approximately medium effect size while the matched trainees’ effect size of .37 is viewed as a small or medium effect according to the same criteria. Author Manuscript Performance differences on the knowledge test among staff at various employment levels were expected. Specifically, it was predicted that individuals with staff positions requiring higher levels of education (e.g., RN, LPN) would outperform those with positions requiring less education (nurse’s aide). The nursing degrees and certifications held by trainers and trainees were as follows: 6% nursing assistants, 26% certified nursing assistants, 6% certified medication aides, 22% LPNs, 22% RNs, and 18% other professionals (e.g., social workers). The job categories were consolidated into two groups: nursing aides (including certified nursing assistants and medication aides) and nurses (LPNs/RNs). The remaining 18% (other professionals) were omitted for this analysis. Most of the trainees, were women, precluding valid gender comparisons. Table 3 displays the statistical comparison of two staff position categories with different educational requirements. Author Manuscript For both groups and on both the pre- and the posttest, nurses outperformed nursing aides. Mean scores for nurses were 63.73 pretest and 64.10 posttest; for nursing assistants, 55.76 pretest and 59.11 posttest. A statistically insignificant difference (p < .001), using independent (non-paired) t-tests, was found. In every case, the mean performance for nurses was over a full standard deviation higher than that for nursing aides. With effect sizes of 1.378 on the pretest and 1.037 on the posttest data, staff position could be characterized as producing a large effect by Cohen’s (1969) criteria. An analysis of variance was also performed to simultaneously examine the effects of age and staff position, (i.e., nurses and nursing aides) on test performance. For purposes of this analysis, the classification variable age groups consisted of two levels depending on whether the chronologic age was above or below the mean age for the entire group. The two-way analysis of variance showed no significant age effect in total score and no interaction effects with staff position. Age effects were not further examined. A two way analysis of variance was undertaken to examine the effects of pre/post status and the two Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 11 Author Manuscript categories of staff position on total knowledge test score. No interaction effects between pre/ post status and staff position were indicated. In summary, both pre/post status and staff position proved to be significant in terms of knowledge scores, whereas age did not. Author Manuscript The knowledge test was designed to be of an appropriate difficulty level for both the trainers and the trainees. Since trainers were generally more highly qualified than other staff members from the facility, it was expected that they would find the test somewhat easier than trainees. A statistically significant difference in the expected direction was found between trainer and trainee means on the pretest, with (df=250) =2.422, p < .01. Trainers answered 76% of the questions correctly on the pretest while trainees responded correctly to 72% of the questions. Because the trainers and trainees performed well on the pretest, the amount of improvement that could be reflected by posttest performance was necessarily small. Because the test was somewhat easier than would be ideal (Ebel & Frisbie, 1986) for this population, the potential to detect differences (using dependent t-tests) between the preand the posttraining knowledge scores for trainees was probably somewhat compromised. Nevertheless, differences between the two means were significant (p < .001). Attitude Questionnaire Outcomes Author Manuscript The attitude questionnaire consisted of 15 statements with which staff were to designate agreement or disagreement. Examples of items on the attitude survey included: “Most older people are unable to learn anything new;” As a group the elderly are quite different, but they are stereotyped by society; “Most elderly have more limitations than abilities;” “You can’t teach an old dog new tricks;” and “Aging tends to bring out the worst in people’s personalities.” The items were scored first by combining the original 5-point scale into three categories (agree, disagree, and undecided), and then comparing staff’s responses to the desired response as determined by the Project Director (KB) and Training Coordinator (MS). Trainer and trainee responses in the desired direction on the pre- and posttest are combined in Table 4. Author Manuscript Analysis of attitudes was conducted by computing the percentage of individuals who chose a response in the desired direction on each of the 15 statements listed in Table 4. Both trainers and trainees had relatively positive attitudes toward elderly in the pretest condition. For trainees, the pre-to posttest changes were overwhelmingly in the desired direction with 11 of 15 items changing in the positive direction. Trainers showed a less dramatic pattern, with increases on only five items and slight decrements on the remaining items. Matched trainees showed the most positive changes in attitudes on 13 out of 15 items. The discrepancy between the trainers and trainees may be the result of trainers having substantially more positive attitudes to begin with. For them, the programs may have served as more of a reality check on their degree of enthusiasm. Attitudinal barriers often interfere with effective care of geropsychiatric residents in nursing homes. The clinical observations of the Abbe nurses were consistent with those described in the literature. Prior to the training program many of the problem behaviors that erupted from mental illness or threats to the person’s mental health (e.g., their sense of competency, control, independence, and autonomy) were wrongly attributed to the resident being “manipulative” or “difficult” rather than impaired or threatened. Staff attitudes about the Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 12 Author Manuscript older resident frequently contributed to the problem, either by avoidance, by allowing the resident to escalate to the level of aggression, or by getting into power struggles in which no one could win. In summary, Phase II evaluated revised training content based on Phase I evaluations, using a two-way interactive telecommunications approach that enabled the training staff to broadcast to sites in eight counties simultaneously. Participant responses to the training were highly favorable. Not unexpectedly, nurses outperformed nonprofessional staff on tests of knowledge with increases in knowledge scores. Positive attitudes were noted from pre- to posttests for all trainees. IMPLICATIONS FOR PSYCHIATRIC MENTAL HEALTH NURSES Author Manuscript Nursing homes have replaced state mental hospitals as the primary site for residential care of mentally ill elderly. Moreover, the number of nursing home residents with mental illness and behavioral problems is expected to increase dramatically in the next few decades. Despite the prevalence of psychiatric problems in long-term care settings, they are frequently overlooked, undiagnosed or misdiagnosed, or labeled “incurable.” Although the need for treatment is clear, and empirical evidence of effective psychosocial interventions for the elderly exists, mental health professionals in general have very little contact with this population (Burns, et al., 1993). Thus, staff education and consultation are critical to the success of any mental health service plan for nursing homes (Borson, Liptzin, Nininger, & Rabins, 1987). Educating long-term care personnel about geriatric mental health issues emerges as a timely and important role for psychiatric/mental health nurses. Author Manuscript Author Manuscript Potential outcomes of providing training for long-term care personnel in the management of behavioral problems include better quality patient care, higher levels of staff satisfaction with care, and diminished attrition rates (Payne & Lyons, 1987). However, effective training relies not only on meaningful program content, but also on strong support and leadership from administration and supervisory staff, and effective strategies for teaching the content, In many cases the way in which things were done was as important as what was done. Based on results from the first 3 years of this training project, nurse trainers (who come to view themselves as role models and mentors to the nursing assistants) will train small groups of staff over a short period of time, and they will begin by training key staff and the natural leaders in the facility first. Compliance with the behavioral management and modification techniques (e.g., how to handle aggressive and acting out behaviors in both cognitively impaired and cognitively intact residents) can also be improved by helping staff feel that they are privileged recipients of the training material, which supports their self worth, and by continually reinforcing concepts and strategies taught in the program in the practice arena. Rewarding staff for attendance at training sessions (e.g., certificates, letters in personnel file, buttons, write-ups in newsletters, special parking privileges, etc.) improves their sense of achievement, acknowledges them and the job they do, and bolsters their self-esteem. Importantly, all staff are involved in the implementation phase and their honest feedback regarding potential barriers is actively solicited. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 13 Author Manuscript CONCLUSION Mental illness is a growing concern among elderly living in nursing homes. The behavioral problems associated with mental illness are often not effectively managed by long-term care personnel. Psychiatric nurses can play a vital role in articulating mental health concepts and teaching behavioral management strategies essential to therapeutic success in nursing homes. Through the development, implementation, and evaluation of training programs such as the one described in this article, psychiatric/mental health nurses can have a positive impact on both the knowledge and attitudes of long-term care staff. Acknowledgments Supported by Grant #D10NU27118–01, Division of Nursing, Bureau of Health Professions. Author Manuscript REFERENCES Author Manuscript Author Manuscript Aiken LH (1987). Unmet needs of the chronically mentally ill, will nursing respond? Image, 19(3), pp. 121, 125. Almquist E, & Bates D (1980). Training program for nursing assistants and LPNs in nursing homes. Journal of Gerontological Nursing, 6(1), 622–627. [PubMed: 6904449] Baltes M, Burgess R, & Steward R (1980). Independence and dependence in self-care behaviors in nursing home residents: An operant-observational study. International, Journal of Behavioral Development, 33, 489–500. Baltes M, Honn S, Barton E, Orzech M, & Lago D (1983). On the social ecology of dependence and independence in elderly nursing home residents: A replication and extension. Journal of Gerontology, 38, 2556–2564. Barton E, Baltes M, & Orzech M (1980). Etiology of dependence in older nursing home residents during morning care: The role of staff behavior. The Journal of Personality and Social Psychology, 37, 423–431. Birren JE, & Sloane RB (1983). Manpower training needs in mental health and illness of the aging Background issue paper prepared for the Committee on Mental Health and Illness of the Elderly. Borson S, Liptzin B, Nininger J, & Rabins P (1987). Psychiatry and the nursing home. The American Journal of Psychiatry, 144(11), 1412–1419. [PubMed: 3314537] Burns BJ, Wagner HR, Taube JE, Magaziner J, Permutt T, & Landerman LR (1993). Mental health service use by the elderly in nursing homes. American Journal of Public Health, 83(3),331–337. [PubMed: 8438968] Bureau of the Census. (1989). Projections of the population of the U.S., by age, sex, and race: 1988– 2080 Current population reports. (Series P-25, No. 1018). Washington, DC: U.S. Government Printing Office. Burgio LD, Butler F, & Engel BT (1988). Nurses attitudes toward geriatric behavior problems in long term care settings. Clinical Gerontologist, 7(3/4),23–24. Caston R (1983). The role of education and health care delivery structure in quality nursing care for the mentally ill patients in nursing homes. Educational Gerontology, 9,425–433. Cohen J (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press. Coons D (1991). Training direct staff members to work in dementia care units In Coons D (Ed.). Specialized dementia care units (pp. 126–149. Baltimore: The Johns Hopkins University Press. Department of Health and Human Services. National Center for Health Services Research and Health Care Technology Assessment and Health Care Financing Administration (1987). National Medical Expenditure Survey: Institutional Population Component. Washington, DC: Government Printing Office. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 14 Author Manuscript Author Manuscript Author Manuscript Author Manuscript Dixon J (1978). Evaluation criteria in studies of continuing education in the health professions: A critical review and a suggested strategy. Evaluation and the Health Professions, 1(2), 47–65. [PubMed: 10308062] Ebel RL, & Frisbie DA (1986). Essentials of education measurement (4th ed.). Prentice-Hall, Inc. Finkel S, & Denson M (1990). Psychopathology in later life. Comprehensive Therapy, 16(9), 17–24. Frazier S (1986). Foreword. In Harper MS, & Lebowitz B (Eds.), Mental illness in nursing homes: An agenda for research DHHS Publication No. (ADM) 86–1459 (pp. iii-iv). National Institute of Mental Health, Washington, DC: U.S. Government Printing Office. German P, Rovner B, Burton L, Brant L, & Clark R (1992). The role of mental morbidity in the nursing home experience. The Gerontologist, 32(2), 152–158. [PubMed: 1577308] Hameister DR (1977). A design of inservice education for nurses’ aides in a nursing home setting. The Journal of Continuing Education in Nursing, 8(2),6–12. Hameister DR (1978). Examining relationships between educational institutions and in-service training for nurses’ aides Long-Term Care and Health Care Services Administration, Winter, 6–7. Guy D, & Morice A (1985–1986). A comparative analysis of behavior management in the nursing home. Clinical Gerontologist,4, 11–17. Harper M (1986). Introduction. In Harper MS & Lebowitz B (Eds.), Mental illness in nursing homes; An agenda for research DHHS Publication No. (ADM) 86–1459 (pp. 1–7). National Institute of Mental Health, Washington, DC: U.S. Government Printing Office. Kemper P, Spillman B, & Murtaugh C (1991). A lifetime perspective on proposals for financing nursing home care. Inquiry, 28(4), 333–344. [PubMed: 1761306] Levenson SA (1987). Innovations in nursing home care Generations, pp.74–79. Liptzin B (1986). Major mental disorders/problems in nursing homes: Implications for research and public policy In Harper M & Lebowitz B (Eds.). Mental illness in nursing homes: An agenda for research. DHHS Publication No. (ADM) 86–1459 (pp. 41–57). National Institute of Mental Health, Washington, DC: U.S. Government Printing Office. Meunier GF, & Holmes TR (1987). Measuring the behavioral knowledge of nursing home employees. Clinical Gerontologist, 6(3), 11–22. Ouslander J (1989). Medical care in the nursing home. Journal of the American Medical Association, 262, 2582–2590. [PubMed: 2572706] Payne MB, & Lyons JD (1987). Preparation of geriatric aides for patient care and certification Geriatric Nursing, May-Jun, 125–127. Rabins P, Storer D, & Lawrence M (1992). Psychiatric consultation to a continuing care retirement community. The Gerontologist, 32(1), 126–128. [PubMed: 1740248] Reagan J (1986). Management of nurses aides in long-term care settings. Journal of Long-Term Care Administration, 14(2), 9–14. Rovner BW, & Rabins PV (1985). Mental illness among nursing home patients. Hospital and Community Psychiatry, 36(2), 119–128. [PubMed: 3972338] Rovner B, Kafonck S, Filip L, Lucas M, & Folstein M (1986). Prevalence of mental illness in a community nursing home. American Journal of Psychiatry, 143, 1446–1449. [PubMed: 3777239] Rovner B, German P, Brant L, Clark R, Burton L, & Folstein M (1991). Depression and mortality in nursing homes. Journal of the American Medical Association, 265(8),993–996. [PubMed: 1992213] Smith M, Buckwalter KC, & Albanese M (1990). Geropsychiatric education programs: Providing skills and understanding. .Journal of Psychosocial Nursing, and Mental Health Services, 28(12), 8–12. [PubMed: 2126559] Smyer M, Brannon D, & Cohn M (1992). Improving nursing home care through training and job redesign. Gerontologist, 32(3),327–333. [PubMed: 1499997] Still C (1986). Major mental health problems in the nursing home: A medical director’s perspective In Harper MS, & Lebowitz B (Eds.), Mental illness in nursing homes: An agenda for research. DHHS Publication No. (ADM) 86–1459 (pp. 57–65). National Institute of Mental Health, Washington, DC: U.S. Government Printing Office. Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 15 Table 1. Author Manuscript Means and Standard Deviations of Trainee Evaluations across Centers for Phase II Program* Component Program, Program A Scale** 1 Overall quality of the program 4.88 (.52) B C D E 6 Poor * F Excellent 4.80 (.32) 4.73 (.51) 4.99 (.46) 4.97 (.55) 4.98 (.39) Relevance of topic 5.01 (.42) 4.97 (.39) 4.89 (.41) 5.18 (.39) 5.13 (.43) 5.14 (.27) Usefulness of program content 5.01 (.46) 4.92 (.41) 4.81 (.50) 5.09 (.48) 5.10 (.43) 5.12 (.33) Program organization 5.00 (.49) 4.90) (.42) 4.80 (.46) 5.08 (.49) 5.01 (.50) 5.01 (.34) Overall presentation quality 5.04 (.48) 4.93 (.39) 4.87 (.44) 5.05 (.50) 5.01 (.52) 5.09 (.31) A = Mental health and illness; B = Communication; C = Issues of control; D = Depression; E = Dementia; F = Management of aggression. Author Manuscript ** Mean rating of trainees selecting between 1 and 6 on a 6-point scale where 1 = Poor and 6 = Excellent. Author Manuscript Author Manuscript Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 16 Table 2. Author Manuscript Knowledge Test Results for Trainers and Trainees for Phase II Mean Number Correct (80 items)* Trainers Pre Post Matched Trainees Pre Post Pre Post (N = 153) (N = 153) (N = 42) (N = 42) (N = 329) (N = 187)** Mean 65.05 67.71 58.15 61.02 59.03 61.35 Standard deviation 3.93 4.40 7.09 5.51 7.24 5.43 Reliability .43 .64 .75 .63 .77 .62 Effect size*** .64 t(41) = 2.98, p < .01 * Trainees .45 t(514) = 4.78, p < .001 .37 t(152) = 3.17, p < .001 Tests with more than 20 items omitted were deleted from analysis. ** Author Manuscript This number reflects staff attrition during the training. *** Effect size = (Post mean – Pre mean)/Combined group SD. Author Manuscript Author Manuscript Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 17 Table 3. Author Manuscript Trainee Pre- and Posttest Results by Job Classification for Phase II Pre Job classification N* Nursing aides 193 89 Post SD N* Mean SD 55.76 6.76 108 59.11 5.12 63.73 4.60 44 64.10 4.50 Mean All Trainees Nurses t(280) = 10.21, t(150) = 5.67, p < .001 p < .001 1.378 1.037 Effect size** Matched Trainees Author Manuscript Nursing aide 90 56.48 6.60 90 59.39 5.03 Nurses 41 63.84 4.02 41 64.76 3.97 Effect size** * t(129) = 6.606, t(129) = 6.03, p < .001 p < .001 1.355 1.185 These numbers do not reflect those whose job classification was categorized as “Other.” ** Effect size = (Post mean – Pre mean)/Combined group SD. Author Manuscript Author Manuscript Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30. Smith et al. Page 18 Table 4. Author Manuscript Results of Attitude Survey of Trainers and Trainees in Phase II % Responding in Positive Direction Trainers Trainees Matched Trainees Post Pre Post Pre Post N = 328 N = 193 Change** N = 328 N = 193 Change** N = 153 N = 42 Change** 1 93 89 −.27 86 86 *.04 84 88 *.08 2 79 97 +.29 78 82 −.02 81 84 −.03 3 64 89 +.41 55 69 +.25 56 68 +.24 4 100 100 −.31 95 97 .00 97 99 +.03 5 95 97 −.12 87 91 +.08 88 92 +.17 6 54 81 +.41 55 63 +.11 54 65 +.17 7 98 97 −.05 83 86 +.10 84 88 +.10 8 98 100 +.03 92 96 +.10 92 97 +.11 9 100 100 −.01 96 95 −.01 95 97 +.10 10 67 78 +.27 50 53 +.03 49 52 + 02 11 81 78 −.10 55 58 +.04 54 55 +.04 12 100 95 −.22 90 93 +.06 93 93 +.04 13 10 100 −.09 97 96 .00 98 95 −.04 14 69 64 −.10 50 53 +.07 48 52 +.10 15 86 83 −.09 86 86 +.01 88 86 +.01 Author Manuscript Pre Question ** Change was computed by subtracting the pretraining mean from the posttraining mean. Author Manuscript Author Manuscript Issues Ment Health Nurs. Author manuscript; available in PMC 2019 August 30.