|
Program Outcomes Research
Pre- and Post-Tests of S.E.E. Program Audiences
From the outset, the S.E.E. program has been subject to careful evaluation. In order to determine the effectiveness of the program, each audience undergoes pre- and post-testing. All tests and ratings were done anonymously. These studies help guide the improvement process of the program as well as documenting program outcomes. The results show highly significant improvements in participant knowledge about epilepsy and its treatment, and highly significant reductions in patients’ fears of death and brain damage (the main cause of psychosocial dysfunction in epilepsy), and significant reductions in patient’s hazardous medical self-management practices following the program (please see Table 1.) This last result is important because the UCLA research showed that two-thirds of people with epilepsy were doing something physically hazardous in their own medical self-management.
Table 1
Objective Measures of S.E.E. Program Effect
(Includes U.S. and New Zealand audiences)
N = 361
|
|
|
|
|
|
|
|
|
|
Pre-Test
|
|
Post-Test
|
|
|
|
|
Subscale
|
Mean
|
s.d.
|
|
Mean
|
s.d.
|
|
t-test
|
P
|
|
|
|
|
|
|
|
|
|
|
|
Fear of Death and Brain Damage
|
2.84
|
2.58
|
|
0.84
|
1.39
|
|
12.9
|
.00001
|
|
Hazardous Medical Self-Management Practices
|
6.39
|
2.96
|
|
2.20
|
2.06
|
|
22.0
|
.00001
|
|
Total Misconceptions About Epilepsy
|
14.65
|
6.57
|
|
5.66
|
4.05
|
|
22.1
|
.00001
|
These immediate post-test results were replicated in the controlled, long-term follow-up study (Epilepsia 31(1), 75-82.) It was gratifying to find that the gains patients made immediately after the program were maintained and even furthered four months after patients had attended the program. The impact of the program is sometimes dramatic. It is not unusual for participants to cry at the point in the program when unrealistic fears of death and brain damage are dispelled. Parents of children with epilepsy have been particularly relieved and are most vocal about their appreciation.
On the post-test, participants were asked if they thought the program would help them with several problems that people with epilepsy and their families face in everyday life. The list of problems was derived from the NIH study at UCLA, and from the work of others. Overall, participants expected the S.E.E. program to help them with quite a number of the problems they were facing (Please see Table 2.)
Table 2
Subjective Ratings of Improved Psychosocial Functioning
(Includes U.S. and New Zealand audiences)
In percent, N = 1,295
|
|
|
|
|
|
|
|
|
How will this program affect your…
|
Will make much worse
|
Will make some worse
|
Will have no effect
|
Will make some better
|
Will make much better
|
Total % better
|
|
Ability to work with the doctor
|
|
|
8
|
36
|
56
|
92
|
|
Ability to cope with depression and anxiety
|
|
|
8
|
40
|
51
|
91
|
|
Ability to cope with family problems
|
|
|
12
|
40
|
48
|
88
|
|
Ability to find or keep a job
|
|
1
|
30
|
33
|
36
|
69
|
|
Improve self-esteem
|
|
|
15
|
33
|
52
|
85
|
|
Ability to cope with social stigma
|
|
|
13
|
37
|
49
|
86
|
|
Sense of being in control of epilepsy
|
|
|
10
|
32
|
57
|
89
|
|
Confidence in dealing with epilepsy
|
|
|
3
|
29
|
68
|
97
|
|
Will help improve seizure control
|
|
|
5
|
38
|
57
|
95
|
|
Reduction in fear
|
|
|
11
|
27
|
62
|
89
|
|
Improvement in ability to manage medications
|
|
|
10
|
30
|
60
|
90
|
Patients who answered that the S.E.E. program would have “no effect” upon a particular problem area often felt compelled to write an explanation of their answer in the margins of the answer sheet. They explained that they answered “no effect” because they were not currently experiencing any difficulties in that area of their life. We also followed up on the 1% that indicated that they thought the program might make their ability to find or keep a job “some worse.” They noted that the program informed them of possible stigma that they might face in job-hunting that they had not considered before. They were concerned about possible discrimination, but acknowledged that they had learned useful skills for overcoming it if they encountered it.
Participants were also asked to make subjective ratings of the S.E.E. program on their post-test. Our interest here was to determine the overall impact of the program from the eyes of the participants. How easy it was the program to understand? Would participants recommend the program to others after attending it? Possibly the strongest recommendation, would participants be willing to spend another two days attending the program again? (Please see Table 3.) Over the years, many S.E.E. program participants attend the program a second time, and some have attended three or more times. It is not unusual for a patient to bring family members who did not attend a first program to the second program – and for family members to do the same for patients who were reluctant to attend a first program. Initially we were very concerned that the two-day format would discourage people from attending. On post-test evaluations, by far the most common criticism the S.E.E. program receives is that participants would like to see it extended to three days!
Table 3
Global Subjective Ratings of the S.E.E. Program by Participants
(Includes U.S. and New Zealand audiences)
In Percent, N = 1,295
|
|
|
|
|
|
Total:
Good / Excellent
|
|
How would you rate S.E.E. ?
|
|
|
Poor
|
Fair
|
Average
|
Good
|
Excellent
|
|
|
|
|
1
|
15
|
84
|
99
|
|
Would you recommend S.E.E. to others ?
|
|
|
Recommend Against
|
Probably Not
|
No Opinion
|
Recommend For
|
Highly Recommended
|
|
|
|
|
|
10
|
90
|
100
|
|
Was the S.E.E. presentation understandable to you ?
|
|
|
Very Unclear
|
Somewhat Unclear
|
Average
|
Clear
|
Very Clear
|
|
|
|
1
|
2
|
18
|
80
|
98
|
|
Would you like to attend again ?
|
|
|
Never
|
Not Likely
|
Don’t Know
|
Probably Would
|
Definitely Would
|
|
|
|
4
|
6
|
31
|
60
|
91
|
Since about one-fourth of S.E.E. audiences were made up of professionals, we also asked them to make anonymous ratings of the S.E.E. program. These professionals were primarily nurses and physicians. Also included were a few pharmacists, EEG technicians, psychologists, social workers, and other health professionals (please see Table 4.)
Table 4
Summary of Anonymous Professional Ratings of S.E.E.
N = 502
|
|
Poor
|
Fair
|
Average
|
Good
|
Excellent
|
Total: Good & Excellent
|
|
Instructor
|
|
Knowledge
|
|
|
|
4
|
96
|
100
|
|
Stimulation
|
|
1
|
1
|
11
|
88
|
99
|
|
Communication Skills
|
|
|
1
|
7
|
91
|
99
|
|
Topic Material
|
|
Informative
|
|
|
|
5
|
94
|
100
|
|
Relevant
|
|
|
|
9
|
90
|
100
|
|
Comprehensive
|
|
|
1
|
9
|
90
|
99
|
|
Useful
|
|
|
1
|
9
|
90
|
99
|
|
Method of Presentation
|
|
Organization
|
|
|
2
|
15
|
82
|
97
|
|
Pace
|
1
|
2
|
5
|
28
|
63
|
91
|
|
Learning Environment
|
1
|
1
|
5
|
23
|
70
|
93
|
|
Audience Participation
|
1
|
2
|
7
|
21
|
69
|
90
|
|
Audiovisual
|
|
Quality of Audiovisuals
|
1
|
2
|
7
|
21
|
69
|
90
|
Professionals have been enthusiastic about the S.E.E. program. This was a surprise to us. The program was, from the outset, made for the lay audience. Nevertheless, from the first program, professionals attended. We were concerned that the span of the audience would be too great if professionals attended. We would have to successfully address an audience that ranged from our baseline educational level of the sixth grade all the way to board certified neurologists. Yet the combination was a success. Professionals liked taking the seminar with consumers. On the post-test they point out they especially liked to hear how consumers respond to the information presented. They also liked hearing consumers describe their personal experiences during the seminar. At the same time, professionals noted the information provided by the S.E.E. program was sophisticated and state-of-the-art. We finally relented to professionals’ requests and agreed to seek approval for the program for professional training. The S.E.E. program has been approved for 14 hours of AMA Category I credit for physicians at various institutions, including UCLA, Vanderbilt, Northwestern, A.I. duPont Institute, VA Medical Centers, University of Washington and others. S.E.E. is approved to award 17.6 hours of continuing education for nurses through NCNA, an accredited Approver by the American Nurses Credentialing Center’s Commission on Accreditation. Professionals continue to make up an important part of the S.E.E. program audience.
|