|
|
 |
Plenary Session:
Evaluation of a CME Program Using Pre- and Post-Tests |
| |
|
By Tamar Hosansky, Editor, Medical Meetings Magazine
MONTREAL—Continuing with the theme of measuring CME's effectiveness that was discussed by David L. Schlumper in the plenary session, "How Physicians Learn about Drugs," Lewis A. Miller, MS, Chairman, Intermedica, Inc., Darien, Conn., said that measuring behavior change requires more intensive surveys than the usual ones which measure physicians' satisfaction with an event. Providers need to obtain pre- and post-program data.
To illustrate, Miller used an example from the Mayo Clinic. Before an activity about Parkinson's disease and movement disorders, registrants were asked: 'What is your first choice drug for the initial treatment of Parkinson's disease?' After the activity, they were asked the same question. In addition, they were asked: Is this prescription different from what you prescribed before the conference?
Physicians can also be asked at the end of a program to give an example of something they learned that they would use in practice. Three months later, providers can follow-up with a post-survey to determine whether they had made those changes.
Measuring CME outcomes does not have to be expensive, said Linda Casebeer, PhD, Associate Professor, University of Alabama School of Medicine in Birmingham; and President, Outcomes, Inc. "Everybody has a fax machine. We do a one-page fax with case vignettes. We can do that for less than US$5 per physician." Physicians may be slow to use computers, as Dr. Sklar had discussed, but Dr. Casebeer said she has had good success using the older technology of faxes. "There's something about getting these cases by fax that makes them seem a little more urgent," she said.
Case vignettes were proven effective in measuring physician competence, she said, in a study published in the Journal of the American Medical Association in April 2000 (Peabody JW, et al., Comparison of vignettes, standardized patients, and chart abstraction: A prospective validation study of three methods for measuring quality.) The results demonstrated that case vignettes can measure physician competence. "What is so important about this study is that you don't need hospital or HMO data," she said. Vignettes focus on actual patient care, are non-invasive, and cost-effective.
Due to budgets constraints, commercial supporters are increasingly interested in seeing a return on their investment in CME, she said, and will award funds to CME providers who can demonstrate that ROI. And, as she and the other speakers discussed, there are a variety of ways for measuring ROI. "Prescribing is the obvious way, but industry is in business to change patient health outcomes. Physician, provider, and industry—where do those interests in ROI merge? It all comes together in a very similar place—[improving patient care]."
Conferences Rule
The final speaker circled back to the beginning of the conference, picking up on the theme of how physicians learn. Despite the fact that research has shown that traditional live activities with lecture formats are less effective than other methods in changing healthcare professionals' behavior, physicians still picked international conferences as their hands-down favorite method for obtaining education, according to a study published by the Board of Medical Education of the British Medical Association in December 2000, reported Edwin M. Borman, MB, ChB, Chairman, International Committee, British Medical Association, and U.K. Representative to the European Union of Medical Specialists.
Of the more than 2,000 doctors who participated, 40% of respondents chose international conferences as their preferred forum for continuing medical education activities. External courses came in second, with 22% of the votes; while other methods, such as internal meetings, e-learning, and informal discussions all lagged far behind.
While Dr. Slotnick and other speakers pushed for more interactive activities, Dr. Borman noted that physicians still preferred the most passive forms of learning such as conferences, while more participatory learning forums such as ward rounds, were the least popular.
And the Winner Is...
Nevertheless, GAME participants certainly seemed to enjoy interaction. After the last session, on demonstrating CME's ROI to pharma companies, the presenters divided the group into teams; each was allocated a betting pool of 500 points. As in "Jeopardy," players had to place their bets before they heard the questions. Facilitators then asked questions, based on the final session. Each team arrived at a consensus as to the answers, and the panelists — acting as judges — scored the responses. The winning team received a prize. OK, so it was only a pen, but the exercise produced a lot of laughter and encouraged attendees to talk to each other and reflect on what they had learned. |
|
|
