Evaluation Minutes March 2001
March 19, 2001
Evaluation Committee Meeting
Members Present: Penny Asbury, Helen Baker, Kathleen Bors, Jodie Jackson, Alex Lubman, Ken Shannon
Members Absent: Anna Reno
Others Present: Jacquelyn Copenhaver, Jane Horst, Kelly Fritz, Elizabeth Richmond
Evaluation Committee membership: Jodie reported that she emailed the Evaluation Committee members who had not attended in over a year a memo asking them if they were still interested in continuing on the committee and stating that no response would be accepted as a withdrawal from the committee. Several members were officially removed from the committee membership as a result of this memo. Helen Baker noted that there were no community members on the committee and suggested that Hilda offer a general invitation for community members to join.
Update on Electronic Evaluation: Reminder: Jodie reminded members that there are two questionnaires being implemented in the new RHEP evaluation:
- The Medical Student Baseline Data Questionnaire
- Student Evaluation of Rural Field experience (SERFE) (for all students)
Jodie informed the Evaluation Committee that 3rd and 4th year medical students were emailed to complete this questionnaire on 1/3/01. There was a technical malfunction between 1/3/01 and 1/9/01. Apology email stating that problem was resolved was sent out on 1/9/01. The 3rd reminder email was sent out on 2/11/01. As of 3/14/01, a total of 120 baseline questionnaires had been completed (out of a total of approximately 400 3rd and 4th year medical students).
Jodie informed the Evaluation Committee that the plan for improving the response rate is as follows:
- a modified 4th email will be sent out next week.
- Persistent non-responders will next be contacted with a hard copy memo.
- For future years, the plan is to get this questionnaire completed by medical students in their 2nd year. (On January 26, at the WVU Rural Health day, we captured 77 baseline data questionnaires from WVU 2nd year students. July 24 will present a similar opportunity to capture MU 2nd year students and WVSOM has given the completion of this form as an opportunity for WVSOM 2nd year students to complete their computer competency requirement this semester.)
Jodie informed the Evaluation Committee that, due to a technical malfunction, the SERFE was not available until 2/9/01. 49 evaluations out of a possible 135 RHEP rotations since January 1 have been completed. 2/3 of these were completed in the first 15 days of March, so most of our mediocre response rate can most likely be attributed to technical bugs.
Jodie informed the committee that the plan to improve response rate is as follows:
- 1st reminder email to arrive one week before completion of the rotation;
- 2nd email 1 week after completion of the rotation;
- 3rd email will be 3 weeks after completion of their rotation;
- Alex and campus coordinators will get an email notification regarding non-responders 5 weeks after completion of their rotation.
- Site coordinator will get notification that a student has not completed an evaluation of a previous rotation before the student gets to their consortia for a subsequent rotation
The possible problem of missing RHEP rotation evaluations for 4th year
medical students who complete an RHEP rotation in the Spring (i.e., shortly
before graduation) was discussed. In order for campus coordinators
and campus faculty to follow-up with these students before they graduate,
the decision was made to ask Mike to email the campus coordinators a list
of these 4th year medical students to the campus coordinators at the following
times:
WVU 4th year medical students: April 15th and April 30
WVSOM 4th year medical students: (Jodie to ask Patti Crawford)
MU 4th year medical students: (Jodie to ask Mike McCarthy)
There was discussion about whether it would be possible to permit administration of a hard copy of the SERFE when the "site coordinator is in a pinch". One such example given was when the local internet provider system goes down and there is not internet access. This happens frequently in some consortia. The site coordinators were asking, would it be possible in a case such as this to administer a hard-copy SERFE just to those students who are in their last week of rotation? One site coordinator suggested numbering the hard-copy SERFEs given to site coordinators and limiting the number given so as to discourage overuse of the hard-copy. This issue was tabled in order to give the evaluation team that has been working on the revised evaluation forms time to consider the pros and cons of this request.
There was a request from site coordinators to recommend a formula for use when inputting the "number served" in TRACKER for newspaper articles written by students as part of their community service requirement. Should they consider the entire circulation of the newspaper or a certain percentage? Opinions differed regarding whether this was an issue for the Community Service/Outreach Committee or the Evaluation Committee. Jodie was concerned that utilizing the entire circulation of the newspaper might appear to look like an attempt to overinflate community service numbers. One person suggested differentiating between newspaper circulation and other community service projects in the report. It was decided to defer this to Hilda for some advice.
Approval of memo to School Committee regarding RHEP student definitions:
The committee felt that it was not the place of the Evaluation Committee
to determine the policy underlying the student definitions of "IN", "OUT",
"EXTERN", "OTHER RURAL" AND "RHEP" but to make sure that the definitions
were consistently utilized. Hence, since there was no argument presented
to change the definitions, the committee approved a memo to the School
committee that essentially said "In order to assess the impact of RHEP,
it is important that we consistently define/identify an RHEP student.
This has been done through Policy 09-02 (which replaced Policies 95-05
and 95-06). The Evaluation Committee recommends that we strictly
adhere to the Policy as written. Below is the current policy:"
The changes made by the Evaluation Committee as the meeting are in bold.
Jodie will present this memo at the next School Committee meeting.
Should residents by reported in TRACKER? A Site Coordinator requested
that the Evaluation Committee address this issue in order to assure consistency
of reporting by all consortia. The Evaluation Committee decided that
whether or not residents should be reported in TRACKER was an issue for
the Advisory Panel but, if they are reported, that they should NOT be reported
in the Student Roster as is currently being done by a few consortia.
Jodie will present this issue for consideration at today's Advisory Panel
meeting.
