1998-99 Evaluation Report

Ithaca Evaluation Group

October, 1999

Executive Summary 

The MentorNet program pairs undergraduate and graduate women who are considering careers in engineering and science with mentors who work in science/engineering industries. Mentors and protégés apply to the program, are matched by MentorNet staff, and then communicate for at least one academic year using email. In 1998-99, 515 pairs from 25 schools and more than 200 companies participated in the program.

The evaluation for the MentorNet project in 1998-99 was goal-oriented, concentrating on explicit and implicit program goals concerning the mentor-student process. Evaluation activities included midyear and year-end web-administered questionnaires with an email and telephone survey follow-up of non-respondents to gauge bias. Unobtrusive (but consented) monitoring of all email traffic among a random sample of participants was also conducted. Other evaluation activities included meeting with the project Advisory Board and key project staff, reviewing information on the recruitment and matching process, analyzing the MentorNet electronic discussion lists, and reviewing newsletters and other data sources.

This report summarizes analysis of MentorNet-provided and Ithaca Evaluation Group (IEG)-acquired data. Analyses include generation of baseline statistics, assessment of factors related to satisfaction with mentor-protégé matches, and inspection of outcome measures for differences related to student degree program, matching, and university. Content analysis of email communications and questionnaire free-response items was done using a coding system developed during the 1997-98 year.

Participants' assessments of their mentoring relationships are very positive. On a five-point scale (1-5), participants responded very favorably(>4) to queries concerning the comfort asking and answering questions of their mentoring partner, their post-mentoring interest in their major, their post-mentoring interest in working in industry, their willingness to recommend the MentorNet program to others, and their perceptions of their mentoring partners' interest in them. Mentors indicated strong interest in participating in the program again next year.

Analysis of the frequency of email communication reveals that email exchanges were more frequent earlier in the year. Frequency of email initiation is a significant predictor of overall mentoring success--strong positive correlations exist between reported frequency of email contact and mentor and student satisfaction with the match. This relationship holds even when satisfaction with frequency of email contact is held constant.

To stimulate mentoring exchanges and remind participants to remain in contact, MentorNet's mentoring specialist periodically provided email prompts. Part of our analysis focuses on the perceived utility of this prompting by participants. Although both groups had favorable responses overall, we did find differences between mentor and protégé responses, with a discrete minority of mentors reporting that they were unnecessary. We suggest experimenting in 1999-2000 with an adaptive prompting system to provide explicit coaching to some mentors only when it is requested or its need is identified.

The evaluation found few differences between male and female mentors with respect to outcomes - they report comparable levels of email traffic, similar satisfaction with quality of matches, and similar ratings of most other outcomes. Three differences surfaced in relation to discussion topics: half as many male mentors reported discussing their future career plans as did female mentors; female mentors were twice as likely as male mentors to discuss the treatment of women at their company; and students with female mentors were more likely to report that they discussed "balancing career and life" than students with male mentors. For most conversational topics and other outcomes, however, there were no observed differences that were related to mentor gender.

On the midyear and final questionnaire, participants were asked about the content and value of their email exchanges, using both topic checklists and free-responses (essay) questions. More than 75% of participants discussed their backgrounds, the mentor's job, college life, and the protégé's career plans. Most questionnaire respondents also discussed social interactions and/or jokes, the industry workplace, balancing career and life, managing time and/or stress, and job hunting and/or interviews.

Three open-ended questions asked mentors and students to identify the most useful topic they discussed, the most valuable aspect of email monitoring, and the positive outcomes of the mentoring experience. Detailed analysis of these responses and of actual email traffic (received through email monitoring) generated a number of content themes underlying ementoring.

a) Impartiality. One key feature of ementoring is that it provides protégés with an opportunity to ask questions of an impartial person. Many of the mentoring relationships develop an atmosphere in which the mentor is a confidante who is "safe" to bounce ideas off or to whom one can air insecurities or other concerns. For example, one student commented, "The most valuable aspect was being paired up with a complete stranger and getting to know their life, as well as them getting to know you. It was valuable to me to be able to go to a neutral person on work place, scholastic, and personal matters."

b) Personal relationship and encouragement. Personal interactions associated with mentoring surface in student reports as another of the most valuable aspects of mentoring. Students' identify different aspects of their relationship as being particularly valuable at different stages of their college careers. Freshmen emphasize the encouragement they receive from their mentor, while seniors appear to view their ementoring relationships as the starting point of professional networking. One student commented, "This (ementoring) reaffirmed my belief that I want to be an engineer. It helped me to decide that working in industry would be good, and when I was feeling bad because of school or whatever she (my mentor) gave me encouragement."

c) School matters and coursework. Students who are at the beginning of their college careers are most likely to find discussions of school matters useful. Often, mentors' comments about their similar difficulties with coursework in college help students persist: "She (my mentor) was very helpful...with difficult classes, such as chemistry. She encouraged me and reminded me that it was an introductory course that was meant to weed out the people who aren't serious about science majors. Her positive comments and the way she related my situation to her college experience helped me to succeed in that class."

d) Job workplace & skills. Learning more about mentors' jobs and workplace environment is the first or second most frequently mentioned topic by freshmen and sophomores in all three free-response questions. This category includes understanding what skills mentors used in their jobs, balancing a career and family, and mentors' perspectives on women in the science and engineering workplace. One protégé commented, "I found to be most helpful the duties my mentor had at her job. Industrial Engineering is so broad, it was good to hear some specifics. It also helped me to learn which classes, my mentor actually USED on the job." Another protégé wrote, "I feel better about becoming a female engineer, as it is a male oriented environment. I am more certain that the path I am choosing to succeed later in life and career is correct."

e) Job hunting, future plans, and careers. Many students are curious about what the real world of engineering in industry entails and appreciate the chance to learn more about life after school from their mentors. Not surprisingly, as student progress through college, more and more of them identify future plans and careers as useful topics of discussion. Students commented on the helpful advice that they received about interviewing--"Some tips that I received for interviewing helped me to get a really good internship offer"--and about preparation to enter the engineering workforce--"I am more aware of what to expect from my new job and career path. I know I am not alone....;"

f) Self-confidence. One program outcome cited by students is the impact of the mentoring process on their self-confidence. This is an important finding because low self-confidence has been identified as one of the key factors that contributes to women's exodus from engineering and other scientific fields. One junior wrote, "I gained a great deal from this e-mentoring experience. For one, I learned to think more positively about myself and be more confident in my abilities. I learned that failure will happen, but you have to get up one more time than you get knocked down to succeed. Most importantly, I've gained a true role model in my mentor. My mentor is the first female engineer I've met. She's everything I've aspired to be and more."

Mentors identify different positive outcomes and valuable aspects than do students. Their responses indicate that they participate because they are committed to helping women students in engineering and science and are willing to share their knowledge and experiences. Positive outcomes reported by mentors included the satisfaction of being helpful, sharing knowledge, acting as a sounding board for students' ideas, and helping their protégés work through personal problems. Interestingly, mentors frequently stated that participation in the program helped them to improve their interpersonal skills. For example, "This experience helped me to explore my leadership potential since I was able to guide, listen, help and assist someone not only in the academic and professional aspect but also the personal aspect." The mentoring process also gave experienced scientists and engineers the opportunity to examine their own career decisions and aspirations.

Overall, evaluation data indicate that the project had a very successful first full year of mentoring. MentorNet achieved a 280% increase in participation relative to 1997-98 while simultaneously enhancing mentors' and protégés' mentoring success. The program seems to be well positioned for the 1999-2000 program which will entail another scale-up of similar magnitude.