We finished collecting all Summer of Code midterm evaluations yesterday, and the results are promising. 90% of our students received successful midterm evaluations from their mentors, which is a slight decrease compared with 95% last year. When taking a look at why our numbers had dropped this year, our conclusion is that the lengthened program timeline allowed mentors and students more up front time to interact and work on project planning; in cases where there was not a good match between a student's schedule or skill set, the discrepancy became apparent much more quickly than in previous years.
While we learned a great deal from the feedback we received from the community as part of the evaluation process, three key themes emerged:
Communicate Early, Communicate Often
Nearly all students who received rave reviews from their mentors were in contact with their mentor multiple times per week, often communicating via voice as well as email and chat/IRC. Mentors who required status update reports from their students also felt they were better able to monitor student progress and provide guidance where needed, and many mentors reported that their best students sent regular status updates unprompted.
Life Happens
A few mentors lamented that they'd prefer to spend more time with their students, most often stating that it was their busy schedules which precluded a closer working relationship. Many students found that they had underestimated the amount of time it would take to complete key project milestones given other commitments. Most participants who reported that they'd like to more effectively manage their SoC tasks had a solid plan in place for devoting more time to SoC in the final month.
More Time is Better
The lengthened timeline was quite well received by the community. Many mentors reported that they used the community bonding period to better acquaint their students with the inner workings of the project, as well as to help them get up to speed with the project's tools and documentation. Several students used the community bonding to do more in depth research on their project topics, some finding that particular academic papers helped them optimize their project plan and resulting code before even getting started. Perhaps the coolest result of the community bonding period, many returning SoCers reported that the extra time gave them the opportunity to help newer students learn more quickly. We're looking forward to seeing if including the community bonding period encourages more students to stay on as long-term contributors to their projects.
If you'd like to learn more about the program evaluations, you can check out the mentor evaluation questions and the student evaluation questions. If you're curious about responses to a particular question, post a comment and let us know!
Google Summer of Code Blog
News on Google' s program to encourage
student participation in open source development.
Tuesday, July 31, 2007
Midterm Results Are In!
Subscribe to:
Post Comments (Atom)
6 comments:
I am quite curious about the most common project failure reason (did the student disappear? lack of communication? poor code quality?), and about how many students got a "Yes" answer by their mentors to question #7 ("Would you want to work with this student on a daily basis, e.g. sitting next to you in an office?").
Thanks a lot for the great job you guys @ google keep doing :),
cheers,
Savino Sguera
Awesome! Do you guys know how many students failed on last gsocs and are participating this year?
Cheers!
We can note the same question, many times, in both surveys. Do you do any kind of data cross ? Is there any case where the information provided by the student totally differ from the one provided by the mentor ?
Any student just gave up, like we saw last year ?
By the way, the comunity bonding period was a fantastic idea.
@ savino sguera: We had a few disappearing students, but overall most students who did not pass their midterm evaluation communicated very little with their mentors and had produced little or no code by the evaluation period.
Just looking at students who passed their midterm evaluation, ~ 700 mentors said they'd work next to their student on a daily basis. To be clear, though, quite a few folks who said "no" thought the question was a bit strange since open source development is by nature distributed.
Thanks for your kind words.
@ adriano monteiro marques: No data on that front, but for what it's worth I recognize no names from students who failed last year returning again this year. (Yes, I remember all 600 of you from 2006, or at least your email addresses. :)
@ igor feghali: We do cross reference student responses with mentor responses and, in most cases, their comments and responses to similar questions match. When they don't, it's usually a case of a student feeling s/he hasn't done very well, while the mentor thinks progress has been quite good.
We had around 40 students quit the program for various reasons, like health, time constraints, finding another job, etc.
Post a Comment