By: Averil Spencer
I spent a good deal of my time in college analyzing development/aid projects. I critiqued how these projects were run, who was chosen to participate, what elements were incorporated into the program, etc. now that I am not sheltered in the ivory tower of academia, I realize this world of monitoring and evaluation is not so black and white. Designing a program is difficult enough, but then assessing it for social impact is a definite challenge. I am a firm believer that all aid and development related programs should have stringent M&E before, during, and after the program. An Acumen employee one told me that their approach to M&E is like a doctor taking a patients pulse. They want to tap into and monitor the living elements of the program and not have to do an “autopsy” of the project once it is over or something went wrong, which is often what happens to aid projects. I know everything I do not want to do for the impact assessment of Camp VOICE but as I sat down to devise a plan for evaluation, I also realized I knew very little about what positive elements I wanted to incorporate.
As impact assessment is being finalized, I knew that we wanted to measure the academic progress of girls along with the ever-elusive gender empowerment aspect of the camp. When it comes to M&E, academic progress is easy to evaluate because there are a wide range of tests. Our team decided testing English proficiency and communication were the top priorities. We are using the NCERT test along with elements of the Test of English as a Foreign Language (TOEFL). Variations of the NCERT test will be given to all girls in all 11 schools in grades 6-9, the girls who might attend camp. We will also give them a listening and speaking section from the TOEFL. With these two tests we will be able to gauge their English comprehension and communication skills before the camp. All girls will also be tested at the beginning of the next academic year after summer break. They will take the same TOEFL test because even within that test there is variation on answers and will not be confounding, and they will take the 5th grade NCERT test. This test is especially important because it is used in the Gray Matters Capital (GMC) affordable private school (APS) rating report. GMC has a rating tool that assesses the finical, academic, and infrastructural situation of low-income schools. By using the NCERT test for our girls, we will be able to compare campers’ results with the control group of girls who did not attend camp but also the results of students from 250 schools within the GMC network. We will also be testing teachers who are assisting with the camp in the same manner and with a control group.
With English communication skills monitored, we also wanted to see if the camp affected girls’ attendance, class participation, and GPA. These elements will be evaluated through data collected after camp and going through ledgers of past attendance and grade records. These schools have impeccably kept records concerning attendance, grades, and money flowing into the school. Classroom participation will be more difficult to assess but we are fortunate to have a capable team who is devising a rubric for classroom observations. By structuring classroom observations, we will be able to see if girls who went to camp are more confident in school and participate in classroom activities and lessons.
Our ultimate challenge for this camp will be to see if we can come up with an assessment system for gender empowerment. World bank, the IMF, and countless aid agencies have tried to quantify gender empowerment and thus far, there has not been an overwhelmingly accepted metrics. An important aspect of the camp is providing girls the tools, training, and support to empower themselves within their families and communities, however they see fit and is comfortable for them, an assessment of gender empowerment is important. As of now, the team is still searching for a way to quantitatively and qualitatively measure this effectively. For Camp VOICE, girls’ empowerment would be defined as resourcefulness, conflict resolution skills, access to support systems, and self-esteem. We believe these characteristics/skills are crucial for girls in low-income communities in Hyderabad to have in order to minimize their vulnerability and increase their opportunities.
Over the next two weeks, we will be creating the assessment tools for the camp and begin preliminary data gathering. Stay tuned to hear how we actually go about tackling our big challenge… quantifiable gender empowerment metrics.
 I also hesitate to use gender empowerment too frequently especially when I want to quantifiably assess it. Ever since the UN decided to incorporate gender into aspects of development (through WID- women in development and GAD- gender and development) programs, the term has become a buzzword and used to increase funding for projects without the proper understanding of culturally complex gender issues.