FOOD SECURITY OF ONE THOUSAND TRIBAL FAMILIES IN CHAKAI, JAMUI DISTRICT, BIHAR

Contract NO.PD:IND-3-001-10

EVALUATION REPORT

Manab Chakrabortymanabc@gmail.com

in.linkedin.com/in/manabchakraborty

Date: 19 July 2013

1

Map of Bihar to be inserted

2

AcknowledgementsThe evaluator thanks all the people who supported this evaluation by contributing their time and insights. In particular, they express their gratitude to Mr. Kashinath Metya and Mr. Samim Molla at Action for Social Advancement (ASA) and Mr. Subhashis Roy of Lutheran World Relief (LWR) , who were instrumental in helping organize the evaluators’ visit to Jamui. All errors and omissions remain the responsibility of the author.

Disclaimer: The views expressed in this report are those of the evaluator. They do not represent those of ASA, LWR or of any of the institutions referred to in the report.

Author: Manab Chakraborty wrote this report.

Table of Contents

I. Executive Summary............................................................................................................................ 4II. Introduction and Development Context......................................................................................7

(a) Project Objectives..............................................................................................................................................7(b) Project Evaluation Objectives.......................................................................................................................7(c) Evaluation Methodology................................................................................................................................8(d) Development Context of Project Area......................................................................................................8(e) Theoretical Underpinnings of the Project..............................................................................................9

III. Project Approach and Strategy......................................................................................................10(a) Project Approach............................................................................................................................................10(b) Implementation Strategies.........................................................................................................................10

IV. Evaluation Findings.......................................................................................................................... 11(a) Relevance...........................................................................................................................................................12(b) Effectiveness.....................................................................................................................................................14(c) Efficiency............................................................................................................................................................ 15(d) Impact..................................................................................................................................................................16(e) Sustainability....................................................................................................................................................17

V. Conclusions......................................................................................................................................... 20VI. Recommendations............................................................................................................................ 21VII. Overall Assessment and Closing Thoughts...........................................................................23

VIII. ANNEXES..........................................................................................................................................................24Annex 1: Project Area Demographics, Chakai Block...................................................................................24Annex 2: Evaluation Questions............................................................................................................................25Annex 3: Documents Reviewed...........................................................................................................................27Annex 4 Status Report on Achievement of Project Objectives...............................................................28Annex 5: Itinerary of Evaluator............................................................................................................................30Annex 6: Chronology of the Project....................................................................................................................31Annex 7: Photos of Project Area..........................................................................................................................32

3

4

I. Executive SummaryThis report is the evaluation of the project “ Food Security of One Thousand Tribal Families in Chakai, Jamui District, Bihar” implemented from July 2010 to July 2013 by Action for Social Advancement (ASA) with support from Lutheran World Relief (LWR). The LWR grant amount was US$203,224. The project goal was to “enhance agricultural production to attain food security among 1,000 tribal farmers in 20 villages of Chakai block in Jamui District by June 2013”. LWR and ASA commissioned Mr. Manab Chakraborty, an external evaluator to carry out terminal evaluation of the project to assess the success of the project in achieving the goal and objectives of the project and the overall effectiveness of the interventions, distill major learning on what worked well and what didn’t which would help ASA to have internal reflection on the project design, approaches, processes, organizational procedures to improve future action to maximize effectiveness as a development agency.

The evaluator developed a plan to conduct the evaluation study using a participatory methodology that included desk review of secondary material, interview of key informants, FGDs with farmers, SHG leaders, and local NGOs. A list of evaluation questions related to Relevance, Efficiency, Effectiveness, Impact, and Sustainability guided much of the study.

Relevance

The project proposal was based on a sound analysis of the 2009 drought situation, which highlighted the need to develop a more skilled agricultural extension service and build irrigation assets. The project was also relevant in that it addressed genuine needs of tribals on the basis of an appropriate analysis of the prevailing agrarian situation.

The project proposed a realistic set of activities, clearly linked to achievable objectives of increasing farm production and diversifying crop range.

Efficiency

The project activities took place within budget, and a no cost of one month was granted by LWR. The project budget was reasonable in the sense that none of the planned costs were excessive or irrelevant to the objectives

Effectiveness

There is ample evidence of the effective implementation of the planned activities, particularly those related to training and monitoring. If one area of weakness may be identified, it concerns the dissemination of monitoring information.

Impact

The project essential delivered on the planned medium term impacts, at least in relation to training and monitoring. However, the picture in relation to household food security though positive, is more nuanced. Interviews with SHGs and individual farmers revealed that pre-project household food supply came from own production (5 months), collection from forests (2 months), purchase from market (6 months). Post-project, food supply come from own production (8 months), collection from

5

forests (1 month), purchase from market (3 months). The increase in yield of paddy by 50%- 200% on pre-project cultivated land and barren land brought under cultivation due to irrigation and growing vegetables throughout the year has boosted food availability from own sources. Some visible impacts of higher food production are visible: a) there is more disposable cash income from sale of vegetables; b) Migration of women has almost stopped. Able but unskilled men still migrate for menial jobs that support their families for six months, while literate youth prefer going to garment factories in Karnataka. c) there is very keen interest to adopt new farming methods and crops.

Sustainability

The project has built solid institutions in the form of women SHGs and water user groups. Both these institutions have their own dynamics to survive well beyond the life of the project. The creation of trained village resource persons as frontline agriculture extension agents is a lasting contribution to the area.

Conclusions

- The project improved food security in 1013 households.

- The project execution was of high quality and responsive to the needs of ASA and the drought situation in Chakai.

- The relevance of the project to LWR would have been enhanced if an assessment of ASA’s management capacity had been carried out –ASA was capable of delivering more than envisaged in the project!

- The project was effective, in that most of the planned activities took place and were implemented to a high standard.

- The project’s efficiency was satisfactory, it produced value for money and resources allocation was appropriate.

- The project delivered on most of the expected short-term impacts. However the communication loop was incomplete, in that SHGs in the field were not systematically made aware of the use of the data they collected.

- The project contributed to the sustainability of ASA, both in financial terms and because it enhanced the organization’s skills and knowledge base.

Recommendation to LWR

Be explicit about expected added value of partnership. LWR is encouraged to develop further partnerships on the model of the one it engaged into with ASA. It is recommended that LWR should be as explicit as possible, when dealing with potential partners, about the added value it expects to bring to the partnership and about the benefits it expects to draw from it.

Make a realistic assessment of the level of support to be provided. The partnership between LWR and ASA was fruitful; ASA exceeded on almost all community mobilization and physical construction targets. The project could have done more if greater amount of financial resources were provided by LWR from sources within or through co-funding arrangements.

6

Consider having an explicit “inception stage” at the beginning of projects. This project commenced without any time to mobilize men, material, and money before jumping into action. Good projects allow for breathing space to make significant adjustments to the project design, and also take into consideration of changed circumstances since the project was originally designed.

Recommendations to ASA

Consider seeking new partnerships. The project with LWR was beneficial in enhancing ASA’s credibility, and special skills in building capacity of farmer institutions. ASA should develop further similar partnerships with multiple donors, provided they are based on a sound capacity building strategy and focus on satisfying clearly defined needs.

Include all poor households rather than only tribal households. The project reached out to 1013 households of the 1811 households in the project villages. An inclusive approach would have covered all poor households willing and able to participate in the project.

Review the process of dissemination of project impact. Social media network, and multi-media modes should be used to disseminate success, and draw funding support.

Have more locally recruited staff with vernacular skills: Only one of the ASA Jamui staff speaks Santhali dialect, this causes a communication gap between beneficiaries and ASA superviso

Overall assessment and closing thoughts

This evaluation has shown that projects based on a partnership between organizations with different comparative advantage may be relevant and effective. This was the case here, and it is clear that ASA drew significant benefits from the partnership in terms of capacity building and expertise on introduction of new agricultural practices.

Such partnerships, however, should be based on explicit and shared strategic outlooks on the part of both partners, and should be predicated on sustained communication both at operational and strategic level. Participation of senior ASA management in at project start up meeting had unintended negative impact on staff morale. The lapses in communications at strategic level were an issue in this project, which would have added value to implementation.

A mid-term strategic review of the project would have made the project more effective and relevant. For example, LWR by focusing only on scheduled tribes (17% of the population) in this project missed out an opportunity to reach out to Dalits (another 17% of the population), its’ other mandated target group. A strategic review might have reflected on this miss. It is a good practice to hold strategic reviews that should be encouraged.

7

II. Introduction and Development Context

This report is the evaluation of the project “ Food Security of One Thousand Tribal Families in Chakai, Jamui District, Bihar” implemented from July 2010 to July 2013 by Action for Social Advancement (ASA) with support from Lutheran World Relief (LWR). The LWR grant amount was US$203,224. The project goal was to “enhance agricultural production to attain food security among 1,000 tribal farmers in 20 villages of Chakai block in Jamui District by June 2013” (see Annex 1).

(a) Project Objectives

The Project objectives were

a. To establish irrigation facilities in 60 acres of land benefiting 100 tribal farmers through construction of water harvesting structures by 2012.

b. To improve the capacity of 1000 tribal farmers on improved agriculture practices through training, exposure visit, demonstration and hand on practice by June 2013

c. To organize 1000 village women into 60 self help groups (SHGs) and develop their capacity to access capital, fair price agricultural inputs and marketing services by 2013.

The women beneficiaries were organized into SHGs, imparted training on participatory, and facilitated by ASA, implemented multiple project activities, viz. land water development (farm ponds, lift irrigation, and check dams), enhancement of agricultural productivity through introduction of SRI, SWI, commercial vegetable cultivation, introduction of biological compost etc.

(b) Project Evaluation Objectives LWR and ASA commissioned Mr. Manab Chakraborty, an external evaluator to carry out an end evaluation of the project to assess the success of the project in achieving the overall goal and objectives of the project and the overall effectiveness of the interventions, distill major learning on what worked well and what didn’t which would help ASA to have internal reflection on the project design, approaches, processes, organizational procedures to improve future action to maximize effectiveness as a development agency.

Specific areas of assessment are:

1. The achievement of project outcomes against each of the set indicators/ targets during the project this period,2. The overall extent of project performance considering the Relevancy,

Efficiency and Effectiveness, Outcome/ Impact and Sustainability of the interventions made or outcomes achieved, and,

3. Analyze the strengths and weaknesses; issue and challenges of major components of this project and provide recommendation for future.

8

(c) Evaluation MethodologyThe methodology of the evaluation is set out in the TOR of the evaluation issued on June 2013, and LWRs Evaluation Report Guidance. A list of evaluation questions related to Relevance, Efficiency, Effectiveness, Impact, and Sustainability guided much of the study (see Annex 2).

In accordance with the agreed process, the evaluator developed a plan to conduct this evaluation using a participatory methodology that included, but not limited to:

Desk review of project documents (as per A n n ex 3 ) Key informant interviews Meeting with 5 Village Resource Persons (VRPs) Meeting/FGD with SHG and WUG members in 7 villages Meet representatives of UCO Bank, PRADAN, and World Vision1.

The full list of people met is in An n ex 4 .

Each of the Evaluation Questions listed in Annex 1 was discussed with relevant stakeholders: issues of project design, activities, management, human and financial resources and capacity building were discussed with LWR and ASA managers; issues of general impact of increased agricultural production with NGO managers in Jamui.

This evaluation did not present any particular methodological challenge and the evaluators were able to follow the standard methodology without problems. Project documentation, internal reports and financial statements were made available to the evaluator. Public documents giving contextual information about Jamui were also used.

The visit to Chaka i took place from 12 to 18 Ju ly inclusive.

(d) Development Context of Project AreaChakai is a Block in Jamui District of Bihar State, India.  It is located 51 KM towards South from District headquarters Jamui and 203 KM from State capital Patna towards west. Chakai is one of the 10 blocks in Jamui district. The total population of the block in 2011 was 2,34,932 (2001 187,034). There are 522 villages grouped in 23 Panchayats, and 0 towns in this block. Between 2001 and 2011, Chakai has recorded rapid increase in literacy; overall literacy rate was 49.12% in 2011 up from 36.11% in 2001. Increase in female literacy was even more impressive: from a mere 19.02% in 2001 to 36.95% in 2011. Scheduled caste and Scheduled tribe constitute 17.63% and 17.68% of the total population. The total area of Chakai is 72659.83 hectares, 43% of which is under cultivation. As far as the land use pattern is concerned Rice and Wheat are the prominent crops. Small quantity of Maize, Jack fruit (Hindi Katahal, botanic name Artocarpus heterophyllus) and Mahua (Botanic name Madhuca longifolia) are also grown by small land holders.

1 UCO Bank has extended banking facilities to ASA sponsored SHGs. Since 1990, PRADAN has been running a large number of livelihood initiatives In Chakai block. World Vision works in 40 villages of Chakai block on raising agricultural productivity.

9

The project target population is Santhal tribal households living below poverty line. They have limited arable lands on which low yield rainfed crops are grown. Significant amount of barren or wastelands exist, in the absence of land development measures these are used for open grazing. Supplementary subsistence is provided by minor forest produce, hunting, and fishing. Growing state control over forest resources and deforestation by organized by mafia is a matter of great conflict between Santhals and forest department. Of late, Naxalites have been able to feed on the general discontentment about mis-governance fuelling sense of injustice and alienation among tribals. The lack of health care, education, and gender discrimination hit women hard. Hence, livelihood generation and increase of income of these tribal farmers by various means is a basic imperative in the area.

(e) Theoretical Underpinnings of the ProjectThe project may be usefully viewed using Theory of Change. The Project seeks to achieve long term goal of food security using a set of building blocks – using activities, interchangeably referred to as outcomes, results, accomplishments or preconditions. Each change is tied to specific interventions that bring about intended and unintended outcomes. A Theory of Change would not be complete without an articulation of the assumptions that stakeholders use to explain the change process, and the preconditions that is required to effect social change. Assumptions elaborate both the connections between outcomes and the expectations about how and why proposed activities will bring them about.

ASA team has noticed strong, direct relationships between agricultural productivity, hunger, poverty, and sustainability. 100% of Chakai’s population live in rural areas and make their living from agriculture. Hunger and child malnutrition is rampant. Since, the tribals primarily obtain their income solely from subsistence farming (without the benefit of pro-poor technologies and access to markets), the incidence of malnutrition is high. Therefore, improvements in agricultural productivity aimed at small-scale farmers can benefit the rural tribal first and enhance food security. 

In a major study, the International Water Management Institute argues that managing rainwater and soil moisture more effectively, and using supplemental and small-scale irrigation, hold the key to helping the greatest number of poor people (Molden 2007). Water investments and policies for upgrading rainfed agriculture that go beyond controlling field-level soil and water to bring new freshwater sources through better local management of rainfall and runoff Increased agricultural productivity enables farmers to grow more food, which translates into better nutrition and, under market conditions that offer a level playing field, into higher farm incomes. With more disposable income, farmers are more likely to diversify production and grow higher-value crops. LWR and ASA adopted a three-pronged agriculture intensification strategy tapping rivulets and ponds for irrigation, raising cropping intensification, and transfer of farm production know-how.

LWR and ASA have not defined food security. Food security has three components, namely availability, accessibility and utilization. Both for ASA and LWR, greater

10

availability of food through own production was the principal concern under this project2.

III. Project Approach and Strategy

(a) Project Approach ASA and LWR designed the project strategy, with interviewees suggesting that ASA took a lead role in the initial design phase while LWR assumed a greater role in the first few months of implementation. LWR-India office jointly worked with ASA during the project formulation stages that includes problem analysis, project conceptualization and proposal development. Written in the months following crop failures in 2009, the proposal was focused on rebuilding capacity of marginal farmers by introducing new knowledge, technology, and inputs. This was justified by the situation prevailing at the time: dependence on monsoon when 90% of the crops are grown, high run off due undulating terrain, traditional methods of cultivation, and negligible crop diversification.

According to ASA and LWR staff, there were three main reasons for this for targeting tribal households engaged in smallholder agriculture:

Santhal tribals form a distinct socio-cultural group, hence needed particular attention. Government and other NGO have had a general thrust on economically disadvantaged farmers, but did not have any particular focus on tribal households.

Smallholder farms are suitable for labor-intensive crops such as vegetable cultivation. Crop diversification offered a good means to impart new knowledge, technology, and inputs.

ASA and LWR both have an ideological preference for tribal women in agriculture. They reckon narrowing their focus on tribal women maximizes the social impact of their investment.

(b) Implementation StrategiesASA and LWR had markedly different strategic outlooks on the project. ASA representatives indicated that they viewed the project as an opportunity to contribute their expertise to tribal welfare and to reinforce its on-farm activities. For LWR the project’s added value was in generat ing r ight advocacy messages as wel l as in the development of policy papers and in-house training skills. Both visions overlapped when it came to community centered livelihood development.

In the event, the two organizations were largely successful in reaching their common strategic objectives on training and enhancing agricultural productivity. However, a number of operational difficulties occurred in the course of project implementation, which staff on both sides ascribed to lack of field staff capacity, excessive workload and logistical challenges related to communications. While 2 For the purposes of this evaluation, and following United States Department of Agriculture ("Food Security in the United States: Measuring Household Food Security". USDA. Retrieved 2013-07-11), food security for a household signifies access by all members at all times to enough food for an active, healthy life. Food security includes at a minimum (a) the ready availability of nutritionally adequate and safe foods, and (b) an assured ability to acquire acceptable foods in socially acceptable ways (that is, without resorting to emergency food supplies, scavenging, stealing, or other coping strategies.

11

these challenges were real, the evaluator found that they were compounded by the lack of strategic dialogue between the two organizations and by the fragility of the project management structure.

The original Logical Framework and reports submitted by ASA shows that activities were centered on three key areas namely a) training, and capacity building of farmers, SHGs and project staff; b) Creating SHGs and WUGs and nurturing them; c) Building of irrigation assets. The project log frame had clearly articulated quantifiable intended outcomes against each project activity, but had not specified what might be desired medium term impacts.

IV. Evaluation Findings

Table 1 Simplified Logical Framework

Objective 1: Enhance agricultural production to attain food security

Activity Intended Outcome Actual Outcome

Training and capacity building

4 Project staff, 60 SHGs, 13 WUGs, 10 VRPs play active role in project management and implementation

4 Project staff, 65 SHGs, 26 WUGs, 15 VRPs play active role in project management and implementation

SHG and WUG Institution creation and nurturing

Capacity to manage own affairs1013 farmers in 65 SHGs, 196 members in 26 WUGs formed, and nurtured.

Building irrigation structures, and placing land development measures

Improved access to water for agriculture in 60 acres owned by 100 farmers

Improved access to water in 420 acres owned by 512 farmers

Objective 2: To improve capacity of 1,000 tribal farmers on improved agriculture practices

Training and capacity building

4 Project staff, 10 VRPs have developed plans to disseminate inputs to 1000 farmers on improved agriculture interventions

4 Project staff, 15 VRPs have developed plans to disseminate inputs to 1013 farmers on improved agriculture interventions

SHG and WUG Institution creation and nurturing

Farmers have improved land and have adopted improved

agriculturFarmers have improved land and have adopted improved agricultural practices (SRI, SWI, PVSP3, and vermin compost)

20 farmers have improved land and 1013 farmers have adopted improved agricultural practices

Building irrigation structures, and placing land development

Land leveling of 150 acres and 13 pond side plantation etc.

Land leveling work not undertaken

Farmers have improved land and have adopted improved agricultur

3 Participatory Varietal Selection and Promotion (PVSP) is one of ASA’s central interventions under the productivity enhancement activities. PVSP intends to understand the felt and perceived needs of the farmers for suitable crop varieties and allow them to test, identify, and adopt/spread the suitable “farmer preferred” varieties from a “basket of choices” provided to them.

12

measures

40% increase in paddy production for 1000 farmers

192 paddy assessments show 40% increase in paddy production.

Food security increased by 2-3 additional months, up from 5 months pre-project to 8 months post project.

1000 farmers attain food security for 7 additional months

Objective 3: To organize 1000 women in 60 SHGs

SHG Institution creation and nurturing

60 SHGs have set roles and responsibilities of office bearers and members for delivering credit, low cost agri inputs and market services

61 SHGs have set roles and responsibilities of office bearers and members for delivering credit, low cost agri inputs and market services

Financial systems set in 60 SHGs Financial systems set in 61 SHGs

60 SHG managing their banking transactions independently

3 SHG managing their banking transactions independently

The following findings stem from the evidence gathered by the evaluator.

(a) RelevanceThe droughts of 2009, 2010 and 2012 underlined the timeliness of the project and endorsed the need to develop a more skilled agricultural extension service and build irrigation assets. The project was also relevant in that it addressed genuine needs of the tribals on the basis of an appropriate analysis of the prevailing agrarian situation.

The project proposed a realistic set of activities, clearly linked to achievable objectives of increasing farm production and diversifying crop range. The project design further contributed to its relevance, by making good use of the comparative advantages of both ASA and LWR: the strengths of ASA listed above were complemented by the capacity of LWR to provide funds, mentoring, and international expertise based on its worldwide experience.

The project was lucky not to suffer from excessive staff turnover. ASA triumvirate, Mr. Kashinath Metya, Project Director, and Mr. Yogesh Dwivedi, Theme Manager Agriculture provided thought leadership to the project, while Mr. Samim Molla, Team Leader at field level provided much needed continuity of operations.

However, against this generally positive background, the relevance of the project was hampered by two key factors:- The project design failed to build potential for scaling up should the project be

successful. The budget neither included an element aimed at strengthening ASA’s capacity to manage large projects, nor made provision for rolling over to new geographical areas. The approach was based on the assumption that LWR National office had the skills and resources to supervise the project. The project design had not been preceded by an analysis of the institutional capacity of ASA or LWR. In the event the design proved to be over-optimistic; the senior levels of LWR while made of highly skilled, motivated and effective people, were thinly staffed. This meant that the LWR staff had little or no spare capacity to go beyond focusing the project activities and plan ahead for the future expansion.

13

o Another factor hampering the relevance of the project was that it lacked a comprehensive advocacy strategy. Absence of hard titles among the farmers is a major area of concern among the tribals, which requires policy level intervention. However, there was no attention given to such hard to crack policy issues. The project focused on developing internal skills and setting up monitoring and reporting processes: it did not provide for a specific strategy to bring the new information to the attention of the public and institutional stakeholders. Instead, the project implicitly relied on the existing LWR media and advocacy/lobbying operation, which itself lacked the resources to act effectively and in a timely manner on the information received.

Because of these factors, the relevance of the project was reduced. The resources needed to deliver the outcomes and the medium-term impacts were not all available to a sufficient degree. It is possible to identify three reasons for the project design’s failure fully to take management resource needs into consideration:

o ASA and LWR that designed the project did not have a record of working together. Both had a positive image of each other but they had not actually cooperated on a project before. As a result they did not have direct experience of each other’s strengths and weaknesses.

o The operational level quarterly meets between the two organizations conducted took the form of how to implement the project at the field level, rather than a strategy discussion focused on the achievement of advocacy outcomes.

o The two organizations did not entirely share the same strategic outlook on the project and did not have identical expectations of its outcomes. Very broadly, LWR was more focused on methodology and innovation, while ASA was more interested in field-level delivery.

(b) EffectivenessThere is ample evidence of the effective implementation of the planned activities, particularly those related to training and monitoring. If one area of weakness may be identified, it concerns the dissemination of monitoring

Box 1: An Example of project activity: the July 2010 start-up workshop

The aim of the workshop was to build common understanding between LWR and ASA on the project based interventions (objectives, activities, implementation schedule) and budget. The one-day workshop, held at Patna, covered the following issues:-Project Goal, Objectives, and Strategies-Project Log Frame-Reporting templates, monitoring and evaluation- Financial reporting and audit-Evidence based documentation of the project interventionsSenior officials from LWR and project executives of ASA attended the workshop. The main speakers at the workshop were from ASA.

14

information. Here is an overview of effectiveness for each area of activity listed in the summary logical framework:

Development of training course materials. This area of work has undoubtedly been fulfilled – indeed this has been done beyond expectations. The course materials on crop planting are comprehensive: they cover national norms on extension service monitoring, as well as domestic and other applicable laws and principles in India. The training materials cover issues that go beyond the sole crop plantation and post planting periods, and extend to the day-to-day monitoring of the use of irrigation assets by SHGs and raising user fees for maintenance. Beneficiaries identified and appreciated the training materials and recounted on the benefits derived from the skills learnt at several o f t h e training sessions. The fact that the ex t e r na l experts were invited more than once ensured that they were able to help review the methodologies developed initially and to adapt them to the Chakai situation in a participatory way, listening to the feedback of beneficiaries and ASA staff.

Implementation of training and exposure visit component. This was by far (in terms of resource used) the most important set of activities under the project. Training activities have included training of project staff and Village Resource Persons (VRPs) improved farm practices, construction of irrigation structures, management of SHGs and WUGs through exposure visits, and structured class room courses. Like wise, participants of SHGs, WUGs, and individual women farmers were taken on exposure visits, and formally trained by VRPs in land development, improved agricultural inputs, operation of saving bank accounts, and household planning for food self sufficiency.

The participants evaluated each training sessions, providing very positive feedback – a view reiterated by the participants who met the evaluator. Some VRPs who had produced monitoring reports in accordance with guidance received during the training sessions noted that they did not know how ASA processed their reports. (Only one field executive managed all the 15 VRPs; it was not humanly possible to devote much time to each VRG). In practice, the effectiveness of the project was achieved through “sheer force”, with ASA staff doing uncounted amounts of overtime.

Extension of irrigation in 60 acres. This set of activities was also largely implemented as planned. Key milestones were:o Construct 10 lift irrigation structures with pump sets and pipes

benefitting 390 farmers in 367 acreso Construction of 10 farm ponds benefitting 122 farmers covering 53

acresAdoption of Improved agriculture practices by 1,013 farmers. The

project has been able to touch the lives of 1,013 farmers through a variety of soft skills training methods. However, it is difficult to quantify how much hands on support were provided by VRPs and if the farmer’s food sufficiency has increased beyond 5 months estimated before the

15

implementation of the project. Further, rigorous research would be needed to draw on qualitative impact of the outreach work of VRPs.

Self-Help Groups And Water User Groups (WUGs). Records show that the project was able to organize 1,013 women in 65 SHGs, and 196 persons into 15 WUGs. WUG members are a sub-set of SHG members.

As per information made available to the evaluator, paddy production under the improved SRI method, has been double that of traditional methods. What is more encouraging is average yield under SRI has recorded 19% increase in 2013 over 2012 showing internalization of knowledge among the farmers.

Table 2: Paddy yield under traditional and SRI method

Year Average yield (ton/Ha): Traditional

Average yield (ton/Ha) : SRI

2011-12 3.19 6.97

2012-13 3.84 8.35

Sample size ?? 192

(c) EfficiencyThe project activities took place within budget, and a no-cost extension was granted by LWR. The project budget was reasonable in the sense that none of the planned costs were excessive or irrelevant to the objectives. Significant areas of spending included the following:

Program Operations and Activities: These were reasonable (43% of the project’s total, US$ 86,860). This head included expenses incurred on training, exposure tours, site demonstration, irrigation structures, and farm equipment etc.

Personnel costs. These were reasonable (29% of total budget, US$59,567). Salary costs were only budgeted for the proportion of staff time spent on the project. No fringe benefit was offered to staff.

Budget provision for consultants (US$5,348, 2.63% of total budget), and Overhead (US$ 9,677, 4.5% of total budget) is deemed on the lower side by this evaluator. The evaluator recommends that as a thumb rule the budget provision for external consultants and overhead should be 5% and 10% respectively.

Due to favorable currency exchange rate, ASA had 17% more Rupees over what was costed and approved by LWR. This helped them to achieve considerable more with same amount of dollars. LWR Country Director granted change of use in budget line on as and when needed basis. Table 3: Snapshot of Actual Target Achieved Against Project TargetsOverachieved

No. of households reached (1013/1000) No. of SHGs formed (65/60) No of WUGs formed (26/13)

16

No. of Lift Irrigation systems installed (15/10) Demonstration units (875/240) Exposure visits organized for village resource persons (4/2) Training of SHGs in SHG management (5/1) Man-days spent on VRP/Project staff training in SHG management (240 /56)

Underachieved Land leveling around ponds (0/13) Land development measures undertaken in no. of villages (2/20) Crop yield assessment (192/1000) SHGs having their independent relationship with banks (16/61)

(d) ImpactThe project essentially delivered on the planned medium-term impacts, at least in relation to training farmers, creating irrigation assets and raising agricultural productivity. The materials developed as part of the project doubtless enhanced ASA’s training and professional capacity. The same goes for the training-of-trainers element, which also contributed to the organization’s capacity to acquire and disseminate expertise on farmer’s handholding, knowledge dissemination and project and on-farm activity monitoring.

However, the picture in relation to household food security though positive, is more nuanced4. Interviews with SHGs and individual farmers revealed that pre-project household food supply came from own production (5 months), collection from forests (2 months), and purchase from market (6 months). Post-project, food supply come from own production (8 months), collection from forests (1 month), purchase from market (3 months). The increase in yield of paddy by 30%- 50% on pre-project cultivated land and barren land brought under cultivation due to irrigation and growing vegetables throughout the year has boosted food availability from own sources. Some visible impacts of higher food production are visible: a) there is more disposable cash income from sale of vegetables; b) Migration of women has almost stopped. Able but unskilled men still migrate for menial jobs that support their families for six months, while literate youth prefer going to garment factories in Karnataka. c) there is very keen interest to adopt new farming methods and crops.

Table 4: Impact on Food Supply on Randomly Selected HouseholdsBeneficiary Name Pushpa Tudu Barki Murmu Churki Marandi Sunita ??Village Garmah Garah Chandnya BurhiantanrOwn supply 2010 6 month 3 months 6 months 5 monthOwn supply 2013 8 month 6 months 8 months 8 monthForest supply 2010 0 3 months 3 months 2 monthsForest supply 2013 0 1 month 0 months 1 monthMarket Purchase 2010 6 months 6 month 3 months 5 month

4 Household food security assessment is divided into the self-sufficiency rate (S) and external dependency rate (1-S) as this divides the largest set of risk factors. Although households may desire a high self-sufficiency rate to avoid purchasing externally supplied food, this may be difficult to achieve especially where small holdings with low productivity prevail. Conversely, high self-sufficiency achieved through agricultural intensification can leave households vulnerable to production risks.

17

Market Purchase 2013 4 months 5 month 1 month 3 month

In sum, due to availability of irrigation water, and adaption of vegetable gardening, there is more food on the table for all beneficiaries. Undoubtedly, availability of assured irrigation has given unprecedented water security and certainty of double crop. This itself is a major impact.

It is not clear what impact increase in farm income is having on household consumption and nutrition. There is certainly an increase in ownership of consumer durables, and also intake of processed foods such as potato chips. The increased income has not gone for children education; beneficiaries cite lack of schools in the area as the reason for not investing in education.

Communication channels between field-based staff, ASA and LWR office have indeed been reinforced, as evidenced by the production and transmission of monitoring reports. However, the information was not processed into analyses and advocacy material. As mentioned above, the key reason for this weakness was the lack of advocacy and analytical capacity at headquarters level of ASA and LWR.

In this context, the project can be said to have gone some way towards fulfilling its long- term development objectives (to enhance higher food production by tribal households), but the evaluator found that the opportunity to highlight successes was not fully used: a more explicit and better funded strategy to process the data into analytical reports and advocacy material would have helped the project achieve more.

(e) SustainabilityThe project contributed to ASA’s sustainability in the sense that it helped enhance the skills of its staff and the credibility of ASA as a whole in relation to agricultural programme implementation. The project represented a massive transfer of knowledge and technical know-how to farmers, which would last for a long time. The cadre of VRPs will serve as catalysts for continuous

“Thanks to ASA project, my family is better off today than before the project. We have surplus vegetables to sell and donate to the chuch”. Pushpa Marandi,Santhal farmer from Bihar

Forty five old Pauline eyes swell with tears and body shivers as she recounts many years when her family of eight had to scrape with whatever was available. The months of July and August was particularly hard, the only recourse then was to borrow money at 114% (APR) from a moneylender at Bhatpara. Her family owned 0.12 hectares (2 acres) of land which produced paddy, potatoes, and some vegetables hardly enough for six months. Her husband Samuel Kisku would migrate to public worksites for menial jobs, while she would go to forest to pick dried twigs to sell in the local market. The six kids would be left on their own devices, intermittently attending the school in the next village. The elder children tended cattle on village common land.

In July 2010, things took a dramatic turn with the commencement of ASA project. She had reluctantly agreed to give a small patch to Parimal Mondal, ASA link worker, to demonstrate how to grow paddy seedbed. After few weeks, when she saw that the paddy plants grew in a healthy vibrant manner, her confidence in ASA team increased. Under Parimal’s guidance, she planted 0.06 hectare using Systematic Rice Intensification (SRI) method, while the other 0.06 hectare she planted under traditional method. the results were astounding: the yield under SRI method and traditional method was 7 quintals and 4 quintals respectively. Moreover, under SRI she used only 1.5 kg of seed in contrast to 120 kg under traditional method, while under traditional method and resplendent manner. This hands-on experience convinced her of superiority of SRI method.

Simultaneously, she took to vegetable cultivation in a big way. She abandoned potato, to grow a variety of creepers, three types of gourds, brinjal, tomato, chilly, radish, and water mellon. In 2013, she sold vegetables worth Rs. 25,000 in monsoon, Rs. 23,000 in winter, and Rs. 41,000 in summer – a total of Rs. 89,000 (USD 1780).

Life today has changed in many ways. Neither she nor her husband goes outside the village in search of work. Her family today has enough rice and wheat to meet their needs all year round. The sale of vegetables brings much needed cash to pay for tuition fees and upkeep of three boys (Rs. 45,000 a year), fertilizer, seeds, and prepaid mobile phone connection. She got two of her daughter married in style spending Rs. 20,000 in hard cash in each of the marriage. She has also bought two Hero bicycles for family use.

She is an active member of ASA-sponsored village self-help group (SHG) and water user group. She regularly deposits Rs. 10 every week at the all-women SHG meet. Her daughter Rosina is the local village resource person appointed by ASA.

She is also able now to generously contribute in kind to the local Church, social events in the village, and Congregation of Teresian Carmelites, where one of Pauline’s daughters is a nun.

What next? In 1987, to pay for her husband’s hospitalization in Ranchi, she had mortgaged 0.2 hectare (4 kattha) of land to the local moneylender. As soon as her eldest son completes graduation this year, she will start repaying the loan and redeem the land. But more immediately, she wishes to pick up few gifts and a nice sari for herself to attend Parimal’s marriage in September. After all she owes a lot to Parimal and ASA, without their help she would have been a bonded labor to the local moneylender. She thanks Lord Jesus for all His blessings: she has a life of dignity and full of happiness.

18

SHGs tend to have shorter life span wherever they serve as conduit for cash and in kind contributions. SHGs also tend to do well with charismatic individual leaders backed by supporting institutions. The SHGs would sustain so long a self-fulfilling activity such as mutual saving fund exists, or there is an expectation of external largesse which can be availed only if a quasi-corporate form of functioning is maintained. With plenty of Government schemes search for good SHGs, the ASA sponsored would live many years after the current project is closed. As of date, the SHG leaders and participants have expressed commitment to continue with their

“Thanks to ASA project, my family is better off today than before the project. We have surplus vegetables to sell and donate to the chuch”. Pushpa Marandi,Santhal farmer from Bihar

Forty five old Pauline eyes swell with tears and body shivers as she recounts many years when her family of eight had to scrape with whatever was available. The months of July and August was particularly hard, the only recourse then was to borrow money at 114% (APR) from a moneylender at Bhatpara. Her family owned 0.12 hectares (2 acres) of land which produced paddy, potatoes, and some vegetables hardly enough for six months. Her husband Samuel Kisku would migrate to public worksites for menial jobs, while she would go to forest to pick dried twigs to sell in the local market. The six kids would be left on their own devices, intermittently attending the school in the next village. The elder children tended cattle on village common land.

In July 2010, things took a dramatic turn with the commencement of ASA project. She had reluctantly agreed to give a small patch to Parimal Mondal, ASA link worker, to demonstrate how to grow paddy seedbed. After few weeks, when she saw that the paddy plants grew in a healthy vibrant manner, her confidence in ASA team increased. Under Parimal’s guidance, she planted 0.06 hectare using Systematic Rice Intensification (SRI) method, while the other 0.06 hectare she planted under traditional method. the results were astounding: the yield under SRI method and traditional method was 7 quintals and 4 quintals respectively. Moreover, under SRI she used only 1.5 kg of seed in contrast to 120 kg under traditional method, while under traditional method and resplendent manner. This hands-on experience convinced her of superiority of SRI method.

Simultaneously, she took to vegetable cultivation in a big way. She abandoned potato, to grow a variety of creepers, three types of gourds, brinjal, tomato, chilly, radish, and water mellon. In 2013, she sold vegetables worth Rs. 25,000 in monsoon, Rs. 23,000 in winter, and Rs. 41,000 in summer – a total of Rs. 89,000 (USD 1780).

Life today has changed in many ways. Neither she nor her husband goes outside the village in search of work. Her family today has enough rice and wheat to meet their needs all year round. The sale of vegetables brings much needed cash to pay for tuition fees and upkeep of three boys (Rs. 45,000 a year), fertilizer, seeds, and prepaid mobile phone connection. She got two of her daughter married in style spending Rs. 20,000 in hard cash in each of the marriage. She has also bought two Hero bicycles for family use.

She is an active member of ASA-sponsored village self-help group (SHG) and water user group. She regularly deposits Rs. 10 every week at the all-women SHG meet. Her daughter Rosina is the local village resource person appointed by ASA.

She is also able now to generously contribute in kind to the local Church, social events in the village, and Congregation of Teresian Carmelites, where one of Pauline’s daughters is a nun.

What next? In 1987, to pay for her husband’s hospitalization in Ranchi, she had mortgaged 0.2 hectare (4 kattha) of land to the local moneylender. As soon as her eldest son completes graduation this year, she will start repaying the loan and redeem the land. But more immediately, she wishes to pick up few gifts and a nice sari for herself to attend Parimal’s marriage in September. After all she owes a lot to Parimal and ASA, without their help she would have been a bonded labor to the local moneylender. She thanks Lord Jesus for all His blessings: she has a life of dignity and full of happiness.

19

work.

WUGs are self-sustaining so long collective action for harnessing water, distributing them, and maintaining irrigation assets result into considerable gains for all concerned, and exceeds gains that would be made by pure individual household action. ASA and LWR have built a solid foundation by creating few good WUGs.

Water users' groups play a very significant role in the management of water resources at field level, particularly insofar as the distribution and use of irrigation water is concerned. Users' groups under this project function in a variety of capacities, ranging from advisory to managerial and from coordinating to quasi-judicial. In the absence of general legislation governing the water users, the conduct of these associations are largely left to voluntary compliance of informal codes of behavior.

However, in view of their importance as an adjunct to policies aimed at sustainable use of scarce natural resources, and devolving to them selected governmental responsibilities particularly in the field of irrigation water management, the formation and functioning of irrigation water users' groups tend to be governed by specific legislation. Bihar needs subordinate legislation provides rules for the formation, membership structure and functioning of groups, including in particular, a model internal statute. In future ASA and LWR can take the lead in advocating for legitimization of water user association and provide thought leadership.

The project itself has achieved a reasonable degree of sustainability in the sense that its benefits have outlived the project period itself. However, ASA as an organization remains fragile and donor-dependent (it is also facing continuing security risks in Jamui where Naxalite activities have increased of late). A more integrated fundraising strategy based on multi-year programming rather than single projects is probably achievable at this point, ASA having achieved a high level of credibility with donors. Work on multi-donor finance should be part of any such programming exercise.

Of late, ASA has

awakened to the potential of leveraging government resources. ASA has garnered some support under National Mission for Micro Irrigation (NMMI); launched by the Union Ministry of Agriculture in June 2010 for increasing water use efficiency, crop productivity and farmers income. Under NMMI, the project has received 73 drip irrigation units valued approx. USD 58,000 (Rs. 2.9 million) and mobilized farmers contribution of USD 5,800 (Rs. 0.32 million). ASA is adapting micro irrigation with fertigation in a big way. The gains with fertigation in non-rainfed vegetable cultivation are significant (Bhaskar 2013).

Table 5: Productivity gains (tonnes per acre)

Box 3: Ms. Churki Marandi, beneficiary at Majlakhola Village

Chruki is certain that the benefits of the project are here to stay. The project has brought water to her parched 3 acres of land, and what is even better taught her how to grow paddy using SRI system, vegetables in summer and winter. This knowledge of new improved practices no one can take away. She grows enough for 8 months now compared to 6 months before the project.

20

Crop Rainfed Canal/Tubewell

Micro-irrigation with fertigation

Papaya NRFC 25-30 60-80 (14 months crop)Tomato 4-5 15-25 30-50 (Achieved 60+tonnes)Onion NRFC 8-12 18-20 (Achieved 22+)Capsicum NRFC 10-15 25-40 (Achieved 45+)Okra 2.5-3.0 5-7 12-15 (Achieved 15+)Green Chilly 5-10 15-20 25-40 (Achieved 45+)Ginger NRFC 8-10 12-18 (Achieved 22+)Turmeric (Dry) NRFC 1.8-2.0 2.5-3.5Source: Bhaskar 20131) NRFC =Not a rainfed crop. 2) Wheat and rice are crops where research is underway, and major breakthrough using drip irrigation for these crops is just round the corner, say experts. This could improve yields upwards of four times the current levels.On 19th June 2012, ASA launched Mahila Kisan Sashaktikaran Pariyojna (MKSP) with US$ 852,000 (Rs. 42.6 million) under National Rural Livelihood Mission of Government of India (http://www.mksp.in/). MKSP will build on the successes of the current project, and also extend to 4 other locations in Khagaria and Purnea districts of Bihar. The aim of MKSP project is to enhance status of small women farmers by improving their strategic role in agriculture through building two tier women led collectives, increased agriculture productivity and ensuring efficient backward & forward market linkage systems. ASA has formed a producer company to professionally market ever growing vegetable production of its’ SHG members. The producer cooperative has 835 members each of whom have bought shares of Rs. 200 each. Mr. Ashis Mondal, ASA Director and a sitting member of National Advisory Council5 (NAC) played a helpful hand in ensuring grant support to producer companies in the Central Government budget 2013.

In March 2013, ASA also facilitated distribution of 90 agricultural contains seeds, and fertilizer supplied by the Jamui District Agriculture department.

V. ConclusionsThe conclusions presented here are based on the findings set out in the previous section and on the contextual information presented in section III.

a) The project was relevant to the needs of ASA and the situation in Chakai. Its relevance was enhanced by focusing on improving food security in tribal households through a series of drought-proofing measures.

b) The relevance of the project to LWR would have been enhanced if an assessment of ASA’s management capacity had been carried out. Such an organizational audit could have brought to light the relative lack of resources at project management level and responded to it, either through additional budgetary provisions in the project, or by allocating management responsibilities differently within the existing team. See chapters III (ii) and the findings on relevance.

5 The NAC of India is an advisory body set up to advise the Prime Minister of India. The task of the National Advisory Council (NAC) is to provide inputs in the formulation of policy by the Government and to provide support to the Government in its legislative business. NAC is also called as Shadow Cabinet. Sonia Gandhi is the chairperson of NAC.

21

c) The project was effective, in that most of the planned activities took place and were implemented to a high standard. F a r m e r s drew tangible benefits from the project. However it lacked a comprehensive advocacy strategy, and relied implicitly on the outreach work of LWR as a whole to deliver the advocacy element of this project. The lack of resources at LWR level hampered the effectiveness of this approach, leading to relatively weak dissemination of the information gathered by project. See Evaluation Findings Section IV on relevance and effectiveness.

d) The project’s efficiency was satisfactory, it produced value for money and resources allocation was appropriate. See Evaluation Findings Section IV on efficiency.

e) The project delivered on most of the expected short-term impacts. However the communication loop was incomplete, in that beneficiaries in the field were not systematically made aware of the use of the data they collected. They also lacked information about the data gathered by their counterparts in other projects.

f) The project contributed to the sustainability of ASA, both in financial terms and because it enhanced the organization’s skills and knowledge base. The project gave ASA foothold in Bihar, which helped it to launch new initiatives in neighboring districts of Bihar and Jharkhand. One of the major change in ASA approach has been a desire to tap into government schemes to further its objectives. Convergence with government programmes would not only add to scaling up but also make funding more diversified and hopefully, reduce the risk of fluctuations in income.

VI. RecommendationsThis section, recommendations are addressed separately to ASA and LWR. The recommendations are based on the findings and conclusions set out above

Recommendation to LWR

a) Be explicit about expected added value of partnership. LWR is encouraged to develop further partnerships on the model of the one it engaged into with ASA. It is recommended that LWR should be as explicit as possible, when dealing with potential partners, about the added value it expects to bring to the partnership and about the benefits it expects to draw from it.

b) Make a realistic assessment of the level of support to be provided. The partnership between LWR and ASA was fruitful, but ASA clearly demonstrated it had the capacity to do more. LWR could have, on the basis of capacity audit of ASA, could have provided more financial resources from within and through co-funding arrangements. LWR managers indicated that they devoted more energy to the partnership than they originally expected. It is recommended that this issue be carefully considered in any future partnership, including by conducting an organizational audit of the partners.

c) Consider having an explicit “inception stage” at the beginning of projects. This project commenced without any time to mobilize men, material,

22

and money before jumping into action. Good projects allow for breathing space to make significant adjustments to the project design, and also take into consideration of changed circumstances since the project was originally designed. The start up meeting agenda should reflect on what adjustments, if any are required. Hence, the start-up meetings require participation of senior management of the partner agency, which was not the case with the project under review.

LWR should consider encouraging partners to adopt this practice, thus instituting an “inception phase” in the first few weeks of projects. This would

give implementers an opportunity to review the project logical framework, while ensuring that they do not unnecessarily delay the start of actual activities.

d) Adapt and develop new tools for assessing tribal women’s empowerment. To understand change, we need to measure impact with right tool. LWR may consider adapting USAID created in 2012 Women’s Empowerment in Agriculture Index (WEA) to directly capture tribal women’s empowerment and inclusion levels in the agriculture sector. The index considers five factors to be indicative of women’s overall empowerment in the agricultural sector: decisions over agricultural production; power over productive resources such as land and livestock; decisions over income; leadership in the community; and time use. Tribal women may be considered empowered if they equal or surpass a pre-set score in at least four of the components.

Recommendations to ASA

a) Consider seeking new partnerships. The project with LWR was beneficial in enhancing ASA’s credibility, and special skills in building capacity of farmer institutions. ASA should develop further similar partnerships with multiple donors, provided they are based on a sound capacity building strategy and focus on satisfying clearly defined needs. See Conclusion (b).

b) Review the process of dissemination of project impact to ensure that written documents are appropriately edited in a way that is consistent with the needs of target audiences. Social media network, and multi-media modes should be used to disseminate success, and draw funding support. See Conclusions (e) and (f).

c) Develop a multi-year strategic plan, which can be used as a framework when seeking support from funders. ASA should attempt to identify one or two strategic partnership donors to provide support including core funding, while other donors remain project based. ASA should also consider dedicating more management-level resources to address fundraising issues. See conclusion (viii).

d) Have more locally recruited staff with local vernacular skills: All of the ASA supervisory staff are non-Santhals recruited from outside the area. Only one of them speak local Santhali dialect. ASA should either invest in vernacular skills of the staff to interact with the beneficiary population, or make concerted efforts to recruit Santhals. Though 10 of the 15 VRPs are female,

23

all the supervisory staff of ASA Jamui unit are male, which causes gender imbalance.

VII. Overall Assessment and Closing ThoughtsThis evaluation has shown that projects based on a partnership between organizations with different comparative advantage may be relevant and effective. This was the case here, and it is clear that ASA drew significant benefits from the partnership in terms of capacity building and expertise on introduction of new agricultural practices.

Such partnerships, however, should be based on explicit and shared strategic outlooks on the part of both partners, and should be predicated on sustained communication both at operational and strategic level. Participation of senior ASA management in at project start up meeting had unintended negative impact on staff morale. The lapses in communications at strategic level were an issue in this project, which would have added value to implementation.

A mid-term strategic review of the project would have made the project more effective and relevant. For example, LWR by focusing only on scheduled tribes (17% of the population) in this project missed out an opportunity to reach out to Dalits (another 17% of the population), its’ other mandated target group. A strategic review might have reflected on this miss. It is a good practice to hold strategic reviews that should be encouraged.

24

VIII. ANNEXES

Annex 1: Project Area Demographics, Chakai BlockSr.# Village name No. of HH

in villageTotal Population

SC ST Project Beneficiary HH

1 Karmakola 26 128 - 128 262 Bathanakola 57 330 - 247 433 Bahrakola 59 703 154 703 594 Rahima 120 659 - 448 825 Ghama 27 149 - 149 276 Chandnya 31 186 - 186 317 Mahisapatthar 131 572 36 264 608 Pasara 48 155 - 155 489 Burhiatanr 68 205 20 178 59

10 Majhalakola 53 144 - 136 5011 Dohaman 72 454 - 257 4112 Konjhi 61 267 - 267 6113 Tara Khar 62 375 - 230 38

14Jamuniya Tand 92 275 - 105 35

15 Chir Patthar 77 877 37 475 4216 Baghopatar 128 678 - 678 12817 Pojha 162 632 124 389 9918 Prachi 263 1,046 793 253 1519 Dhamna 167 644 94 391 5720 Chehera 107 394 38 338 12

Total 1,811 8,873 1,29

6 5,977 1,013Source: Samim Molla, Team Leader, ASA Jamui

25

Annex 2: Evaluation QuestionsDAC

Criterion

Evaluation Question Related sub-questions

Relevance To what extent was the project, as designed and implemented, suited to context and needs at the beneficiary, local, level?

- Were the objectives of the project in line with the needs and priorities for local development, given the context?

- Should another project strategy have been preferred rather than the one implemented to

better reflect those needs, priorities, and context? Why?

- Were risks appropriately identified by the project? How appropriate are/were the strategies developed to deal with identified risks? Was the project overly risk-averse?

Efficiency To what extent was the results were achieved as a result of the project interventions, or were there any major external factor?

- How well did management systems (finance, HR, procurement etc.) operate efficiently to support project?

- Was there a reasonable relationship between project inputs and project outputs?

- Did institutional arrangements promoteEffectiveness To what extent was the

project, as implemented, able to achieve objectives and goals?

- To what extent have the project’s outcomes/outputs been reached?

- To what extent was the project implemented as envisaged by the project document? If not, why not?

To what extent the project reached out to the target households? How were the households identified?

- Were the project activities adequate to make progress towards the project objectives?

- What has the project achieved? Where it failed to meet the outputs identified in the project document, why was this?

Outcome/

Impact

What were the intended and unintended effects on people and their institutions? To what extent has the project put in place processes and procedures supporting the role

- To w h a t e x t e n t have t h e r e a l i za ti o n o f p r o j e c t objectives and project outcomes had an impact on the specific problem the project aimed to address?

- Have the targeted beneficiaries experienced

26

of people’s institutions contributing to resilience of local agriculture?

tangible impacts? Which were positive; which were negative?

- To what extent has the project caused changes and effects, positive and negative, foreseen and unforeseen, on agriculture development?

- Are the project-sponsored SHGs likely to have a catalytic effect on Panchayati Raj Institutions (PRIs), and local government in a participatory manner? How? Why? Examples?

- Has the project ben able to create demand for farm based livelihood prototypes, and attract new resources from private and public sources?

Sustainability To what extent has the project, as designed and implemented, created what is likely to be a continuing impetus towards local development?

- To what extent has the project established processes and systems that are likely to support continued impact?

-Are the involved parties willing and able to continue the project activities on their own?

LWR value add To what extent LWR’s accompaniment contributed in meeting the project activities?

- What was L W R able to accomplish, through the project that could have been achieved by alternative projects, other donors, or other stakeholders (Government, NGOs, etc.).

- Did project design and implementing modalities exploit LWR’s comparative advantage in the form of an explicit mandate to focus on irrigation systems, training on improved agriculture practices and upfront credit for farmers to invest in better seeds and tools?

27

Annex 3: Documents Reviewed

Bhaskar, R.N., Policy watch: Drip irrigation key to productivity, Monday, Jun 24, 2013, 11:19 IST | Agency: DNA

UNDP. Human Development Report 2013 – "The Rise of the South: Human Progress in a Diverse World", UNDP, New York

Indicus Analytics. Bihar Development Report 2010, Indicus Analytics, New Delhi. Also available at http://www.slideshare.net/indicusanalytics/bihar-development-report-2010

Molden, D. (Ed). Water for food, Water for life: A Comprehensive Assessment of Water Management in Agriculture. Earthscan/IWMI, 2007.

Websites

Madhyam Foundation http://madhyamfoundation.org/pdf/annualreport_08-09-part-2.pdf

Chakai demographics http://imaginebestcom.ipage.com/districts/jamui/chakai.php

World bank website htt p:// web. worldbank . org

Irrigation Management in Wheat video, Digital Green video, 2012 test.digitalgreen.org/analytics/video/?id=10000000020634

Vegetable Gardening video, Digital Green, uploaded on June 6, 2012 http://www.youtube.com/watch?v=UNDLM6h3_Zk

Pr oj ect docum ent ation:

ASA Proposal Document, 2010Start up Workshop Report, ASA, 31st July 2010Project Budget July 2010-June 2013Partner Progress and Financial Report, 18.10.2010, 14.01.2012 Tour Report by Subhashis Roy, LWR. 6th and 7th of November 2012

Mahila Kisan Sashaktrikaran Pariyojana Launching Event, June 2012 http://www.asaindia.org/Project%20launch%20workshop/MKSP%20Launching%20workshop-Jamui.pdf

28

Annex 4 Status Report on Achievement of Project ObjectivesObj Project Activity Status

#1 To improve access to water to irrigate in 60 acres Capacity to irrigate 420 acres (168 ha) created

1.1 Organise one start-up workshop for ASA project team with LWR

Held on 31st July 2010

1.2 Conduct Concept sharing meeting in 20 villages for development of common understanding on project

94 concept sharing meeting conducted in 20 villages

1.3 Conduct facilitation to form 13 Water Users Group (WUG)

26 WUG formed

1.4 Conduct 2 Trainings of 2 days duration for 52 WUG leaders on execution of construction work, management and maintenance and distribution of water

2 x 2 days x 196 members +1x1 days x196 Members = 784 +196 = 980 mandays training given

1.5 Train 100 farmers (user group wise) on construction and maintenance of irrigation structures and distribution of water

322 farmers trained

1.6 Construct 3 lift irrigation structures with pump sets and pipes benefiting 60 farmers covering 40 acres

10 LI benefiting 390 farmers covering 367 acres

1.7 Construct 10 farm ponds benefiting 40 farmers covering 20 acres

10 farm pond implemented covering 122 farmers benefiting 53 acres

#2 To improve the capacity of 1000 tribal farmers on improved agricultural practices

1013 tribal HHs on improved agricultural practices

2.1 Selection and training of 10 village resource persons as extension agents

15 village resource persons promoted

2.2 Organise 2 exposure visits for 10 VRPs and 4 project staff

4 exposure visits arranged for 15 VRPs, 4 Project staff and more than 100 SHG members organized. 60 inter village exposure also arranged covering all 1013 farmers.

2.3 Conduct 2 phase training of 1 days duration for 10 VRPs and 4 project staff on improved agricultural interventions

12 training arranged per cropping session

2.4 Develop 12 no. of demonstration units on SRI/SWI/PVSP/vegetable cultivaon and vermin compost

875 demonstrations: Monsoon 390 (65 x 2 x 3), Winter 292, and Summer 193.

2.5 Land leveling, bunding, soil treatment in 150 acres and side plantations in 13 ponds

Not done

2.6 VRPS Conduct 80 trainings in 2 phases (40 trainees in each batch) for 1000 farmers (25 in each batch) on improved agricultural interventions

6 training x 6 phases x 28 participants for 1013 farmers conducted.

2.7 Organise 2 Farmer’s convention for sharing and cross learning among farmers

1 farmer convention conducted Dec. 2012

2.8 Undertake land development measures in 20 villages

Land development measures took in 2 villages only covering 20 farmers

2.9 Provision of inputs (seed/seed treatment material, pesticides, fertilizers, plant nutrient material) to 1000 farmers through SHGs

1013 farmer got inputs

2.10 Providing technical support through VRPs to 1000 farmers during execution of agricultural activities

Done

29

2.11 Conduct 6 rounds of Assessments to measure crop yield for 1000 farmers involving agriculture experts

2 rounds of paddy yield assessment conducted by block agriculture officer in 192 fields

Obj Project Activity Status#3 To organize 1000 village women into 60 Self-help

groups and develop their capacity to access capital, fair price agriculture inputs and marketing services by 2013

1013 tribal women in 65 SHGs organized (61 SHGs surviving in June 2013).

3.1 Form 60 SHGs in 20 villages covering 1000 women farmers

65 SHG promoted in 20 villages

3.2 Conduct 2 training of 2 days duration for 10 VRPs and 4 project staff on SHG Formation and management, and family based food sufficiency planning (56 man-days)

2 training x 2 days for 15 VRPs + 4 Project staff + 6 training x 1 day for 15 VRPS and 4 Project staff organized (240 man-days)

3.3 Conduct one exposure visit of 10 VRPs and 4 project staff to successful SHG groups

1 exposure visit outside state and 17 inter village exposure visit arranged.

3.4 Conduct 1 Training for 60 SHGs on SHG Formation and management, and family based food sufficiency planning

5 training arranged for 65 SHGs

3.6 Conduct 20 Refresher Training (one in each of the 20 villages) for 60 SHGs

20 Refresher Training arranged for 20 Villages + 2 Training organized for SHG leaders (30 persons batch)

3.7 Interface meetings of SHGs with financial and agriculture resource institutions

UCO Bank, Chakai opened accounts of 16 SHGs, discussion with SBI and Bihar Kshetriya Gramin Bank is going on intermittently

30

Annex 5: Itinerary of Evaluator

27th June 2013 Concall with Kashinath Metya, Programme Director ASA, Mr. Subhasish Roy, Programme Manager, LWR and Mr. Samim Molla, Team Leader ASA Jamui regarding TOR of the Evaluation

9-10 July 2013 Study of Project documents

12th July 2013 Traveled from Delhi to Deogarh

13th July 2013 Interviewed beneficiaries, Ms. Rosina Kishku, VRP, Parimal Mondal, ASA staff at Ghumah and Garah village

14th July 2013 Interviewed beneficiaries, and Mr. Puran, VRP at Chandnya and Majlakhola village

15th July 2013 Interviewed beneficiaries at Karmakola,Burhiatanr, and Barakola visited ponds in both locations

16th July 2013 Made presentations to Mr. Kashinath Metya, and Mr. Samim Molla of ASA and Mr. Subhashis Roy, and Mr. Ravinder Kr. Singh of LWR

17th July 2013 Called PRADAN, World Vision, Madhyam Foundation, UCO Bank, and BDO, Chakai

18th -19th July 2013 Returned by train from Deogarh via Asansol to Delhi

31

Annex 6: Chronology of the Project1 July 2010 Commencement of the Project

31st July 2010 Start up Workshop held at Patna

August 2010 SRI Concept sharing meetings started

September 2010 SHG promotion initiated

October 2011 ASA-LWR 1st join reflection meeting

November 2010 First Water User Group formed; Wheat Intensification & new vegetables introduced

December 2010 First lift irrigation system implemented

January 2011 1st farm pond implemented

February 2011 2nd lift irrigation system implemented

March 2011 Summer vegetable gardening introduced

April 2011 Introduce group farming initiative

June 2011 3 more farm pond implemented

July 2011 Monsoon crops introduced among 1013 farmers

October 2011 1st yield assessment conducted in presence of official of block agriculture department

December 2011 65 Self Help Group promoted

November 2011 Scale winter crops with more than 600 farmers

February 2012 Summer vegetable with more than 200 farmers

March 2012 6 Dug well implemented

April 2012 3rd lift irrigation system implemented

July 2012 Monsoon crop (SRI, maize & monsoon vegetable) repeated with all 1013 farmers

October 2012 Winter crops with more than 800 farmers.

November 2012 2nd yield assessment of paddy conducted in presence of govt. officials.

January 2013 ASA-LWR 10th joint reflection meeting

March 2013 4 more Lift irrigation, 5 farm pond implemented

June 2013 3 more Lift irrigation, 1 farm pond implemented

32

12-19 July, 2013 External Evaluation of the Project

Annex 7: Photos of Project Area

Photo 1 ASA Team with Kamalphool SHG Members at GhamahPhoto 2 Lift irrigation at Ghamah

Photo 3 Meeting with Majlakhola SHG Photo 4 Chilly field at Chandnya village

Photo 5 A typical Santhal House Photo 6 Drip irrigation set

33