Guidelines for Pre-funding Evaluation Visits

Topic: Guidelines for visits (Originally written for pre-funding visits but can be used for review visits also)
Author: Premnath, Asha-Pune
Date: 15 Aug 2003

Step 1: Research

Collect all relevant documents regarding the proposed project from the Asha person who received the proposal and/or the organization directly. This could include:
- The proposal (ofcourse)
  - The proposal should include two very clear and distinct sections 1) About the organization and 2) Focussed information about the project that is proposed for funding. If this information is not available or satisfactory, please put in efforts to collect this information.
  - Make sure the project proposed is very clearly defined and you can clearly demarcate the boundaries of the project from other projects that the same organization may be running (and may be receiving funds from other sources). Understand what the funds will be used for. (All this can often be done via email or phone or via a local contact)
    - Why? This is in order to pre-empt a possible problem at the time of review of the project after funding --- sometimes money is directed by organizations into other projects run by them without consulting Asha. Also, it is difficult to evaluate the impact of Asha funds if the funds go into a bigger pot where it looses its identity. Yet another reason is that sometimes an organization will be involved in activities (for example, activism, income generation schemes) other than education and one does not want Asha funds being used for those purposes unless approved by Asha.
  - Make sure you know which are the geographical locations for which funds have been requested (example-- school in this village; sometimes, more specifically, only 6-7 standard of non-formal school in say village ABC). If you cannot do this over phone or email, then the visit to the organization office will often help you clarify the situation. Sometimes, if the organization's office is at the project site, then one needs to keep some time out before the site visit begins to discuss these matters.
    - Why? Sometimes, organizations run several schools at different locations--- often funded by many sources. In order to be able to evaluate the effectiveness of our effort, we need to know which are the schools we shall be funding, if at all. Also, one needs to be able to evaluate the merit of each proposed location by looking at local problems (by talking to children and parents) --- this is an important purpose of the site visit.
  - Also make sure you have clarity on heads against which funds have been requested (example: teacher's salary, stationery, furniture etc). If you cannot do this over phone or email, then the visit to the organization office will often help you clarify the situation. Sometimes, if the organization's office is at the project site, then one needs to keep some time out before the site visit begins to discuss these matters.
    - Why? One of the goals of the site visits is to see if the requested funds are reasonable and required for the purposes mentioned. So one needs to know exactly what funds are being asked for. If the project is funded, one also needs to use this information at a later stage in order to evaluate the progress, verify the investments made, record the changes observed etc.
- Supplementary information
  - Website of the organization
  - Brochures of the organization
  - Annual reports of the organization
  - Any newspaper/ magazine articles etc that may have appeared about the organization. Internet search may help here also.
  - It is now-a-days a good idea to check on the internet (Google) for any other reports about the organization. Sometimes, you can get reports written by other funding groups. Sometimes, one can also get ratings by organizations such as the Give Foundation.

Try finding out about the organization through contacts --- for example, sometimes, faculty in many academic depts or organizations involved in community programs, development , social studies or those offering social service degrees can be good places for information. In our experience, references from respectable sources are the best way to identify genuine and good organizations. (Almost everything else can be faked.)

Step 2: Visit to the organization's office

It is normally preferable to visit the organization's office in advance of the site visit. Even if the organization's office is located at the project site, it helps to first discuss the project proposal with the office bearers of the organization before proceeding for the site visit.
Start by explaining to the office bearers the following:
- About Asha and how it works, what it funds etc
- About the procedure from project proposal to project funding and review. They should know what to expect.
- And finally, what you hope to expect in the evaluation visit and associated discussions.
What to discuss? The main aspects of the proposal:
- Organization: Ask questions to discover more about the organization and its various projects. The questions should probe the following:
  - History of the organization and legal status; Registration; Associated organizations/ trusts -- especially affiliations; FCRA clearance
  - Confirm absence of affiliations to religious or political organizations
  - Get clarity on vision and goals of the organization: What is their mission? And where do the education projects fit in? Do they have clear long-term plans and if so, what are they? How do they plan to grow and stabilize the organization? Are they thinking of future operations in term of volunteers/ employees and funds?
  - Does the organization have local support and/or participation?
  - What are their achievements at the ground level?
  - Finances and accounting procedures: It is a good idea to get a feel for how accounts are kept? In particular, how do they distinguish the accounting of funds from different sources targeting different goals? Ask to see previous reports to other organization? What is the quality of reporting especially financial? Are they getting audited?
- Project vs Organization: Find out how and where the proposed project fits in the scheme of things of the organization
  - How important is the education project in the organization's agenda?
  - Is the organization committed to the education projects? (Sometimes the organization may be committed to other causes and education may be a peripheral effort --- in such cases, one needs to judge how effective or successful the effort could be.)
  - The education project should be clearly defined and distinguished from the other projects. Overlaps (for example, teachers doubling as volunteers for another effort) if any need to be identifed.
- Project: Get clarity on the proposed project
  - Define scope of project clearly in terms of
    - Geographical locations (is it village A or village B or district C)
    - particular schools or particular classes (is the balwadi or the primary school or class 5-10? etc)
    - particular groups (is it girls only or boys only or only physically challenged kids etc) served
    - scope of activity for funding (only basic education or vocational or health education etc)
  - Understand need of the project clearly and purposes it will serve
  - Define deliverables or metrics (or encourage project to define deliverables) against which say a project can be evaluated after funding. (This is especially important for Asha since the person coming for a review person may be a different person from the person evaluating the project for funding. Such metrics help in maintaining continuity in project monitoring.) Examples would be:
    - Number of kids who will be kept in school (and will not drop-out)
    - Number of students who can read sentences
    - Number of students who can do simple math
    - Number of students who can write a post card
    - Number of students who can maintain an account book.
    - Number of students who can stitch a complete dress.
    - Number of child marraiges or average age for marraige of girls.
  - Understand all the budget items. Your understanding should be sufficient to justify to an Asha projects group gathering why each budget head costs as much as quoted.
  - Get clarity on all the people who will be directly involved in running the project. You should have a complete list of people (and some impressions on their backgrounds) at this stage so that you can meet as many as possbile during the site visit. If you happen to meet them, try to understand their level of commitment, why they are committed etc.

Step 3: Visit to project site(s)

Based on the above discussions, clearly identify and decide in discussion with your host
- the sites and schools that you would like to visit in the time available
- the people you would like to meet and discuss with
  - the communities served
  - the people who will be running the projects, the teachers etc
  - the students (This ofcourse is the most interesting part of the exercise!!!)
- Other things you may wish to see like the library they might have created, the teaching materials, the attandance books, the system for keeping track of grades etc
Note carefully the following:
- the current state of affairs in the community
- try and put down numbers on the "metrics" defined above --- this will be the baseline status. This can be done in consultation with the teachers.
- the current status as far as items on the budget is concerned. For example, if somebody has asked for furniture, one should see what is the status of classroom at present --- are there tables? Are the needs justified? Verify some numbers like salaries if possible with the teachers directly.
Feel free to take notes, write down proper names etc
Try and take photographs which would serve the following purposes:
- Provide a "first-hand" feel to the Asha projects group gathering evaluating the project at the funding chapter.
- Can be used for publicity especially if the project is to be projected to donors for funding.

Step 4: Concluding discussions at Organization Office

This is the opportunity to:
- Clarify doubts and unanswered issues
- Collect or ask the organization to send certain required documents etc to you (if you need them). Some times the originating Asha chapter may have asked for a particular document.
- Inform the organization about the forward path and time lines to expect from Asha.

Step 5: Writing an Pre-funding Evaluation Visit Report

Please ensure that the report is written as early as possible after the visit while things are still fresh in your mind. The most important parts of the report are the personal impressions which you must document as early as possible.
Typical report should have the following categories
- General information
  - Organization name
  - Project name
  - Type of visit/ report (Is it an evaluation visit or review etc)
  - Visit goals (bulleted points in brief)
  - Visit date
  - Visit by whom
  - Contact persons and information
    - Organization
      - (and how to geth there)
    - Asha chapter originating process
    - Asha-Pune
- History, background and current status: Bulleted points on history of this organization (and this proposal) with Asha if any.
- About the organization
  - Summary about geographical area of operation, subject areas of activity, funding sources, key people etc (just refer bak to proposal wherever required)
  - Impressions/ observations gathered from visit, discussions, research etc
- About the area served and the community
  - Summary and personal impressions/ observations. Assessment of needs.
- About the proposed project and project proposal
  - Highlights of discussions and insights which came out as a result of your discussions.
  - Funds asked and budget proposed: Comments and suggestions
- Site visits and impressions
  - Desciption of each site visited with links to suitable images. Include personal impressions and data collected which will not be there in the proposal etc
- Recommendations on path forward.
- Appendix 1: Point-wise answers to any questions asked by the Asha chapter that originated evaluation of this project. Attach documents asked for.
- Appendix 2: Action points with time lines
  - For organization
  - For Asha-Pune
  - For other Asha chapter
- Appendix 3: Photographs

==================================================

Inset: Guidelines for carrying out Monitoring and Evaluation (from Dr Sandeep Pandey)

1. What is the motivation of the organization? Why are they working? Organizations can be classified in two categories – those that are working because money is available and others which during the course of their work need money and prepare a proposal. For example, an organization working because money is available will take up AIDS awareness programme even though the pressing need of the area might be to take up anti-malaria programme.

2. What is the backgound and commitment of people involved in the organization? Have they given sufficient thought to their work before preparing the proposal? Are they going to continue working on the issue even after the project is over? Or are they merely implementing the project like a contractor where the project begins when the funding arrives and is wound up when the funding dries up. Are they working only for salaries with ideological commitment taking a back seat?

3. What is the relationship between local people and the organization in organization’s area of work? What is the extent of involvement of local people in organization’s work? What opinion do the people of the area have about the organization? Do the local people contribute their time or effort towards the organization’s activities? Does the organization have a social support base in the area?

4. What is the administrative structure of the organization and how democratic is its functioning? How is the relationship between the main volunteers of the organization (or people who run the organization) and the grassroots level volunteers? How many women hold key positions or play important role in decision making within the organization’s hierarchy? Are the volunteers local or from outside? Do the main volunteers perform only administrative duties or actually spend time at the grassroots?

5. If the organization runs a school what is the socio-economic background of the children and how many girls attend?

6. Do the children continue their education after graduating from the school of the organization or drop out? If they drop out, how many of them do that and for what reasons? Does the organization give a thought to what will happen to the children after they leave it’s school?

7. What is the distance of other schools (Govt. or private) from the school of the organization and how do they run? Why do the children prefer the school run by the organization?

8. If it is a Non-Formal Education (NFE) Centre being run by the organization how many of the children attending this programme of the organization go to other formal schools in the area? (This question should be asked to the children and not the teachers or the volunteers of the organization.)

9. Which are the other funding agencies giving money to the organization and for what programmes? Ensure that the organization is not receiving funds from more than one agency for the same programme. How much of the infrastructure and other resources, like vehicles, are being used by the organization and how much of it lies underutilized?

10. Accounts related information. What is the salary/honorarium that the teachers get? Does it conform with the figures in the account books? What is the amount being spent on the children for snacks/food, clothes, etc., according to the budget and whether it conforms with the real situation?

11. Before going for monitoring and evaluation the documents concerned with the proposal and organization should be studied carefully.

12. Questions should also be asked of grassroots volunteers and local people outside the organization when the main volunteers (or people who run the organization) are not present to get a true picture of the state of affairs.

13. If the organization is running many centres and it may not be possible for you to visit all of them, visits should be made of centres randomly selected by you rather than letting the organization show you centres of its choice. Organizations are known to run their sample centres which they would prefer to take you to.