Monitoring and evaluation is a vital component that determines the effectiveness of a corporation's assistance by establishing clear links between past, present and future initiatives and results. The process helps in improving the programme performance and achieving desired results. It provides opportunities for fine-tuning, re-orientation and planning of the programme effectively, without which it becomes impossible to measure the success and impact of the programme even if the approach is right.
2. IndexMonitoring and Evaluation & it’s importance1
Monitoring and Evaluation framework in the Project Program Cycle02
Monitoring and the inclusion of RBM03
Types of Monitoring04
Tailor-Made Evaluation Studies06
Evaluation Questions & RBM08
Six key steps for M&E Planning09
Minimizing Bias and Errors10
CSR in India - Industry Overview11
Fiinovation Expertise12
Some of our Initiatives13
Organization Brief14
Awards & Recognition15
3. What Does Monitoring and Evaluation Mean?
Monitoring Evaluation
Why is Monitoring and Evaluation Important?
Organisations use monitoring and evaluation for two key purposes:
01
Does your project
have an M&E plan?
Does your project
have an M&E binder?
1
Have you developed
all monitoring forms?
3
Are monitoring data regularly used during
M&E meetings or other events?
8
Are staff and partners trained
on data entry and analysis?
7
Are staff and partners trained on
using the monitoring forms?
4
Have you created a
monitoring database?
6
Have you conducted a quality
check of the monitoring data?
5
If your answer is to any of the questions above, the project team should work to complete the‘No'
step(s). This review tool has not been designed to be used in isolation and refers to the package of
practices being offered at Fiinovation, and this handbook is for further guidance at each review stage.
To learn about their own activities and results, and to support internal planning and development
To be accountable to their stakeholders.
Monitoring is a routine and systematic
collection of information against a plan. The
information might be about activities, products
or services, users, or about outside factors
affecting the organisation or project.
Evaluation is about making judgements about the
value of any component part of an organisation
or its products, services or benets, or
about the organisation as a whole.
Organisations need evidence of their efciency and effectiveness for funders, commissioners and
investors. They also need to communicate achievements to a wider public. Charities are now
specically required to report the achievements of the charity over the year. However, the role of
monitoring and evaluation is providing learning and improving the organisation is of equal importance.
The following eight questions shall help in creating a better understanding on the need of
monitoring and evaluation:
2
Monitoring & Evaluation
4. M&E Framework in the Project
Program Cycle
02
Initial Needs
Assessment
Log frame and
indicators
M&E planning
Baseline
assessment
Midterm
evaluation
and/or reviews
Final
evaluation
Dissemination
and use of
lessons
This is done to determine whether a
project/program is needed and, if so, to
inform its planning.
This involves the operational
design of the project/program
and its objectives, indicators,
means of verification and
assumptions
This is the practical planning for
the project/program to monitor
and evaluate the log frame's
objectives and indicators.
This is the measurement of the
initial conditions (appropriate
indicators) before the start of a
project/program.These are important reflection events
to assess and inform ongoing
project/program implementation.
This occurs after project/program
completion to assess how well the
project/program achieved its
intended objectives and what
difference this has made.
This informs ongoing programming.
Although this has been put in the
last section of the diagram but,
reporting, reflection and learning
should occur throughout the whole
project/program cycle.
There is no magic 10-percent sampling rule. It is important to note that the sample size is not related to the size
of the popula on being sampled. A frequent mistake is to conduct surveys among 10 percent of a given
popula on; in fact, it is likely that 10 percent of the popula on is either too many or too few households. With
too many households, the survey is using excessive resources and me; with too few households, the sample
willnotadequatelyrepresentthepopula on.
Monitoring & Evaluation
5. Monitoring and the Inclusion of RBM
Impact
Ÿ Measuring changes at impact level requires a longer time frame, and is therefore dealt with
by evaluation and not monitoring.
Outcomes
Ÿ Are the outputs leading to the achievement of the outcomes?
Ÿ Is there anything happening that should lead management to modify the operations
implementation plan?
Output
Activities
Ÿ Are activities leading to expected outputs?
Ÿ Are activities being implemented on schedule and within the budget?
Inputs
Monitoring Questions and RBM
03
A mistake common for M&E systems is to rely solely on either observa on data or par cipant responses.
Observa on data alone does not provide an explana on of prac ces or behaviors and o en requires large
assump ons on the part of the M&E team. Focus group data (an example of par cipant responses) may not
capture important prac ces that par cipants do not see as relevant and may record instead what par cipants
thinkdatacollec onteamswanttohear.
Result Based Management (RBM) is an approach to project/program management based on
clearly dened results, and the methodologies and tools to measure and achieve them.
RBM supports better performance and greater accountability by applying a clear, logical
framework to plan, manage and measure an intervention with a focus on the results you want to
achieve.
By identifying in advance the intended results of a project/program and how we can measure their
progress, we can better manage a project/program and determine whether a difference has
genuinely been made for the people concerned.
Monitoring and evaluation (M&E) is a critical part of RBM. It forms the basis for clear and
accurate reporting on the results achieved by an intervention (projector program).
In this way, information reporting is no longer a headache, but becomes an opportunity for critical
analysis and organizational learning, informing decision-making and impact assessment.
Ÿ How do beneciaries feel about the work/intervention?
Ÿ What is causing delays or unexpected results?
Ÿ Are nances, personnel and materials available on time and in the right quantities and
quality?
Monitoring & Evaluation
6. It tracks the use of inputs and resources, the progress of activities and the delivery of outputs.
It examines how activities are delivered – the efciency in time and resources.
It is often conducted in conjunction with compliance monitoring and feeds into the evaluation of
impact.
For example, a water and sanitation project may monitor that targeted households receive septic
systems according to schedule.
Types of Monitoring
It tracks effects and impacts.
This is where monitoring merges with evaluation to determine if the project/program is on target
towards its intended results (outputs, outcomes, impact).
It measures whether there may be any unintended impact (positive or negative).
For example, a psychosocial project may monitor that its community activities achieve the outputs
that contribute to community resilience and ability to recover from a disaster.
1. Results Monitoring
2. Process (Activity) Monitoring
3. Compliance Monitoring
4. Context (Situation) Monitoring
04
It ensures compliance with donor regulations and expected results, grant and contract
requirements, local governmental regulations and laws, and ethical standards.
For example, a shelter project may monitor that shelters adhere to agreed national and
international safety standards in construction.
It tracks the , especially as it affects identied riskssetting in which the project/program operates
and assumptions, but also any unexpected considerations that may arise.
It includes the eld as well as the larger political, institutional, funding, and policy context that
affect the project/program.
For example, a project in a conict-prone area may monitor potential ghting that could not only
affect project’s success but, endanger the project’s staff and volunteers.
Monitoring & Evaluation
7. It tracks beneciary perceptions of a project/program.
It includes beneciary satisfaction or complaints with the project/program, including their
participation, treatment, access to resources and their overall experience of change.
Sometimes referred to as it often includes a stakeholderBeneciary Contact Monitoring (BCM),
complaints and feedback mechanism.
It should take account of different population groups, as well as the perceptions of indirect
beneciaries (e.g. community members not directly receiving a good or service).
For example, a cash-for work program assisting community members after a natural disaster may
monitor how they feel about the selection of program participants, the payment of participants and
the contribution the program is making to the community (e.g. are these equitable?).
6. Financial Monitoring
05
5. Beneficiary Monitoring
It tracks the sustainability, institutional development and capacity building in the project/program
and with its partners.
It is often done in conjunction with the monitoring processes of the larger, implementing
organization.
For example, a National Society's headquarters may use organizational monitoring to track
communication and collaboration in project implementation among its branches and chapters.
7. Organizational Monitoring
It accounts for costs by input and activity within predened categories of expenditure.
It is often conducted in conjunction with compliance and process monitoring.
For example, a livelihood project implementing a series of micro-enterprises may monitor the
money awarded and repaid, and ensure implementation is according to the budget and time frame.
Monitoring & Evaluation
8. Tailor-made Evaluation Studies
Evaluations conducted during the life of the project should provide actionable recommendations to
improve the quality of the project in the time remaining. The final evaluation findings must be
incorporated into the strategy design for subsequent programming. The evaluation exercises can be
categorized under three broad heads as enumerated below.
According to evaluation timing According to who conducts the
evaluation
According to evaluation
technicality or methodology
Formative evaluations occur
d u r i n g p r o j e c t / p r o g r a m
implementation to improve
performance and assess
compliance.
Internal or self-evaluations are
conducted by those responsible
f o r i m p l e m e n t i n g a
project/program. They can be
less expensive than external
evaluations and help build staff
capacity and ownership.
H o w e v e r, t h e y m a y l a c k
c r e d i b i l i t y w i t h c e r t a i n
stakeholders, such as donors,
as they are perceived as more
subjective (biased or one-sided).
These tend to be focused on
learning lessons rather than
demonstrating accountability.
Real-time evaluations (RTEs)
a r e u n d e r t a k e n d u r i n g
p r o j e c t / p r o g r a m
implementation to provide
i m m e d i a t e f e e d b a c k f o r
modifications to improve
ongoing implementation.
Emphasis is on immediate
lesson learning over impact
evaluation or accountability.
Summative evaluations occur at
the end of project/program
implementation to assess
effectiveness and impact.
External or independent
evaluations are conducted by
evaluator(s) outside of the
implementing team, lending it a
degree of objectivity and often
technical expertise. These tend
to focus on accountability.
Meta-evaluations are used to
assess the evaluation process
itself. Some key uses of meta-
evaluations include: take
inventory of evaluations to
inform the selection of future
evaluations; combine evaluation
results; check compliance with
evaluation policy and good
practices; assess how well
evaluations are disseminated
and utilized for organizational
learning and change, etc.
Midterm evaluations are
formative in purpose and
o c c u r m i d w a y t h r o u g h
implementation.
Participatory evaluations are
c o n d u c t e d w i t h t h e
beneficiaries and other key
stakeholders, and can be
empowering, building their
capacity, ownership and
support.
Thematic evaluations focus on
one theme, such as gender or
environment, typically across a
number of projects, programs
or the whole organization.
06Monitoring & Evaluation
9. According to evaluation timing According to who conducts the
evaluation
According to evaluation
technicality or methodology
Final evaluations are summative
in purpose and are conducted
(often externally) at the
completion of project/program
implementation to assess how
well the project/ program
a c h i e v e d i t s i n t e n d e d
objectives.
Joint evaluations are conducted
collaboratively by more than one
implementing partner, and can
help build consensus at
different levels, credibility and
joint support.
Cluster/sector evaluations
focus on a set of related
a c t i v i t i e s , p r o j e c t s o r
programs, typically across
sites and implemented by
multiple organizations (e.g.
National Societies, the United
Nations and NGOs).
E x - p o s t e v a l u a t i o n s a r e
conducted some time after
implementation to assess long-
term impact and sustainability.
Impact evaluations focus on the
effect of a project/ program,
rather than on its management
and delivery. Therefore, they
t y p i c a l l y o c c u r a f t e r
project/program completion
during a final evaluation or an ex-
post evaluation. However,
impact may be measured during
p r o j e c t / p r o g r a m
implementation during longer
projects/programs and when
feasible. It highlights some of
the challenges in measuring
impact.
Proper management of an evaluation is a critical element for its success. There are multiple resources
to support evaluation management which identify the key criteria and standards that guide how we
plan, commission, conduct, report on and utilize evaluations. The framework mentioned in the
beginning is to be applied to all evaluation activities. It draws upon the best practices to ensure
accurate and reliable evaluations that are credible with stakeholders.
07Monitoring & Evaluation
10. Evaluation Questions and RBM
08
Impact
Outcomes
Output
Ÿ Were the operations objective achieved? (effectiveness)
Ÿ Did the outputs lead to intended outcome?
Activities
Inputs
Ÿ Are the benets likely to be maintained for an extended period after assistance
ends? (sustainability)
Ÿ What changes did the project bring about?
Ÿ Were the operation’s objectives consistent with beneciaries’ needs and with
agency policies? (relevance)
Ÿ Were there any unplanned or unintended changes.?
Ÿ Were activities implemented on schedule and within budget? (efciency)
Ÿ Were outputs delivered economically?
Ÿ Were stocks of items available on time and in the right quantities and quality?
Monitoring & Evaluation
11. Six Key Steps for M&E Planning
Review the project/program's operational design (RBM/LFA)
Identify key stakeholder informational needs and expectations
Identify any M&E requirements
Scope of major M&E events and functions
Identify the purpose
and scope of the M&E
system
1.
Develop an M&E plan table
Assess the availability of secondary data
Determine the balance of quantitative and qualitative data
Triangulate data collection sources and methods
Determine sampling requirements
Prepare for any surveys
Prepare specic data collection methods/tools
Establish stakeholder complaints and feedback mechanisms
Establish project/program staff/volunteer review mechanisms
Plan for data management
Use an indicator tracking table (ITT)
Use a risk log (table)
Plan for data collection
and management
2.
Develop a data analysis plan, identifying the:
i. Purpose of data analysis
ii. Frequency of data analysis
iii. Responsibility for data analysis
iv. Process for data analysis.
Follow the key data analysis stages:
i. Data preparation
ii. Data analysis (ndings and conclusions)
iii. Data validation
iv. Data presentation
v. Recommendations and action planning.
Plan for data analysis
3.
Assess the projects/program's human resources capacity for M&E
Determine the extent of local participation
Determine the extent of outside expertise
Dene roles and responsibilities at each level of the M&E system.
Plan to manage project/program team's M&E activities
Identify M&E capacity-building requirements and opportunities
Plan for M&E human
resources and capacity
building
5.
Itemize M&E budget needs
i.Human Resource
ii.Capital Expenses
Incorporate M&E costs into the project/program budget
Review any donor budget requirements and contributions
Plan for cost contingency
Prepare the M&E
budget
6.
Anticipate and plan for reporting:
i. Needs/audience
ii. Frequency
iii. Formats
iv. People responsible.
Plan for information utilization:
i. Information dissemination
ii. Decision-making and planning
Plan for information
reporting and utilization
4.
09Monitoring & Evaluation
12. Minimizing Bias and Errors
Minimizing bias helps to increase accuracy and precision. Accuracy means that the data measures
what it is intended to measure. For example, if you are trying to measure knowledge change following a
training session, you would not just measure how many people were trained but also include some type
of test of any knowledge change.
As much as we would like to eliminate bias and error in our measurements and information reporting,
no research is completely without bias. Nevertheless, there are precautions that can be taken, and
the first is to we encounter in our work:be familiar with the major types of bias
1. Selection Bias
Selection bias results from poor selection of the sample population to measure/study. Also called
design bias or sample error, it occurs when the people, place or time period measured is not
representative of the larger population or condition being studied. It is a very important concept to
understand because there is a tendency to study the most successful and/or convenient sites or
populations to reach (which are often the same). For example, if data collection is done during a
convenient time of the day, during the dry season or targets communities easily accessible near paved
roads, it may not accurately represent the conditions being studied for the whole population.
2. Measurement Bias
Measurement bias results from poor data measurement – either due to a fault in the data
measurement instrument or the data collector. Sometimes the direct measurement may be done
incorrectly, or the attitudes of the interviewer may influence how questions are asked and responses
are recorded. For instance, household occupancy in a disaster response operation may be calculated
incorrectly, or survey questions may be written in a way that biases the response, e.g. “Why do you like
this project?” (rather than “What do you think of this project?”).
3. Processing Error
Processing error results from the poor management of data – miscoded data, incorrect data entry,
incorrect computer programming and inadequate checking. This source of error is particularly
common with the entry of quantitative (statistical) data, for which specific practices and checks have
been developed.
4. Analytical Bias
Analytical bias results from the poor analysis of collected data. Different approaches to data analysis
generate varying results e.g. the statistical methods employed, or how the data is separated and
interpreted. A good practice to help reduce analytical bias is to carefully identify the rationale for the
data analysis methods.
10Monitoring & Evaluation
13. 1
CSR Expenditure Among 10 Major Industries
HEALTH
22%RURAL DEVELOPMENT
33%
EDUCATION
23%
ENVIRONMENT
22%
2012-13
Expense on CSR in the Private and Public Sector
20 400 60 80 100
Slum Area Developent
RD Projects
Technical Incubators
PM Relief Fund
Sports
Forced Army Veterans
National Heritage
Environmental Sustainability
Reducing Inequalities
Education, Vocational Skills
Hunger, Poverty, Healthcare
95
5
65
35
18
82
85
15
30
70
95
5
58
42
42
58
63
37
75
25
22
78
PrivateSectorPublicSector
1
As per the annual reports of the top 200 firms for the year 2012-13
2
FICCI and Nextgen
CSR in India - Industry Overview
11Monitoring & Evaluation
14. Fiinovation Expertise
A vast bank of expertise gained over the years across a wide range of sectors and research
methodology. And also, the flexibility to serve a wide range of projects in the sectors of: Livelihood,
Education, Health and Environment.
Defined M&E standards that help improve program quality and positively impact the people we
serve. The standards followed at Fiinovation reflect the key characteristics of high-quality
programs and agency culture that promotes better learning and strengthens accountability to
stakeholders. These are critical elements of a high- performing, dynamic learning organization.
Triangulation of data is another key step practiced at Fiinovation, which involves collecting data
from multiple sources, sometimes using multiple tools, to identify and reduce bias. If triangulation
of qualitative data is not carried out, a risk of biasing or distorting the collected data exists,
resulting in incorrect or incomplete information which might result in skewed understanding of the
circumstances.
By collecting data from multiple sources or with multiple tools, we identify and address
discrepancies or inconsistencies in the data. Triangulation often leads to additional questions or
clarifications, which can be answered through follow-up interviews, discussions or exercises.
Core
Values
We ensure excellent quality of
the actionable reports which is
intended to provide absolute and
detailed insight into the
markets, products, competition
and the overall perspective at an
affordable price.
We ensure secrecy for all the
research projects that we
c a r ry o u t a n d m a i n t a i n
confidentiality as per the code
of conduct.
We guarantee authenticity,
reliability and credibility of the
data, facts and figures that we
gather and disseminate
through primary and secondary
research.
Authenticity
Quality
Confidentiality
12Monitoring & Evaluation
15. 13Monitoring & Evaluation
The Monitoring and Evaluation Team of Fiinovation has been involved in helping its partners to
carry out eld research studies for monitoring, evaluating and assessing the impact of their
proposed and ongoing/complete projects.
A few of our successfully completed assignments in the current year are mentioned below:
Fiinovation partnered with IL&FS Skills to carry out an integrated on-site assessment study
for assessing the current status of the community in terms of their readiness and
acceptability for skill based trainings. The study was carried out to explore the avenues that
shall be available at the proposed site for Power Plant in Nana Layja (Mandvi), Gujarat. Based
on the research ndings of Fiinovation, the project aims to develop skill, employment and
entrepreneurial opportunities for the population of Mandvi Block in Kutch district with
respect to the Nana Lajya Power Plant being put up by IL&FS Group. Fiinovation gathered
adequate information through extensive research, both primary and secondary sources for
the purpose of the study. The Need Assessment of the project was carried out for 3 months
with assessment tools intrinsically focused on the need and acceptance of the community.
Fiinovation has also conducted a summative evaluation study of the skill training institute
(Cairn Enterprise Center - CEC) of Cairn India Limited (CIL) at Barmer (Rajasthan). The study
was carried out for the CSR project of CIL being implemented by IL&FS Skills with a view to
evaluate the program performance and its effectiveness by collecting information that can be
used for further evaluation. Fiinovation developed tailor-made evaluation parameters for
assessing the outcomes of the skill training intervention. The framework used empirical
evaluations aimed at elucidating and substantiating the linkages between the trainings
provided and the intended or observed benets. The project required unbigoted opinion along
with technical expertise which was provided by Fiinovation for assessment and
recommendations for future course of action.
With its team of experts, Fiinovation has successfully conducted evaluation and assessment
of Adult Literacy Centers (ALC) run by M/s JK Tyre & Industries Ltd. at Kankroli (Rajasthan),
Morena( Madhya Pradesh) and Mysore( Karnataka). For this purpose, a detailed research
was carried out to assess the efcacy and effectiveness of the ALCs.
Some of our
Initiatives
16. Afliations
Established in 2008, Fiinovation (Innovative Financial Advisors Pvt.
Ltd.) has created a niche as a multi disciplinary research
organization in Corporate Social Responsibility (CSR) and
Sustainability domain. Headquartered in New Delhi, it has been
forging partnerships between credible NGOs and corporations that
are looking to implement and initiate community based interventions
and initiatives. With a pool of over 4000 credible NGOs spread
across the country, the motive has been to create awareness,
inspire innovative thinking and action with sustainable development
at its core. These NGOs are empanelled post a thorough and
stringent due diligence check through secondary research and on
ground evaluation.
Fiinovation is known in the development sector for enhancing quality
across organizational value chain through its in-house Proposal
Design & Research Laboratory for designing innovative CSR
initiatives. Fiinovation also supports corporations through
conducting need assessment, monitoring and evaluation as well as
impact assessment of their programs.
The distinct practices of Fiinovation include CSR Portfolio
Management (CPM), CSR-CSO Partnership, Initiative Design,
Initiative Management, Monitoring & Evaluation, Impact
Assessment and CSR audit across verticals of health, environment,
education and livelihood, which contribute to INR 15 crores in the
top line of nancial statement.
Fiinovation has been a knowledge partner with various industrial bodies such as Confederation of
Indian Industries (CII) and Institute of Directors (IOD). With CII, the partnership emphasizes on
research for promotion of afrmative action along with CSR and social enterprises. The organization
has deliberated with various industrial leaders at numerous conferences conducted on corporate
governance, environment management and CSR & sustainability during its path breaking partnership
with IOD. Being a research organisation, Fiinovation also contributes to IOD's Master class for
Directors that has a wide spectrum of subjects such as corporate governance, sustainability, HR
practices, nance, corporate social responsibility, environment management and board
effectiveness. Other than this, the organization has been closely associated with other forums such
as NGO BOX and World CSR Congress.
Organization Brief
14Monitoring & Evaluation
17. Awards & Recognitions
Mr. S. Chakraborty along with Smt. Sheila Dixit at the
24th World Congress on Total Quality and Leadership
15
Fiinovation has been conferred with the “CSR Team of the Year” &
“Caring Company Award” at the World CSR Day 2015. It was also the
recipient of the 'Best Social Innovation: Product or Service' at the World
CSR Day 2014. Apart from this, the company was bestowed with the
"Manager of the Year" & "Best Enterprise of the Year - in the eld of
Health, Education, Environment and Livelihood with a focus on CSR &
Sustainability" Awards by the European Business Assembly at the
Socrates Award Ceremony, Summit of Leaders in 2014 at Oxford Town
Hall, United Kingdom.
Mr. Soumitro Chakraborty addressing the gathering
at CII conference - Bridging the Gap: Fostering
growth through Education & Entrepreneurship
Monitoring & Evaluation
18.
19. Media Links
fiinovation
24/30, Ground Floor, Okhla Industrial Estate, Phase III, New Delhi - 110020, India
Phone: 011-42332200 | Fax : 011-42332205
Email : info@inovation.co.in, | Website : www.inovation.co.in
