ALTC Grants Scheme -
Evaluating Projects
 

4. What are the key evaluation questions which the evaluation will address?

  • What processes were planned and what were actually put in place for the project?
  • Were there any variations from the processes that were initially proposed, and if so, why?
  • How might the project be improved?
  • What were the observable short-term outcomes?
  • To what extent have the intended outcomes been achieved?
  • Were there any unintended outcomes?
  • What factors helped and hindered in the achievement of the outcomes?
  • What measures, if any, have been put in place to promote sustainability of the project's focus and outcomes?
  • What lessons have been learned from this project and how might these be of assistance to other institutions?
Project evaluation involves gathering information in order to understand and make judgments about the project and its outcomes. The types and extent of information that are needed will depend on the scope of the project (Section 1) and the purposes and scope of the evaluation (Section 2). The information needed will also depend on the specific object(s) of the evaluation, i.e. the particular element(s) or dimension(s) of the project that you wish to evaluate or are required to evaluate under a funding agreement1. Thus, for example, the evaluation may focus on the project's design, its implementation processes, its outcomes (short-term and/or longer-term), its impact, or a combination of these.

A further consideration in determining what information to gather relates to the context(s) of the project (Section 1). Gathering information on the nature and influence of the context(s) is critically important in evaluation as this will facilitate a deeper understanding and explanation of the particular outcomes that are achieved and of the factors that have enabled them to occur. This information will also inform predictions about impacts of the project.

It can be helpful initially to categorise the required information in terms of four or five broad areas, or four or five key questions to investigate. Depending on the types and extent of information that is needed, the key questions could include, for example, some of the following.

What processes were planned and what were actually put in place for the project?
Were there any variations from the processes that were initially proposed, and if so, why?
How might the project be improved?
What were the observable short-term outcomes?
To what extent have the intended outcomes been achieved?
Were there any unintended outcomes?
What factors helped and hindered in the achievement of the outcomes?
What measures, if any, have been put in place to promote sustainability of the project's focus and outcomes?
What lessons have been learned from this project and how might these be of assistance to other institutions?

These examples have been expressed in the past tense, implying that they would be asked at the end of the project. They should however also be expressed in the present tense and applied during the project's implementation as part of an ongoing evaluation. While the ongoing evaluation may lead to final summative conclusions and reporting, it may also enable evaluation information to be progressively fed back to the implementation team to assist it in monitoring the project and in adjusting or fine-tuning its operation. Each of the four or five key questions would in turn be broken down into a number of specific sub-questions as part of the more detailed data collection planning (Section 5).

1 Note: Current ALTC Grants Program Guidelines indicate that the final project reports should include, amongst other things, 'an analysis of the factors that were critical to the success of the approach and of factors that impeded its success' and 'an analysis of the extent to which the approach is amenable to implementation in a variety of institutions'.