During this phase of the evaluation process, decisions are made in selecting appropriate evaluation designs and methods to accomplish purposes and questions prioritized in the previous phase. These decisions involve a lot of operationalization, including translating stakeholder information which needs more specific evaluation questions. The determination of appropriate evaluation designs and methods, as well as strategies for sampling, data collection, and analysis, involves judgments about the nature of evaluation questions and, thus, a translation of those questions into associated design representations. Hence, a major form of validity affected by decisions at this stage is construct validity. The issues of construct validity arise when an idea or construct (i.e., program, an evaluation question, and outcome) is being transformed into a functional form that can be measured. At this stage, the key validity questions are:
In this chapter, a number of preconditions for construct validity are discussed, including the choice of appropriate study units or participants, representativeness of the unit construct, and representativeness of the intervention or evaluand. Assumptions arise whenever these preconditions are unexamined or taken for granted.
This is a preview of subscription content, log in via an institution to check access.
