8-Step Scoping Framework Maps the Path From Service Provision to Evaluation
Identifying which real-world programs are ready for formal evaluation is often harder than designing the evaluation itself. In a 2019 protocol published in JMIR Research Protocols, Davidson, Randhawa, and Cash set out an 8-Step Scoping Framework designed to help academic teams and partner organizations sort through multiple complex interventions and decide which ones can be evaluated with confidence.
The framework was built for the messy pre-evaluation phase, where services may already be running, data may be uneven, and local organizations want evidence without first having a perfectly standardized program. The paper places early-years provision, specifically Flying Start, at the center of the worked example, but the structure was intended to travel beyond one setting. In practice, the issue is not whether an intervention sounds worthwhile. The issue is whether it can be described clearly enough, measured reliably enough, and governed well enough to support process and outcome evaluation.
Why Pre-Evaluation Planning Often Breaks Down
Complex interventions rarely arrive in a form that is ready-made for research. They are usually embedded in service delivery, shaped by local staffing, changing priorities, and multiple funding streams. Evaluation teams may enter after programs have already accumulated variation across sites or project leads. Without a disciplined scoping stage, the result can be a protocol built around incomplete information, weak outcome definitions, or a program that cannot be separated cleanly from the wider service environment.
This framework responds to that gap. It extends the logic of evaluability assessment by moving upstream, before a formal evaluation design is finalized, and by making the partnership work itself part of the method. The emphasis is not only on whether a program is interesting or important, but whether it is sufficiently specified, documented, and supported by the organization delivering it.
The Eight Steps Begin With Questions, Not Assumptions
The first step is to formulate overarching key questions that cover all programs offered by the organization. This matters because a fragmented set of service descriptions can hide fundamental differences between projects that appear similar on paper. A broad question set creates a common frame for comparison and prevents evaluators from overcommitting to a single intervention too early.
The second step is to gain a detailed understanding of the organization’s work and provision while actively engaging staff. That engagement is not a courtesy add-on; it is how the evaluation team learns how services actually function. Frontline staff often hold the practical knowledge about implementation, feasibility, and local adaptations that never appear in formal documents.
A single program can look straightforward in a brochure and highly variable in delivery.
The third step is to complete a data template for each project or program. This creates a structured record across the full portfolio rather than a narrow description of the most visible intervention. The fourth step is to assess the robustness and validity of the data available across all programs. Here the framework forces a difficult but necessary question: which services have enough reliable information to support meaningful analysis, and which remain too thinly documented to evaluate well?
Sorting Programs by Readiness for Evaluation
The fifth step is where the framework becomes decisional. Teams determine which projects are suitable for evaluation and which require additional data collection before they can move forward. In a service environment, this is often the point at which enthusiasm meets evidence. Some interventions will be ready for immediate protocol development, while others may need better outcome definitions, clearer delivery records, or stronger linkage between activity and intended benefit.
The sixth step is negotiation with chosen project leads, both inside and outside the organization. This is a critical transition because evaluation depends on cooperation from those who control access to the intervention, the data, and the operational context. Negotiation here means aligning research aims with service priorities, resolving concerns about workload or governance, and settling which aspects of the program can realistically be studied.
The seventh step is the development of individual project evaluation protocols. Once a program clears the scoping process, the evaluation can be specified with greater precision. The framework is designed to make this stage more efficient by filtering out projects that are not yet ready and by clarifying the variables, stakeholders, and implementation realities that must be built into the protocol.
The eighth and final step is applying for ethical approval from both the university and the partner organization. In community-based research, this dual route is not bureaucratic excess. It reflects the fact that the project sits at the intersection of academic scrutiny and organizational responsibility, especially when children, families, or local service users are involved.
Flying Start as a Test Case for Community-Based Evaluation
The Flying Start example shows how the framework operates when an organization is delivering multiple overlapping services and wants to understand which elements merit formal study. The authors used that provision to test the process of identifying suitable interventions for process and outcome evaluation, turning what is often an informal judgment into a transparent sequence.
That sequence helps prevent two common failures in applied research: choosing an intervention before its structure is sufficiently understood, or delaying evaluation indefinitely because the service portfolio is too broad to sort through. The framework gives academic partners and service organizations a shared language for deciding what to study, what to postpone, and what needs more groundwork.
For local government settings, early-years programs, and other community services, the practical value lies in making the hidden labor of evaluation visible. Scoping, relationship-building, data appraisal, and protocol development are treated as methodologically serious work rather than administrative preliminaries. This is especially important when the aim is to build an evidence base for services designed around local need rather than standardized national delivery.
A Framework Built for Partnership Work, Not Just Method Papers
The paper’s central contribution is methodological, but its implications are operational. Many interventions that matter in public health and community care are delivered through external partnerships, not through tightly controlled research infrastructures. In those settings, the ability to identify evaluable projects depends on more than analytic skill. It depends on whether teams can understand provision across an organization, compare projects fairly, and decide when the available data are strong enough to justify a formal evaluation.
Because the framework was developed through iterative work alongside local government staff, it reflects the realities of service delivery rather than an idealized research environment. It is adaptable to other community-based evaluations and to situations where academic institutions work with external stakeholders to identify projects suitable for academic study. The article does not claim to solve every problem in complex intervention research. It does provide a clear scaffold for the stage that often determines whether an evaluation can proceed at all.