8-Step Scoping Framework Organizes the Pre-Evaluation Work That Often Gets Overlooked
The evaluation of complex interventions rarely begins with a clean trial question. In practice, the process starts much earlier, with a partner organization holding a mix of services, projects, and local priorities, and an academic team trying to determine which of those offerings can actually be evaluated with rigor. Davidson, Randhawa, and Cash describe an 8-Step Scoping Framework built to manage that formative stage, using work with a local government organization and its Flying Start provision as the applied example.
The framework is not a treatment model or a clinical endpoint study. It is a structured method for deciding what is ready for evaluation, what needs more development, and what can realistically support a process or outcome study. In settings where services are delivered across multiple programs, that distinction is critical because weakly defined interventions can consume time, ethics review resources, and staff effort long before any credible evidence is produced.
This is the point where many evaluations fail before they begin.
The framework extends prior work on evaluability assessment and pre-evaluation planning by making the early operational steps explicit. Rather than assuming that a program can simply be measured once a protocol is written, the authors map the work that comes first: defining the question, understanding the organization, checking the quality of available data, and deciding whether a project is sufficiently defined to move forward.
From Broad Service Portfolios to Specific Evaluation Questions
The first step is deceptively simple: formulate overarching key questions that apply across all programs offered by the organization. In a real-world service environment, that matters because a provider often delivers a portfolio rather than a single discrete intervention. Without a broad question at the outset, teams can drift into fragmented planning, where each project is handled in isolation and the logic linking local service delivery to evaluation disappears.
Step two requires an in-depth understanding of the organization and active engagement with staff. That is not a courtesy exercise. It is the mechanism by which external researchers learn how services are actually delivered, which staff roles matter, where data are generated, and which parts of the provision are sufficiently stable to be studied. The framework treats staff engagement as essential because the people delivering the service usually know where the documentation is thin, where delivery varies, and where a seemingly uniform program is actually several different models.
The third step is the completion of a data template for each project or program offered. This creates a common structure for comparing interventions across the portfolio. Once each service is captured in a standardized way, the team can begin to identify which programs have enough information to support evaluation and which are still too loosely characterized to proceed.
Data Quality Determines Which Projects Can Move Forward
Step four is where the framework becomes especially practical. Each program’s data are assessed for robustness and validity across the full range of provision. If the available information is incomplete, inconsistent, or too variable across sites or teams, the project may not yet be ready for formal evaluation. That is not a minor administrative issue; it affects the credibility of any future findings because the evaluation cannot be stronger than the underlying data infrastructure.
Step five uses that assessment to separate projects that are suitable for evaluation from those requiring additional data development. This sorting function is central to the framework. It prevents organizations from committing to studies that look feasible on paper but collapse once the evidence base is examined closely. It also protects staff from investing in protocols for interventions that still need a definitional or data-quality stage before evaluation can begin.
The process is not linear in a rigid sense. It is iterative, and the paper’s applied example reflects that reality. As the team worked through Flying Start provision, the framework helped distinguish between services ready for process and outcome evaluation and those needing further preparation.
Negotiating Roles, Protocols, and Ethical Approval
Once a project is selected, step six involves negotiating with the chosen project leads, both inside and outside the organization. This is where academic aims and service realities meet. Evaluation cannot proceed without agreement on roles, responsibilities, and the boundaries of the project. In community-based settings, that negotiation also shapes what can be measured, how data can be shared, and who will support implementation.
Step seven moves from scoping into protocol development for each individual project. At this stage, the intervention has been sufficiently defined to support a formal evaluation plan. The framework does not assume that one protocol can fit every service. Instead, it treats each selected project as a separate evaluation unit, which is more consistent with the way local provision is organized in the real world.
The final step is applying for ethical approval from both the university and the partner organization. That dual pathway reflects the partnership model underlying the framework. Community-based evaluations often require clearance from more than one governance structure, and early planning reduces the risk that a project reaches the ethics stage only to stall because its scope, data, or stakeholder commitments were never resolved.
Why the Framework Matters for Community-Based Evaluation
The value of the 8-step model lies in making evaluation readiness visible. Many intervention studies focus on outcomes once a project is already mature, but local services often need a more deliberate route from provision to assessable intervention. By laying out the pre-evaluation sequence, the framework gives academic partners and service organizations a shared language for deciding what can be studied and what still needs development.
The approach is especially relevant to evidence-based services designed to meet local needs, where programs may be responsive, layered, and adapted over time. In those settings, a narrow trial mindset can miss the operational realities that shape whether evaluation is possible at all. The framework directly addresses that gap by showing how to move from broad provision to a set of defined projects with enough structure for process and outcome evaluation.
The authors situate the work in early years provision, but the logic is broader. Any setting in which an academic partner works with external stakeholders to identify projects for formal evaluation can use the same sequence. Public health programs, social care initiatives, school-based services, and other community interventions often face the same problems: diffuse service definitions, incomplete data, and unclear ownership of the evaluation process.
The paper’s most concrete contribution is methodological rather than clinical. It turns a frequently under-described phase of research into an explicit framework, and it does so in a way that is usable for partnership work. What remains unresolved is not whether the framework can organize the pre-evaluation stage, but how widely it will be adopted across different service systems with different governance demands, data maturity, and organizational capacity.