Non-Certified Programs
More than 90 percent of interventions we review do not receive Blueprints for Healthy Youth Development certification because they have not met the rigorous Blueprints criteria . To provide more detail on the reasons for non-certification, Blueprints uses evidence ratings for non-certified programs. The ratings consist of four main categories: ineffective, harmful, inconclusive, and insufficient. The internal staff typically assigns the ratings based on their review of a program, but in some cases the advisory board makes the final determination.
As illustrated below, the ratings are based on the strength of evidence for a program. The Reasons for Non-Certification page offers more detail on how the strength of the evidence is determined.
Blueprints uses one other quite different category for non-certification. Not dissemination ready refers to programs that meet Blueprints criteria for the strength of evidence but are not ready for adoption by users. See below for more on the importance of dissemination readiness.
– Interventions lacking strong supportive evidence are generally reviewed and rated internally by Blueprints staff.
– However, the
advisory board
may also conclude an intervention they have reviewed lacked sufficient evidence for certification despite having made it through the internal review process.
The rating for non-certified interventions is on a continuum. All interventions in our database are provided a rating based on the classification system below.
Not Dissemination Ready
Not Dissemination Ready
Several interventions meet Blueprints standards for intervention specificity, impact, and evaluation quality but fail to meet the dissemination readiness standard necessary for Blueprints certification .
We consider the following criteria when determining whether an intervention is “dissemination ready.” The intervention has the necessary:
– organizational capability,
– manuals,
– training,
– technical assistance, and
– other support required for implementation with fidelity in communities and public service systems.
If an intervention does not meet most or any of the criteria listed above but has a high-quality design, that intervention will receive a “not dissemination ready” rating – meaning it has met criteria for evaluation quality (as determined by the Blueprints Advisory Board ) but has not yet met the dissemination readiness criteria.
Rating system for non-certified interventions
Rating system for non-certified interventions
Below you will find the various classifications for how Blueprints rates non-certified interventions.
– Inconclusive Evidence
Studies of the intervention have one or more of the following limitations, including issues related to measurement or analysis, such as missing information on attrition, no tests of baseline equivalence, and/or only testing a very small or specialized sample. This rating is also given to programs for which there is only one quasi-experimental study that meets Blueprints evaluation quality standards (Blueprints certification requires one randomized control trial or two quasi-experimental design studies that meet Blueprints evaluation quality standards).
– Insufficient Evidence
Studies of the intervention have substantial limitations related to the evaluation design, measurement, or analysis, such as limited or no matching of the control group to the treatment group, no intent-to-treat analysis, incorrect level of analysis (i.e., not adjusting for clustering when the cluster was the unit of assignment), and/or no independently measured behavioral outcomes. Studies of a program rated as insufficient demonstrate one or more of these methodological limitations and may also include one or more limitations listed as inconclusive evidence.
– Ineffective
The evidence for this intervention meets all Blueprints evaluation quality standards but finds null (i.e., nonsignificant) effects on Blueprints outcomes.
– Harmful
Studies of the program meet all Blueprints evaluation criteria but find significant harmful effects on a Blueprints behavioral outcome.
Want to know why interventions fail to meet Blueprints standards? Learn more about the reasons programs are not certified.