Evaluation objectives
The evaluation explored whether Gavi and its partners undertook the right activities to bring about the outcomes sought by the Strategy. The evaluation assessed three priority areas: design, implementation and results achieved, as well as the sustainability of the Strategy. Overall, the evaluation maintained a strategic perspective, as opposed to being a detailed monitoring exercise.
Methods
The evaluation used a retrospective and utilisation-focused approach. The evaluation methodology comprised a Theory of Change (ToC) based approach, with use of mixed methods including document review, data analysis, stakeholder consultations (global and country), key vaccine market analysis and counterfactual analysis.
The full list of planned country consultations was not conducted due to reduced stakeholder availability on account of COVID-19. However, this was not viewed as a limitation per se as a large number of interviews were conducted for the country perspective in relation to the scope and objectives of the evaluation.
Findings and conclusions
Overall, the Gavi Supply and Procurement Strategy 2016-20 has been very relevant, appropriate, and significant in the context of the evolving/maturing market shaping role of Gavi and the supply context. Building on the Alliance partners’ long-term work, Gavi has contributed to improved health across a number of vaccine markets. A number of markets have also seen improvements in supply security, though this has not been consistent. Despite these achievements, Gavi is likely to miss its overall target of achieving moderate to high levels of health across six vaccine markets. In addition, while the achievements related to healthier markets have been significant, more work is needed to ensure that such markets benefit from “true competition”. Some markets have also experienced significant price declines, particularly pentavalent, over the 2016-20 period, although the long-term impact of these price reductions on market health has divided opinion. Other markets have seen either prices remaining relatively stable or increasing, reflecting the challenging supply situation and also Gavi’s appropriate balancing of trade-offs beyond an exclusive price focus. More generally, activities to support innovation have worked well with several improved vaccine products receiving WHO Prequalification and being procured by the Alliance. At the global level, stakeholders have praised Gavi for its improved information sharing and coordination on market shaping.
Furthermore, recognising where Gavi markets are today as well as wider external developments, Gavi is well-positioned at this stage to consider the following for the 2021-2025 strategic period:
- There is a need to more actively consider demand-side issues in the 2021-2025 strategic period.
- Related to the above, and while considered of much value by the Secretariat, partners and broader community involved in market shaping, there are specific suggestions for an improved second iteration of the Healthy Market Framework (HMF) for the 2021-2025 strategic period.
- More widely, several vaccine and Cold Chain Equipment (CCE) markets continue to need active market shaping interventions to support improved health and sustainability over time.
For the evaluation recommendations and Gavi Alliance management response please see the full report and Evaluation Management Response.
