Last week, I helped a terrific NGO apply to a funding program for innovative pilot projects in conservation and development. Their project might succeed; many others will fail to translate to any significant scale. Some will fail because innovation is not a factor that favors success. Novelty adds risk, uncertainty, and guesswork. Innovation usually fails, particularly when grant rules constrain project adaptation. Many of those projects will fail because the incentive begins in the wrong place, rewarding skillful grant applications rather than good project design. Some will achieve middling success at a small scale because the applicants repackaged previous projects to meet the donor’s innovation criteria when they needed help going to scale. Other projects will only succeed because the high ratio of expert staff to beneficiaries and high unit costs bear little resemblance to the large-scale governmental systems that deliver public services. In that sense, those projects are “too small to fail.” A small minority of projects may successfully prove an innovative solution to a persistent problem and simulate large scale. Such is the purgatory of pilot projects.
The pilot project purgatory originates in the confusion of ‘proof-of-concept’ and a pilot of large-scale implementation. I’ve seen countless projects claim success at the scale of hundreds of households, hectares, or schools that then crash upon the rocks of large-scale expansion. These demonstration projects didn’t test execution in the hands of under-resourced and over-worked agents. Non-profits don’t often entrust trials to the hands of the large-scale bureaucracy. The failure is that the supposed pilot projects neither simulate program delivery nor address its limits.
There are a number of solutions, primarily in the hands of donors. First, distinguish between small-scale proof-of-concept and piloting that genuinely models large-scale work. Second, provide a ladder of funding tiers from innovation to large-scale delivery and policy impact (e.g., Innovation Funding by the partners in the Fund for Innovation). Third, invest in local organizations that understand the scale barriers and have a long-term stake in success. Fourth, allow highly responsive work plan and budget flexibility when funding innovations. Fifth, offer funding to scale up prior successes rather than chasing the bright shiny hope of innovation. Sixth, promote the transfer of successful solutions by supporting projects that are not ‘new to the world’ and may only be innovative to the locality or a narrow domain. Seventh, fund experiments – true experiments. And, finally, donors can require that the partners responsible for scale have a voice and hand in any pilot project’s design and execution.
The designers and implementers of projects must also invest in local partners to mimic delivery at scale, rather than implementing experiments by ‘outsider’ organizations that are too small to fail. Grant applicants must have the courage to acknowledge when there is bad implementation of good strategies and try again. Grant partners must resist the temptation to repackage interventions as innovations to satisfy donor language yet obscure the richness of their own experience. The purgatory of pilot projects creates an illusion of innovation and progress while only unproductively turning the treadmill that we all wish to escape.
“Innovation is new stuff that is made useful.”
Max Mckeown, The Truth About Innovation (2008)