Audacious Goals in Vision Research
National Eye Institute (NEI)
Summary
NEI, part of the National Institutes of Health (NIH), recognized that recent technological breakthroughs were propelling paradigm shifts in biomedical research. Vision scientists are pioneering gene therapy, developing stem cell technologies and creating molecular therapies to prevent vision loss and restore sight. Recognizing that a concerted effort might lead to dramatic improvement in treating patients with vision loss, NEI wanted input from the research and patient community to set audacious goals for research. How could NEI alter the playing field to end blindness and catalyze new fields of science?
This Audacious Goals Initiative (AGI) ideation challenge offered a $3,000 prize for each winning submission (up to 20) for proposing audacious goals to stimulate innovation in vision research. To qualify as an audacious goal, a solver's one-page proposal needed to complete the thought: "It would be fantastic if…" Submissions needed to address barriers and next steps and were judged on five equally weighted criteria:
- Audaciousness: bold, transformative, innovative ideas
- Impact: broad implications across biomedical research
- Relevance to the NEI mission: to fundamentally change vision research or vision care
- Scope: beyond the output of a single investigator — collaboration across disciplines
- Feasibility: to be achievable in about 10 years
NEI Director Dr. Paul Sieving (first from left, front row) stands with 10 Audacious Goals competition winners. (Photo courtesy of NEI)
NEI took advantage of the authority in the America COMPETES Act to fund and run the competition. A simple challenge format was chosen to encourage the vision community to think differently and also to attract innovators in disciplines that don't typically participate in NEI research. NEI launched the three-month AGI challenge with great fanfare, advertising in the Federal Register, online scientific journals, social media and at medical conferences. It hosted informational webinars and reached out to various stakeholders through partner organizations and through its research network. NEI consulted an advertising firm to help place ads but ran the challenge through its own website. It created its own platform with a database that de-identified submissions.
Results
NEI received 476 entries in the competition from across the country, with an impressive diversity of occupations and career stages listed on the applications (providing occupation information was optional). Each submission could only have one author, but solvers were encouraged to submit multiple entries. Although the COMPETES Act and NIH rules prohibited NIH employees and foreign citizens from participating in the competition, NEI created a parallel submission process to solicit ideas from these stakeholders—while not eligible to win prize money, they were eligible to receive recognition. With the combined pools of submissions, NEI received a total of 548 ideas. Of those, 198 entries came from current or former NEI grantees, an additional 42 represented grantees from other NIH institutes, but over half of the ideas came from individuals new to NIH.
The review process had two phases. First, a group of clinicians and scientists outside of NIH with broad expertise in relevant disciplines evaluated the submissions against the stated criteria, narrowing the entries to 81 proposals. A federal judging panel then selected 10 challenge winners. The winners were invited to present their ideas to more than 200 cross-disciplinary experts at a 2013 NEI AGI development meeting. This challenge generated valuable contributions from varied stakeholders to inform NEI's strategic plan, energize its research efforts, increase public awareness of vision research and enhance the national effort to reduce the burden of ocular disorders and diseases worldwide.
Areas of Excellence
Area of Excellence #1: "Judge Submissions and Select Winners"
NEI employed a multi-tiered review process to select winners, with technical experts outside of NIH providing input to a federal judging panel. The large number of submissions, most received in the final days of the competition, forced NEI to adjust its initial review strategy and to recruit more technical experts. All entries were de-identified and initially screened by NEI staff; entries that unambiguously failed to satisfy judging criteria were eliminated. The remaining 474 submissions were categorized by primary, secondary and tertiary topics (such as regenerative medicine, neuroscience and healthcare) and sent for technical review by 81 external scientific experts, based on subject matter. NEI identified experts based on extensive medical literature searches and portfolio analysis of NEI grantees. Experts were required to sign a confidentiality agreement and were excluded from reviewing any submission which posed potential conflicts of interest through personal, professional or institutional connections. This first round of judging, which lasted two weeks, identified submissions worthy of more in-depth review. Experts were assigned submissions in their area of expertise and asked to provide a numerical score and technical comments relative to the judging criteria outlined on the Federal Register announcement. They were also invited to review the remaining entries and provide ordinal scores for meritorious submissions. This alternative scoring process allowed experts to single out creative solutions that might otherwise be discarded in a scoring process which relied on averaging results. Numerical scores from all the judges were normalized and combined to provide a rank order of submissions, of which the top 15 percent were included in the final review. The projects identified with the top ordinal scores were also included for review. A panel of senior NIH leaders reviewed the technical comments and scores from the top 15 percent of projects for two weeks. During a half-day face-to-face meeting, they selected 10 prize winners based on the de-identified ideas.
Area of Excellence #2: "Manage Solutions"
NEI invited the competition winners to present their ideas at the AGI Development Meeting in February 2013. In addition to prize awards, NEI paid up to $5,000 for participant travel expenses. This meeting brought together nearly 200 thinkers, leading biomedical researchers and clinicians to transform the winning ideas into a set of audacious goals. Over the course of three days, participants combined and fine-tuned the ideas in break-out groups and arrived at six proposals.
NEI leadership reviewed these proposals and ultimately selected a primary audacious goal: "To restore vision through regeneration of neurons or neural connections in the eye and visual system." Additionally, two other proposals became new NEI research programs: "Intersection of Aging and Eye Disease" and "Molecular Therapy at the Gene Level." Subsequent implementation of the AGI also has involved community input, with external steering committees, workshops and town hall meetings informing the development of funding initiatives. The initiative has impacted funding decisions, as projects related to the initiative can be designated for priority funding with high program relevance. Concerted strategic planning towards the AGI has led to the development of targeted funding opportunities. These goals recognize that many leading causes of blindness like age-related macular degeneration, diabetic retinopathy and glaucoma result from degeneration of neurons in the eye. However, the achievement of this goal will greatly influence all of neuroscience and the treatment of neurodegenerative diseases.
Challenge Type
Ideas
The NEI AGI challenge was well-suited to soliciting ideas from diverse stakeholders including patients, clinicians, engineers and information scientists, who provided inputs beyond the traditional pool of vision researchers.
Scientific
In addition to identifying unexplored areas of vision research, analysis of the overall patterns of ideas was a large factor in selecting the ultimate initiatives. For example, neurosciences and regenerative medicine were two of the most common themes repeated in the submissions, and both these concepts were incorporated into the audacious goal (see attached Word Cloud).
Legal Authority
America COMPETES Act