WRENFIELD — After four years and $2.1 million, the Institute for Applied Probability has completed a machine that predicts whether the institute’s grant proposals will be funded, a capability its directors say the institute has needed since 2019.
The device, which the institute calls the Reviewer, was trained on 1,400 applications submitted over 11 years, along with the reviewer comments, panel scores and eventual decisions attached to each. According to the institute, it correctly forecast the outcome of 91.4 percent of those submissions, though its designers note that the sample consists entirely of proposals the institute chose to submit.
“The instrument does not measure whether an idea is good,” said Dr. Miriam Solt, director of the institute’s forecasting group. “It measures whether the panel will fund it, which is a different and more practical question.”
Work on the device began in 2022, after a proposal to study the outcomes of grant competitions was rejected by two funders, according to the institute’s annual report. The institute spent $1.4 million on hardware and $700,000 on labeling the training data, a task performed by graduate students over three summers. The development office supplied the training data from its own files, including 240 proposals that had been rejected without reviewer comment. Those applications, Solt said, proved the most difficult to model, because the panel’s reasoning was recorded only as a score.
The grants office now runs each draft through the Reviewer before submission. According to the office, 30 percent of last year’s proposals were revised or withdrawn after the machine returned a negative forecast. Three other institutes have asked to test the device, and the first of those units is scheduled to ship in June.
The machine occupies a converted storage room in the institute’s basement, beside a decommissioned spectrometer. It requires 20 minutes to warm up and prints each forecast on a single sheet of paper. A proposal for a second unit, to be installed at the institute’s annex, is due to the Calloway Foundation in March.
