Plots of grades vs. who gets selected

They say that you're in the selectable category, but ever wondered what, really, are your chances of being selected? Historically, the Science Mission Directorate has not reported on selection rates as a function of peer reviewed scores. We are very pleased to share this data with you!

The first plot, below, shows the distribution of grades (along the x axis) vs. how many were selected, or not. The upward blueish bars (and positive values on the y axis) represent selected proposals, and the downward redish bars (and the negative numbers on the y axis) represent the proposals that were not selected.

The format of the figures is described in the text on the web page. In addition, the numbers from which the prior charts were derived has been attached in tabular format. In the table all of the numbers (both # of proposals selected and not selected) are positive, which makes logical sense. The numbers of not selected proposals in the first graphic are associated with negative values on the y axis only because they are represented by downward bars. Its just an artifact of the dumb program I used to make the

A more recent example of this kind of data set for the planetary science division was presented by Jonathan Rall to the Planetary Science Subcommitee in Fall 2017. Please see page 33 of this presentation at:

https://smd-prod.s3.amazonaws.com/science-red/s3fs-public/atoms/files/PSD-PSS_Sept_2016.pdf#page=33

The second plot, below, shows the same data but recast as percentages, so that each vertical bar now shows the percentage chance, overall, that a proposal with the grade along the x axis will be selected. For example, a proposal with a grade of 'excellent' was (at least partially) funded 94% of the time, with an 'excellent/very good' 80%, 'very good' 45%, and so on.

The data from which these plots were derived were based on a sample of 1,597 proposals (20 ROSES program elements) reviewed in 2007 and 2008 for which data was readily available. They are believed to be representative of the approximately 4000 proposals reviewed in that time frame. There are considerable sources of error including the fact that the data across divisions, reviews and in some cases even panels were not normalized in any way. When only one task within a proposal was funded, some program officers reported the overall (lower) grade for the proposal, whereas others reported the (higher) grade for just the funded task. The author makes no apologies, he is just happy to be able to share what data he has.

Please direct questions or corrections on this page to SARA@nasa.gov

Keep Exploring

Discover More Topics From NASA

James Webb Space Telescope

The image is divided horizontally by an undulating line between a cloudscape forming a nebula along the bottom portion and a comparatively clear upper portion. Speckled across both portions is a starfield, showing innumerable stars of many sizes. The smallest of these are small, distant, and faint points of light. The largest of these appear larger, closer, brighter, and more fully resolved with 8-point diffraction spikes. The upper portion of the image is blueish, and has wispy translucent cloud-like streaks rising from the nebula below. The orangish cloudy formation in the bottom half varies in density and ranges from translucent to opaque. The stars vary in color, the majority of which have a blue or orange hue. The cloud-like structure of the nebula contains ridges, peaks, and valleys – an appearance very similar to a mountain range. Three long diffraction spikes from the top right edge of the image suggest the presence of a large star just out of view.

Perseverance Rover

Parker Solar Probe

Juno