Faculty job searches (2) – making the first cut

I appreciate all the interest my first post generated, as well as the fact that the supportive comments (many from people in considerably more prominent departments than mine) have so far outnumbered the doubters. I also want to acknowledge that there is no one-size-fits-all formula for how a search committee should work, what priorities a department should have in its hiring (e.g. whether they want to require new colleagues to come in with K awards), or how successful a new PI could be with any sort of arrangement. If your university/department is doing well, i.e. getting good papers out, getting funds enough to stay afloat and help temporary stragglers, and hiring exciting new colleagues who prosper, then I can’t say that our way of running a job search is better than yours. What I can say is that what I am describing has been typical of all the job searches my department has run since I got here (and I’ve been on 3 or 4 search committees previously), and I suspect it is similar to that of other basic science departments at my medical school, based on the characteristics of their hires.

Okay, on to the mechanics of the search. We had a relatively open-ended advertisement, not focusing on any specific area apart from research that would fall under the broad category indicated by our department’s name. There were four members of the search committee, representing relatively diverse areas spanning human disease, developmental biology, genomics and evolution: one relatively senior faculty member, who had been with the department for 20+ years; myself, who has been here ~10 years, another tenured member who moved here as an established PI a few years ago, and a more junior, tenure-track assistant professor. So, varying interests and varying levels of institutional memory.

We had ~160 applications, and before the committee met, I made a master Excel spreadsheet on which I listed a brief impression of every one. I didn’t read every application from front to back, initially – instead I did more or less what I do for NIH grant applications, when I first get the pile for study section, which is to look at the applicant’s publication list and their research statement (Specific Aims, for a grant). The two questions I had were: are they a good fit and are they productive (not do they have Cell, Nature or Science papers)? Despite what some commenters have stated, the only clear “filter” I applied was of fit – if someone had four CNS papers but they were all focused on microbial pathogenesis, I wrote them off.

To make the first cut, each committee member was assigned 80 applications such that each application was read in full by two members, and each committee member would have 40 applications shared by one other member and 40 by another. The goal was to whittle down the list to the point that we could all read each application very closely. This went surprisingly easily: in one meeting, we went from 160 applications to 27 (I will call this the “long-short list”), with no serious controversy.

Apart from fit, what were the criteria that made an application an easy (if often depressing) “no”? Although we didn’t filter postdocs for K99s or other K-type awards at this point, we definitely filtered established PIs based on their funding. If an Assistant Professor was trying to make a lateral move, but didn’t have an R01 that would last more than a year, there was no way we would consider them. And if the applicant had no first-author papers as a postdoc, even in press, they were not going to get a closer look. And finally, there is what I would call the “meh factor” – “meh” being a frequent comment of mine on applications that I found simply unexciting for one reason or another. This is probably the most difficult criterion to explain or justify, and it will vary from one search committee to another, but it basically comes down to the question of, if this person joins our department, are we going to be excited to hear what they are working on? Is it incremental, or me-too-ish, or is it actually something novel (to us, at least) with a lot of room for growth?

Aha, say the haterz, that’s where you are hiding your filter for Cell-Nature-Science papers!

Not so: I have plotted here the impact factor of the “best” paper (don’t get the vapors, PLoS-ONE true believers) that each of the 27 first-cut applicants had during their postdoc (or within last 5 years, for established PIs):

(Please forgive the ugly Excel formatting – I just realized that I don’t have R on my new laptop, and I didn’t want to wait while it is downloading.)

Three points: first, the papers published by our long-short list applicants cluster about equally between “super-elite” journals (CNS and spin-offs including Nature Genetics) and merely “elite” journals such as PNAS and Genome Research. This is what I meant in my first post – a PNAS paper can still get you an interview. Second, in red I have highlighted the final top eight applicants, including the five that we interviewed. We clearly had plenty of CNS to choose from, yet left more than half on the table. Third, there is clearly some relationship between super-elite publication and whether or not an applicant made subsequent cuts (6/15 of that group, vs. 2/12 of those with merely elite publications), but I personally believe that the later cuts were not based on the IF of any individual paper as much as on the “excitingness” of the research, a factor that can be, imperfectly, related to whether or not the CNS gods choose to smile on a particular topic.

As I said, the first cut went very easily, and I don’t think it would have varied by more than 2-3 applicants with a different committee makeup, a different funding climate, etc. Later rounds, which I will discuss later, were more contingent, and in a different universe many of these 27 applicants could have been invited. So I guess the take-home advice at this point, to go back to my first post, is that (a) CNS papers help, but one can succeed in the job search with a PNAS, an eLife or a PLoS Genetics; (b) K awards are relatively irrelevant as an independent variable – to be sure, very successful postdocs can get K awards, but this didn’t make much difference in our evaluation; (c) pedigree matters: of the 27 first-cut applicants, 8 came from labs of NAS members. There is a lot of tangled cause-and-effect there – in our first round, we certainly didn’t go into a lot of discussion of this PI vs that PI, but the “brand” of a high-profile advisor certainly helps. And of course, you don’t get into the National Academy without being good at producing high-profile papers. I will try to unpack pedigree in future posts, as well as talk about the more nitty-gritty arguments that went into selecting our short-short list of applicants.