Faculty job searches (2) – making the first cut

I appreciate all the interest my first post generated, as well as the fact that the supportive comments (many from people in considerably more prominent departments than mine) have so far outnumbered the doubters. I also want to acknowledge that there is no one-size-fits-all formula for how a search committee should work, what priorities a department should have in its hiring (e.g. whether they want to require new colleagues to come in with K awards), or how successful a new PI could be with any sort of arrangement. If your university/department is doing well, i.e. getting good papers out, getting funds enough to stay afloat and help temporary stragglers, and hiring exciting new colleagues who prosper, then I can’t say that our way of running a job search is better than yours. What I can say is that what I am describing has been typical of all the job searches my department has run since I got here (and I’ve been on 3 or 4 search committees previously), and I suspect it is similar to that of other basic science departments at my medical school, based on the characteristics of their hires.

Okay, on to the mechanics of the search. We had a relatively open-ended advertisement, not focusing on any specific area apart from research that would fall under the broad category indicated by our department’s name. There were four members of the search committee, representing relatively diverse areas spanning human disease, developmental biology, genomics and evolution: one relatively senior faculty member, who had been with the department for 20+ years; myself, who has been here ~10 years, another tenured member who moved here as an established PI a few years ago, and a more junior, tenure-track assistant professor. So, varying interests and varying levels of institutional memory.

We had ~160 applications, and before the committee met, I made a master Excel spreadsheet on which I listed a brief impression of every one. I didn’t read every application from front to back, initially – instead I did more or less what I do for NIH grant applications, when I first get the pile for study section, which is to look at the applicant’s publication list and their research statement (Specific Aims, for a grant). The two questions I had were: are they a good fit and are they productive (not do they have Cell, Nature or Science papers)? Despite what some commenters have stated, the only clear “filter” I applied was of fit – if someone had four CNS papers but they were all focused on microbial pathogenesis, I wrote them off.

To make the first cut, each committee member was assigned 80 applications such that each application was read in full by two members, and each committee member would have 40 applications shared by one other member and 40 by another. The goal was to whittle down the list to the point that we could all read each application very closely. This went surprisingly easily: in one meeting, we went from 160 applications to 27 (I will call this the “long-short list”), with no serious controversy.

Apart from fit, what were the criteria that made an application an easy (if often depressing) “no”? Although we didn’t filter postdocs for K99s or other K-type awards at this point, we definitely filtered established PIs based on their funding. If an Assistant Professor was trying to make a lateral move, but didn’t have an R01 that would last more than a year, there was no way we would consider them. And if the applicant had no first-author papers as a postdoc, even in press, they were not going to get a closer look. And finally, there is what I would call the “meh factor” – “meh” being a frequent comment of mine on applications that I found simply unexciting for one reason or another. This is probably the most difficult criterion to explain or justify, and it will vary from one search committee to another, but it basically comes down to the question of, if this person joins our department, are we going to be excited to hear what they are working on? Is it incremental, or me-too-ish, or is it actually something novel (to us, at least) with a lot of room for growth?

Aha, say the haterz, that’s where you are hiding your filter for Cell-Nature-Science papers!

Not so: I have plotted here the impact factor of the “best” paper (don’t get the vapors, PLoS-ONE true believers) that each of the 27 first-cut applicants had during their postdoc (or within last 5 years, for established PIs):

image007

(Please forgive the ugly Excel formatting – I just realized that I don’t have R on my new laptop, and I didn’t want to wait while it is downloading.)

Three points: first, the papers published by our long-short list applicants cluster about equally between “super-elite” journals (CNS and spin-offs including Nature Genetics) and merely “elite” journals such as PNAS and Genome Research. This is what I meant in my first post – a PNAS paper can still get you an interview. Second, in red I have highlighted the final top eight applicants, including the five that we interviewed. We clearly had plenty of CNS to choose from, yet left more than half on the table. Third, there is clearly some relationship between super-elite publication and whether or not an applicant made subsequent cuts (6/15 of that group, vs. 2/12 of those with merely elite publications), but I personally believe that the later cuts were not based on the IF of any individual paper as much as on the “excitingness” of the research, a factor that can be, imperfectly, related to whether or not the CNS gods choose to smile on a particular topic.

As I said, the first cut went very easily, and I don’t think it would have varied by more than 2-3 applicants with a different committee makeup, a different funding climate, etc. Later rounds, which I will discuss later, were more contingent, and in a different universe many of these 27 applicants could have been invited. So I guess the take-home advice at this point, to go back to my first post, is that (a) CNS papers help, but one can succeed in the job search with a PNAS, an eLife or a PLoS Genetics; (b) K awards are relatively irrelevant as an independent variable – to be sure, very successful postdocs can get K awards, but this didn’t make much difference in our evaluation; (c) pedigree matters: of the 27 first-cut applicants, 8 came from labs of NAS members. There is a lot of tangled cause-and-effect there – in our first round, we certainly didn’t go into a lot of discussion of this PI vs that PI, but the “brand” of a high-profile advisor certainly helps. And of course, you don’t get into the National Academy without being good at producing high-profile papers. I will try to unpack pedigree in future posts, as well as talk about the more nitty-gritty arguments that went into selecting our short-short list of applicants.

Advertisements

21 thoughts on “Faculty job searches (2) – making the first cut

  1. This basically jives with my experience over the past decade or so and you’ve nicely summarized the procedure as I’ve experienced it on a large number of search committees at this public and a prior private medical school.

    Just a bit more granularity regarding NAS membership of mentors–there are also what I like to refer to as “NAS-to-be” (or “potential Noble lab” or some moniker). Essentially these folks trained in outstanding labs (either established or up and coming) and have strong letters from their mentors.

    So you’re right that the pedigree ‘matters’ and seems to count for a lot of cred when it comes to winnowing down from dozens of highly outstanding applicants to the five or so who make the final cut for the on-site visit.

    My experience, though, is that this metric is a surprisingly weak predictor of ultimate success in getting hired as that comes to how the actual visit with the faculty/seminar/chalk talk go.

  2. Thanks for sharing your experience. It is a real eye opener for people like me on the job market. Where do you stand on mentors who write letters for 2 of their trainees applying to the same job opening? Will this discourage the committee from taking any of the applicants seriously?

    • As it happens, we had two people from the same lab on our “long-short” list of 27. We only interviewed one, but I don’t recall that we made that decision along the lines of, “we need to pick one from this lab.” It was more that we wanted a diverse range of research interests, and unsurprisingly these two fell into the same general category (along with several from other labs). The mentor’s letters didn’t betray a preference for one over the other, and we didn’t have to go to the lengths of contacting them to ask who they thought was “better.”

      I went on the job market simultaneous with another postdoc from my lab. We decided to just apply for whatever jobs seemed like good fits, not worrying about overlap, and I think in the end only one place interviewed us both. I think that is a relatively common situation for postdocs coming from large, postdoc-heavy labs, and I don’t think it has any negative effect on the applicants.

  3. Seriously, thank you for writing this. THANK YOU. As a postdoc looking at going into this process in a couple years this kind of information (real data instead of just platitudes and generalities) is invaluable. Can’t wait for the further posts. Thank you.

  4. I was struck by the fact that you disqualified assistant professors whose funding was running out, yet didn’t care about whether newbies had a grant or not. What if the Asst Profs were otherwise productive and were doing exciting research? What if your university was not a lateral move, but an upgrade? Did any assistant professors make your short list at all?

    I ask these questions because, though I’m doing well where I am, a position was just posted in my dream city at a more prestigious university. My R00 and R21 are running on fumes, and I don’t have an R01, but have several scored grants in the NIH system, and three grants at various currently under review at NIH and at foundations. I have two elite papers from my postdoc and I’ve had five papers published from my own lab, two of which were in “almost elite” journals (impact factor 8ish). Will it be a waste of my time to even try for this position? Would you have considered me? The other thing I’ll note is that I still have a lot of startup money left because I had those two NIH grants, so this is not at all about getting another startup package (though I’m guessing that’s what the search committee would assume).

    • There was at least one assistant prof on our short list. I think your position is better than any of the applicants that I referred to, who were more often close to their tenure decision and, on balance, struggling. Since you haven’t had an R01 yet, you’re still a new investigator, and therefore it seems likely that you will get one very soon. The depressing fact is that it is much easier to get one’s first R01 than to renew it, so in some perverse way you might be more attractive right now, without an R01, than you will be when you come up for tenure in 3 years with an R01 more than halfway run out (hoping for the best with your pending applications). This is also why a good postdoc without a K award is often seen as a lesser risk than a funded, near-tenure assistant prof.

      So I don’t think we bounced any applications similar to yours. And in fact, another department on campus recently hired someone in a similar position, albeit via a somewhat more targeted search – the PI is early in their assistant professorship, doesn’t yet have an R01, but they are working in an niche that the medical school wanted a piece of. I guess I would say to go for it, all other things being equal (in particular, if you think it’s worth the risk of alienating colleagues whose support might be crucial at tenure time, should the new position not come through). Good luck!

      • Sorry, but on the one hand it seems that you hire with a long term perspective, i.e., you don’t care that a postdoc doesn’t have funding because you hire them for the potential to bring it in. Yet OTOH, you write:

        “The depressing fact is that it is much easier to get one’s first R01 than to renew it, so in some perverse way you might be more attractive right now, without an R01, than you will be when you come up for tenure in 3 years with an R01 more than halfway run out (hoping for the best with your pending applications).”

        Isn’t this rather short sighted? So someone can get a first R01 — great! If they can’t renew it, how are they possibly still a good hire? Aren’t you counting on the fact that your money-less postdocs will be able to get a first R01 and then some others, too? How are the asst profs “riskier”?

      • This is a really fair point. There was at least one assistant prof on our long-short list, with a good funding outlook. Most of the others were about where I was, before coming up for tenure, when for various reasons I applied for several open jobs around the country, lateral move-style. In other words, okay publications but could be better, an R01 that would see me into tenure but was not at all guaranteed of renewal, and a few smaller, shorter-term grants. I got zero nibbles, which is exactly what a friend from another institution had warned me. Thankfully, I did get tenure and my funding limps onward to this day. But I know now exactly what whoever shitcanned my applications must have seen, a risky outside hire who might prove to be a drain on resources without any track record of contribution or collaboration at the new institution. And I didn’t have more than one R01 at that time, despite having had my own lab for 5+ years. By some strained analogy to the Monty Hall problem, a brand-new PI has all the doors of success (as well as failure) yet to be opened, while a late-stage assistant prof with one aging R01 has already revealed something of their trajectory and it might not be as good as what you are looking for.

        Also, a starting PI builds more than a list of papers and grants, they come in with energy for collaboration and participation that, five years later having moved to a brand new place, might be hard to summon up when pressed by more day-to-day concerns. I think this also plays into how a committee views a postdoc applicant vs an already-independent PI.

      • On a (somewhat) related note: Suppose someone is about to start a prestigious postdoc with the goal of getting a TT job in 2-3 yrs. Would you advise them to focus on getting as many pubs as possible (in high IF journals, whenever possible) rather than writing an F32 or K99? Or do you think one needs to do both?

      • I guess I would say go for the papers. I think you would need at least two years, with good pubs, in order to be competitive for the K99. If you are planning to apply for jobs at that point, then try to do both at the same time. If you don’t get an offer that you like, you may land the K99 in the meantime and be a more formidable applicant in the next cycle.

  5. CNS papers help, but one can succeed in the job search with a PNAS, an eLife or a PLoS Genetics

    Of your top 8 candidates, 6 had IF > 30 papers and none had an IF 6 papers in the same period? From your notes, that doesn’t seem to have come in to play, which might be surprising to some.

    Success in labs that are smaller/less well known might also indicate a bright prospect. If the mentor is a NAS member, a PNAS paper might be a walk in the park, but if the mentor is your average struggling prof in the middle of nowhere and the post-doc hits a couple of high..ish IF papers, are they worth a look too?

    Your comments about K99s etc should be taken in the context of Ola’s statements in the previous posts. Rightly or wrongly, some schools will view grants more favorably.

  6. *some computer issues there. Top paragraph should actually read:

    Of your top 8 candidates, 6 had IF > 30 papers and none had an IF 7 society level journals, would you view them as comparable in any way? It doesn’t seem like that happened, and some might be a little surprised by that.

  7. Yeh sorry I was having problems with the site earlier. Basically, I was asking if, within an IF range, there were certain journals you ranked more than others? You mentioned a few, but there are obviously a fair number of journals with similar IFs, and I was curious about that. The second point was whether any candidate was considered who may not have had a high, high IF paper, but instead published 3 or 4 solid (IF > 7) papers during roughly the same time period as some of the CNS candidates.

  8. OK, thanks for your answer to both of my questions. I can see what you mean by the Monty Hall problem analogy. But at the same time I find it a bit odd that your qualifications were enough to get you tenure but couldn’t generate any interest at institutions at the same level as yours. It’s almost as if the bar is higher for getting a job offer than tenure!

  9. Great post! I found the emphasis on pedigree interesting. Do you feel your committee weeded out candidates with fancy (CNS) publications and K awards because the PI was an unknown entity? Did it come down to similar candidates and the one from the famous lab make it to the final 5? Translation: as a PD who has done well in a young field in a new lab should I be worried come job season?

    • Sorry for the slow reply on this comment. I definitely don’t feel that we weeded out “fancy” candidates from “non-fancy” labs, although the truth is that there weren’t a ton of those in the pile – whether this reflects the greater ease with which well-known PIs can get published in CNS journals, or a general positive correlation between PI prominence and research excellence, or something else, I don’t know. But if an applicant had a CNS-level paper and a research area that fit generally within our search area, they would at least make our “long-short” list.

      Going through that list, of the 2/3 or so who didn’t come from NAS mentor labs, 5 were from PIs whose work I was relatively familiar with, but whom other people on the committee may or may not have known (I assume this worked in the other direction too, with other committee members). But I don’t think that my knowing a PI necessarily made me weigh an applicant better, since I know the names/work of lots of people, only some of whom I consider “good.” Only one applicant came from a truly new PI – they didn’t end up among our interviewees, but going to the PI’s website I see that this postdoc has recently left, having been hired as faculty by a higher-profile institution than mine.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s