Thursday, May 16, 2013

Evaluating research impact: Should grants count?

Warning: This post likely will not be of interest to anyone outside of academia. 

My department is in the midst of our annual faculty evaluations. Each year, our representative "advisory" committee evaluates all of the faculty in our department based on their research impact/influence, teaching, and service over the past 3 years. The collective ratings are factored into the small merit raises we sometimes receive (depending on state budgets). When I get the privilege of serving on the committee, I use as objective an approach as possible for the research component of the evaluation. I look at publication rates, citation rates, and any other metrics that seem reasonable.

Yesterday, I had a fascinating discussion with a colleague on advisory who similarly looks for objective indices of research impact and influence. The discussion led to an interesting difference of opinion, and I'd be curious to hear what others think. So, here's a thought experiment:

Suppose you have two faculty members whose research productivity, publication rates, citation rates, etc. are identical in every respect. The only difference between them is that Faculty member A has two federally funded grants whereas Faculty member B has no grants. How would you rate their research influence/impact:

Option 1: Faculty member A should receive a higher research score because they have grant funding

Option 2: Faculty member B should receive a higher research score because they do not have grant funding

Option 3:  Faculty members A and B should receive the same research score

Please leave your opinion in the comments. I'll post my own answer answer and some further discussion soon.