Roger D. Peng is a Professor of Statistics and Data Sciences at the University of Texas, Austin. Previously, he was Professor of Biostatistics at the Johns Hopkins Bloomberg School of Public Health and the Co-Director of the Johns Hopkins Data Science Lab. His current research focuses on developing theory and methods for building successful data analyses and on the development of statistical methods for addressing environmental health problems. He is the author of the popular book R Programming for Data Science and 10 other books on data science and statistics.
1. Based on your experience with funders (private, NIH, others), what do funders do right vs. what could they improve? For example, if you could change anything about federal funders (NIH/NSF/etc.), such as their approach to peer review or to “high-risk” research, what would you change?
Over my career I’ve had the chance to obtain funding from a wide variety of funders, including federal agencies, non-profit institutes, and corporations. Across these experiences I can say definitively that no one organization does it perfectly, which I think is telling. Ultimately, all funding organizations are run by people, and those people have agendas, interests, priorities, and requirements and anyone who receives funding from them will have to deal with those things. Given the amounts of money we are requesting (often millions of dollars), it’s absurd to think that there wouldn’t be any strings attached.
That said, I think one thing federal funders (my experience is mostly with NIH) do well is to let scientists drive the review process via their numerous study sections. Having served on both sides of this process as reviewer and reviewee, I can say it is by no means a perfect process. But I generally struggle to come up with better alternatives to this process. One thing I appreciate about the NIH in particular is that it does a reasonable job of separating the evaluation of scientific quality from program priorities. I think some scientists kind of take this separation for granted but it’s important to realize that not all organizations do this well.
One issue that I have seen with all funders to some degree is that the program officials have ever-increasingly strong agendas that they are pushing within their own organizations. As a result, there is often a desire to “show off” the research they are funding to higher-ups in their organizations. This often translates to extra work for grantees (making presentations, webinars, attending meetings) that was not explicitly planned for in the grant. I think if funding organizations could manage their incentives so that this kind of burden on grantees is minimized, that would be an improvement.
The question of how to fund so-called “high-risk” research is a good one, and I am somewhat heartened by fact that in my view it doesn’t seem like anyone has answered it! I think funding organizations, whether it’s the NIH or a venture capital firm, are always looking to de-risk investments, so there will naturally be a tension there. Furthermore, organizations with strong internal agendas will want to make sure their investments are “getting results”, which contributes to further de-risking. In my experience, “high-risk” research is difficult to plan for, and is therefore not necessarily the best candidate for grant funding. I think the only hope for paying for such research is to allow researchers the latitude to explore a range of ideas within their existing research programs.
2. Surveys show scientists spend upwards of 44% of their time on proposals, reports, IRBs, budgets, etc.–that is, administrative and regulatory requirements. Is that consistent with your experience? Is there anything that could be streamlined?
I haven’t made any effort to measure this time spent, but my suspicion is that while my percentage of time spent on these kinds of activities is likely far less than 44%, the nature of the work makes it feel like much more. Luckily, for most of my career writing grants, I’ve been able to work with an excellent staff that helped with much of the administrative work associated with federal grants.
3. If you had no constraints in terms of funding or the need to publish, is there anything that would be different about your research?
Ultimately, I think no. Getting grants, publishing, giving talks, and everything else along those lines are just part of a larger process of getting other people to accept your ideas, whatever they may be. Yes, it’s true that it often takes some focused time to come up with ideas in the first place. But academics tend to think that if they have a great idea, then others will instantly recognize it and accept it. In my experience, that’s just not true. Not in academia or really anywhere else. So getting funding and publishing is primarily about bringing other people on board and building a community of people around your ideas. If you want your ideas to have any sort of impact, these are necessary things to do.
4. Any reflections on the role of statisticians in “team science”? How should they be funded and given academic credit for, say, working on a clinical trial headed by a different PI?
When it comes to being funded on large collaborative projects as a statistician, I would say that my experience has largely been positive. In general, I have found that there is a disincentive to properly fund the cost of doing data analysis, as most of the funding tends to go towards data collection. In part, I think it is because the cost of data collection is easier to measure and account for than the cost of data analysis. The question of how to give credit for people contributing to team science goes far beyond statisticians and is an important question for academic institutions. I think the biostatistics community has largely come around to the idea that there are various contributions that we can make, such as methodological development, collaborative contributions, and translational work. Different people weight these activities differently through their careers and I think it’s important that the field allow for and promote a range of different combinations.
5. Given the goal of improving the practice and funding of science, is there anything else I should have asked you?
I think one thing that deserves more discussion is simply the amount of funding that is available out there. The annual maximum allowed direct cost for an NIH grant ($500,000 per year) has not changed in I think over 20 years. That one fact alone I think explains a lot of the problems going on now. In particular, just over the course of my career, I’ve seen people having to apply for a larger number of grants to cover the same amount of work that they might have covered with just one grant in the past. With more grants, there comes more administrative work.