David Allison is one of the most highly-regarded nutrition researchers in the U.S., and is currently the Dean of the School of Public Health at Indiana University-Bloomington. To quote from his official biography, “Allison received his Ph.D. from Hofstra University in 1990. He then completed a post-doctoral fellowship at the Johns Hopkins University School of Medicine and a second post-doctoral fellowship at the NIH-funded New York Obesity Research Center at St. Luke’s/Roosevelt Hospital Center. He was a research scientist at the NY Obesity Research Center and Associate Professor of Medical Psychology in Psychiatry at Columbia University College of Physicians & Surgeons until 2001. He became Dean and Provost Professor at the Indiana University School of Public Health-Bloomington in 2017. Prior to that, he was Distinguished Professor, Quetelet Endowed Professor, and Director of the NIH-funded Nutrition Obesity Research Center (NORC) at the University of Alabama at Birmingham. He has authored more than 660 scientific publications and received too many awards to list in full here.
An important note: Dr. Allison’s views are solely his own, and should not be taken to represent the views of Indiana University-Bloomington, the National Academies’ Strategic Council for Research Excellence, Integrity, and Trust, or any other organization. Thanks to Jennifer Holmes, ELS, for editing the responses for clarity and length.
I have taken the editorial liberty of highlighting a few notable sentences that seemed calling out.
How would you describe your research to a layperson?
I have two major domains of research.
The first I began in the earliest days of my career. I began as an obesity researcher with formal degrees in psychology. When I was an undergraduate at Vassar College, I took a course in my sophomore year called Human Emotion and Motivation. We studied the work of Professor Stanley Schachter, an eminent psychological researcher at Columbia University and a leading theorist in obesity and other areas. His theorizing about and experiments in eating and obesity captured my imagination.
In almost every course I took at Vassar, I had to write a major paper. I found that I could write about obesity from a different point of view in every course. This idea that we can and should address obesity from multiple perspectives, including psychological, economic, physiological, developmental, nutritional, genetic, and so on, became a key theme of my research. So I also started to be seen as a nutrition researcher.
I then branched out to the study of caloric restriction, building on my interest in the effects of obesity and weight loss on longevity in humans, and began studying model organisms such as mice, rats, monkeys, and even fruit flies. Caloric restriction or eating less calories and food energy than one might otherwise eat is an effective way of slowing aging and prolonging lifespan in model organisms for reasons that are still being explored. This led me to be seen as an energetics (defined as the study of the acquisition, storage, and use of metabolizable energy by biological organisms) and aging researcher as well. So that is my first line of research: Obesity, nutrition, energetics, and aging.
My second domain of research concerns the rigor, reproducibility, and transparency of science. As many people know, there are significant concerns about the quality of research in general. Nutrition is often pointed out as an area where trust in research from outside the nutrition science community is low.
This lesser trust is supported by anecdotal statements among scholars studying rigor and science (e.g., Stuart Ritchie in his book Science Fictions) as well as by quantitative survey data from the Pew Research Center. When I study the obesity and nutrition literature, I frequently encounter papers that lead me to believe that new methods are needed in the field, that some existing methods are not well understood, and that some existing methods are misused or misinterpreted.
By teaching statistical methods, developing new statistical methods, detecting errors in the literature and correcting them, and providing tutorials, my colleagues and I try to enhance the quality and trustworthiness of nutrition and obesity research and scientific research in general. Thus, effort to enhance rigor, especially focused on the prevention and detection of statistically oriented errors in the scientific literature, is my other major area of research.
Describe your experience with funders (private, NIH, others). What do funders do right? What could they improve? If you could restructure how federal funders (NIH/NSF/etc.) are organized or how they operate, what would you change?
I have experience in seeking and obtaining grant funding from multiple funders, including government (NIH, NSF, USDA, and others), private industry (pharmaceutical companies, food companies, commodity groups), and private philanthropists and foundations. Each funder, just like each investigator, has their own style and personality.
I enjoy that there are multiple funders because it offers the opportunity to pursue different kinds of research. Some funders offer small amounts of research funding with quick and easy applications that allow one to try out new ideas. Others offer larger amounts of funding and may involve more slow and effortful application processes. Some emphasize creativity, novelty, and ideas that are likely to move the field forward more than incrementally, whereas others emphasize feasibility and likely foreseeable applications.
My experience has been mixed. Sometimes it is glorious and sometimes it is soul-crushing. The reviews one obtains from organized review groups such as at the NIH and NSF tend to be very thoughtful and intelligent (which is not to say that I always agree with them). Also, even though one does not know which peer reviewers on the panel reviewed which specific grant, the peer reviewers at NIH for example are all listed in the review, and the panelists must present their reviews to each other.
By contrast, the greater anonymity in peer-reviewed journal reviews seems to lead to less professional reviews. In my experience, it is not uncommon to receive reviews of one’s manuscript for a peer-review journal that are unprofessional, unintelligent, hostile, and completely off the mark. So, even when I do not agree with the reviews I may get from an NIH panel, they are rarely outside the bounds of intelligence and professionalism.
Surveys show that scientists say they spend upwards of 44% of their time on proposals, reports, IRBs, budgets–that is, administrative and regulatory requirements. Is that consistent with your experience? Is there anything that could be streamlined?
Regarding the amount of time spent on grant application, I do agree that it is far greater than is optimal. However, writing grant applications is helpful because it forces one to sharpen one’s focus, present one’s ideas clearly, consider and explain to others why the ideas are important, and push oneself for rigor. Terry Speed wrote a good article about this years ago.
Clearly, if one is spending nearly all one’s time writing applications, one cannot be doing the research. I don’t know the solution to this because we are in something of a Red Queen game in which each of us needs to run as fast as we can just to stay in place. If one investigator decides to submit fewer applications, then he or she will fall behind those who choose to submit more. If the funders reduce the amount of effort involved in applying, for example by having shorter application processes, then everybody may just submit more applications.
One solution that I think does merit consideration is reducing the use of deadlines. I believe that both the National Science Foundation and the Wellcome Trust have tried this and have shown that eliminating deadlines radically reduces the number of applications investigators voluntarily submit. This is a gentle way of reducing total application number and total time spent on applications without placing any draconian restrictions on investigators.
Another method, somewhat more controversial, would be to provide greater stable support to investigators who have proven themselves to be reliably productive both in quality and quantity. As someone who is now a senior investigator, I may be biased in this regard, but I sometimes feel like saying to the agencies, “You know me. You know I am going to keep applying until I get a grant, and you know I’m going to do the work, and you know I’m going to apply again for new grants after this one runs out, so why don’t you just give me the grant now and give it to me for 7 to 10 years instead of 3 to 5 years and monitor my progress in some not too onerous manner for you or me? Wouldn’t that save us all a lot of time?”
Of course, if a funding agency were willing to do this, people early in their career might justifiably say that they are disadvantaged because they cannot make the same argument as effectively.
Thus, we need a balance at all career levels of finding the optimal approach to helping people get the funding they need to be successful. We may need more funding overall and more stable funding within organizations such as universities so that investigators are not totally dependent on extramural funding.
My sense is that when investigators are not at all dependent on extramural funding, they are not sharpened or motivated by the need to apply for funding. They don’t pull in the extramural funding and then have those extra resources available to do higher quality and more impactful research.
On the other hand, when one is totally dependent on extramural funding, one always goes for the grants one thinks one can get, as opposed to the grants one really wants. In my own career, I notice that I apply for different types of projects when I really need funding, versus when I just want to do an interesting project. In the former case, I tend to put forth safer and more incremental research that is more likely to get funded. In the latter case, I put forward bolder, more creative ideas that are less likely to get funded but more likely to have a big impact.
If you had no constraints in terms of funding or the need to publish, is there anything that would be different about your research?
First, I would hire more scientific staff and make my programs much less lean. I often hear people say that the private sector is more efficient than academia. I think this is true only if we define efficiency in a certain way. In my experience, private companies, pharmaceutical companies for example, are far more efficient than academia in terms of output or problem solved per unit of time. Private companies have greater profit margins and can throw a great deal of money at a problem to move solutions forward quickly.
In contrast, academics have tiny margins. We do not have profits in the strictest sense, although we may have a little excess revenue at the end of the year to tuck away in a reserve account for use the next year. Therefore, we have to operate extremely tightly in terms of budget.
If we define efficiency as output or problem solved per unit of dollar spent, then I think academia is far more efficient than the private sector. But this efficiency of output per unit of dollar spent comes at the price of both slowness and sloppiness.
Therefore, if I had unlimited funding, I would hire more personnel, which would allow me to complete projects more quickly with greater expertise and much more rigorous backup and checking than I do now (and I would argue that the amount of backup and checking I do now probably already exceeds that of many academics).
The second thing I would do differently would be to conduct far more randomized controlled trials testing hypotheses about causal effects. Many of the studies we do are observational because we simply cannot muster the funding to do the rigorous randomized experiment. This is not true in all cases. In some cases, it is simply impractical, unethical, or in some other way impossible to do the randomized experiment. But that would be far fewer times if one truly had much more funding available.
Therefore, I would regularly and rapidly test the causal effects of proposed treatments on longevity in animal models, on markers of aging and health in humans, and on weight loss and food intake in humans and animal models. I would also test educational and incentive-based interventions to improve the performance of scientists themselves with outcomes such as the rigor of their research and reduction of statistical errors.
The third thing I would do differently is to test many more wild and interesting hypotheses. I would let my creative juices flow without fear of coming up empty handed or not meeting reviewers’ approval. Right now, the wild and creative ideas are like the dessert in my portfolio. I can only allow myself a few. With unlimited funding, those would become a large portion of my portfolio.
Fourth and finally, as a complement to the wild and creative ideas, I would test a series of hypotheses with probative research. By probative, I mean studies that move the needle forward. There are many hypotheses for which a great deal of data suggest an answer, but we don’t have the rigorous randomized trials to really nail the answer down. Doing so is often seen as dull because these hypotheses are well established in the literature. But a well-established hypothesis is not the same as a well-tested hypothesis.
Therefore, I would do large, simple, pragmatic probative trials that test hypotheses such as whether eating or skipping breakfast has differential effects on weight; whether consuming more fruits and vegetables, in the absence of other efforts, affects weight; whether increasing or decreasing sleep affects weight; whether daily self-weighing affects weight, and so on. The important point is that these would be done in such a way to yield relatively unequivocal answers. These would indeed be very large, well-powered trials executed with the highest degree of rigor and precision, such that they yield confident answers that in the end nail down certain questions once and for all.
If you could change the organization or management of universities, what reforms would you recommend?
Regarding the organization of universities, there are at least two main things I would consider changing.
The first is how tenure is used. The original aspiration for tenure, as I understand it, is to protect academic freedom. That is, it is to protect the ability of academics to speak their minds freely, to investigate the topics of their choice, and to write what they truthfully believe without fear of reprisal from other parties who may not like their academic findings or opinions. As someone who has experienced attacks because of my statements about scientific matters, I am acutely aware of the vital importance of tenure for preserving academic freedom. As a dean in a school at a university that was the home of Alfred Kinsey, and which now studies controversial topics including sexuality, drug use, nutrition and obesity, sexual orientation and gender identity, and many others, I realize that we must protect the ability of scholars to bravely speak out, to offer their insights, to challenge conventional thinking, and to present their findings regardless of who those findings please and do not please. This is vital and must be uncompromisable.
Yet at the same time the tenure system sometimes becomes an entitlement system, allowing a minority of individuals to persist in taking relatively large salaries for relatively minimal work and producing minimal productivity of value to society. I emphasize that this is the exception rather than the rule. Nevertheless, it is not uncommon. Even a minority of individuals acting in this way creates an enormous financial drain on university systems, which prevents available budget from being used to hire personnel who would spend the public trust with integrity and produce value for our students and society at large.
I don’t know how best to achieve this, but I believe it is an important goal. We must protect the ability of tenure to preserve academic freedom while simultaneously being good stewards of the public trust and the funds we are generously provided.
A second element I would change would be the relationship between departments and centers or institutes. Again, I am not sure how to best do this. I have been both the center director of a university-wide interdisciplinary research center and a dean. The former offers great opportunity to interact across campus and promote research in the broadest, most leading-edge interdisciplinary ways. The latter provides the ability to execute a vision and grow an organization with sustainability. Neither alone offers the full opportunity for growth at the leading edge of science one might wish for.
Conventional departments within conventional schools are highly useful in the training of scholars all the way through PhDs. Yet at the same time, I have seen many departments fail to flourish, many areas of investigation not fully take off, and many budding interdisciplinary scientists struggle to find academic career success because of the parochialism of conventional academic departments.
That is, while the conventional academic department is often excellent for providing that foundational PhD training, they are not as strong at promoting interdisciplinary research. It is not uncommon in my experience for someone to come to me for postdoctoral training, especially when I was the director of an interdisciplinary research center, and receive training in the field of obesity while learning about statistics, physiology, psychology, experimental design, genetics, and other approaches only to struggle to find an academic department willing to hire them because they no longer look like a sociologist or a psychologist or a statistician or a geneticist. They look like a hybrid interdisciplinary scientist, and the conventional department doesn’t know what to do with them. Finding ways for universities to use the power and the control associated with the ability to appoint faculty to promote maximal research impact is an important future goal.
A third point I would make is that universities, like any other large systems involving multiple parties, become political and bureaucratic. Intra-university politics and bureaucracy can bog down progress. Streamlining things for more efficient and dynamic leading-edge progress in research seems important. Maximizing the extent to which faculty governance is a responsibility to behave in constructive ways and minimizing it use in obstructive ways would be a start. Reducing unnecessary paperwork would be a second goal.
I would also like to see universities better balance the use of intramural pilot research funds for both RFP-like approaches (which involve soliciting proposals focused on strategic foci) and RFA-like approaches (very broadly conceived proposals that allow “every flower to bloom”). Many universities use their intramural funding excessively for one of these approaches without optimizing the portfolio for creativity and productivity combined.
Given the goal of improving the practice and funding of science, is there anything else I should have asked you?
With regard to ensuring accountability in science, so many things could be done, but three come to the fore as especially palpable.
1. If there is a single point at which a little bit of leverage applied today could yield the largest impact on improving rigor and science, it would be at the level of journal editors. As my colleagues and I have written elsewhere, while many journal editors are paragons of wisdom, alacrity, intelligence, and moral responsibility, this sadly seems to be far from universal. Far too many seem to substantially lack one or more of those positive traits. The end result is a confused, unmotivated, and uncourageous set of editors who are as much normative as anything else.
Yet, it is the journals that have the power to accept or reject articles for publication, to place notices of concern upon articles, to retract articles, to issue corrections, and to serve as the gatekeepers of reasonable standards. Providing editors first the education, then the resources, and ultimately the accountability for their actions to promote rigor seems to be the first move we should make. My gut tells me that on an effort-per-effort and dollar-per-dollar basis, this could yield the biggest positive increase in rigor in the shortest time.
2. Accountability for the truthfulness of statements made by all parties involved in communicating research findings and interpreting research findings could go a long way. Just as we have US News and World Report rankings for universities and Consumer Reports rankings for the safety of vehicles, it would be good if we had valid ratings and rankings of the quality and fidelity of communications around research results and their interpretation for scientists, university press offices, journals, academics quoted in news media, and especially journalists themselves. Our “Headline vs Study” series in the Obesity and Energetics Offerings which we freely distribute to over 180,000 people every Friday without charge highlights the need for this among journalists. If we can measure this truthfulness and provide data on those measures publicly, people will likely compete to have their measures look good. And if having their measures look good means making their statements more truthfully adherent to the data, then that is likely a good thing.
3. Finally, I would say there is no substitute for a conscience. I have seen people smirk and literally or metaphorically wink while they knowingly bend the truth. And I have seen others stand with pride and unequivocally speak the unvarnished truth regardless of whether it was especially comfortable for or desired by them. How do we bring forward more scientists whose identify as uncompromising, unequivocal, and unvarnished truth-tellers and not as “hypesters” or spin doctors?
Just for fun, what’s an article in the past few years that you wish you could have written?
I’ll offer two perspectives. One is an article someone else wrote that I wish I had written, and the second is an article that has not yet been written but that I wish I had the resources to write.
The article written by somebody else that I wish I had written is one published by Dr. Milkman and colleagues recently (https://www.nature.com/articles/s41586-021-04128-4.pdf?proof=tr). In it, she describes the organization of a group of investigators to conduct multiple randomized controlled trials aimed at addressing questions about which methods work for promoting gym attendance.
While the question itself is of interest (that is, the question of what promotes gym attendance), that for me is not the burning question. What intrigues me is her use of the randomized controlled trial to address practical questions of implementation, the scale of the analysis, the thorough investigation of interpretations around heterogeneity and replicability of findings, and the rigorous efforts to assess cause and effect meaningfully in a domain where such efforts are the exception and not the rule. I salute Dr. Milkman and colleagues for this article and work.
With respect to an article I wish I could write, I consider the question of the verifiability of statistical analysis and results in randomized controlled trials of the childhood obesity intervention literature. As my colleagues and I wrote in this article that received considerable positive attention (https://onlinelibrary.wiley.com/doi/abs/10.1111/obr.12923), there are substantial concerns that the effects of interventions in the childhood obesity literature coming out of randomized controlled trials are distorted by a combination of obfuscation, errors, hype, and perhaps other factors as well. My group and I use the term verification to include reproducibility as defined by the National Academy of Sciences (who define reproducibility as “obtaining consistent results using the same input data, computational steps, methods, and conditions of analysis…”) but to go beyond that.
We say that a result has been verified when not only has it been reproduced but further that the methods used in the initial analysis are shown to be correct or legitimate and that the interpretation of the result is correct given the design, data analysis, and results obtained. In other words, verification is an assessment of the correctness of the design, analysis, reporting, and interpretation of the statistical results. It stops short of a true determination of the correctness of the conclusion, which would require additional factors, perhaps also replication. But it does indicate whether the study, in and of itself, was correctly done and whether the conclusions as reported are accurate and legitimate.
My strong sense is that in the childhood obesity intervention literature (by intervention I include both treatment and prevention and also studies of related variables that may someday be used to guide treatment and prevention), many such studies would not be verifiable. I wish I had the resources to retrieve a very large number of systematically or randomly selected obesity intervention randomized controlled trial datasets, check them for verifiability, and report the results.