The Indian Council of Medical Research (ICMR) has recently unveiled its new Impact of Research and Innovation Scale (IRIS), designed to gauge the influence of biomedical, public health, and related research initiatives it funds. As India’s leading government body for health research grants and agenda setting, this new scale is poised to significantly shape the country’s medical and health research landscape.
Under this new system, research impact will be quantified using “publication-equivalents” (PEs). For instance, a primary research paper or a systematic review in a peer-reviewed journal earns 1 PE. In contrast, a research paper directly influencing policies or guidelines is valued at 10 PEs, a patent at 5 PEs, and a widely adopted commercial medical device at a significant 20 PEs.
The Upsides of a Standardized Approach
This method of quantifying research impact offers several benefits. Primarily, by establishing PEs as a universal “unit of impact,” it creates a consistent framework for evaluating diverse types of research outcomes. This allows ICMR to fairly assess the contributions of researchers across various disciplines – from biochemistry and biomedical engineering to public health – and across different stages of research, including fundamental science, translational studies, and population health initiatives.
Secondly, IRIS broadens the definition of “impact” beyond traditional academic citations. Often, researchers, especially in academia, are driven to produce work that results in highly cited papers, neglecting other forms of valuable contributions. IRIS could shift this paradigm, encouraging scholars to explore a wider range of research activities. Thirdly, by directly linking PEs and the IRIS framework to critical decisions like funding allocation and project prioritization, the scale moves beyond a purely academic concept, ensuring its practical application and real-world influence. The ICMR’s current pilot program, applying IRIS to evaluate existing grants and institutions, strongly signals its commitment to this new approach.
Potential Pitfalls in Research Evaluation
However, it’s crucial to scrutinize where this seemingly straightforward approach to standardized research evaluation might fall short. One primary concern is the lack of a robust theoretical justification for PEs as a definitive unit of research impact in the way ICMR intends to use it. The ICMR’s guidelines state that commentaries, perspectives, and narrative reviews will be assigned zero PEs. This raises a significant red flag: groundbreaking works, such as the 1977 paper that introduced the biopsychosocial model of medicine – a concept that profoundly reshaped medical and public health research – would, under IRIS, be considered to have no impact. Clearly, generating new ideas and fostering critical discussions around evolving evidence are fundamental to research progress and inherently impactful. Yet, this PE-based system could inadvertently deter Indian researchers from engaging in such vital intellectual contributions.
A second major concern is how IRIS could distort the perceived value of different research outcomes. For instance, research directly leading to policy changes is awarded 10 PEs, while commercial devices are given 20 PEs. This disparity implies that impactful public health initiatives, like the RATIONS clinical trial on nutrition in tuberculosis patients or India’s revolutionary Home-Based Neonatal Care program, might be deemed less significant than a commercial robotic surgery device. Such an emphasis risks devaluing fundamental science and academic medicine, potentially pressuring researchers and institutions toward commercialization at the expense of other crucial public health advancements.
While it is within the ICMR’s purview to set priorities, this strong bias towards commercialization could undermine the fundamental ethos of scientific research as a public good. The true value of research proposals often depends heavily on the specific local context. Given India’s past struggles with a weak research ethics culture in biomedical and health fields, there’s a tangible risk that this new PE system could exacerbate such issues. Therefore, it is absolutely essential for our public sector institutions to uphold and champion the principle of ‘research as a public good’.
Ultimately, any scale designed to profoundly influence biomedical and health research in India must be developed with the utmost adherence to rigorous research standards, complete transparency, and clear accountability mechanisms. Although the ICMR note highlights an ongoing pilot and positive initial feedback, the development of such a critical evaluation tool demands a more formalized approach, utilizing established study design and analytical methodologies. For example, the process of assigning PEs to various impact indicators could benefit significantly from a national-level Delphi study, allowing researchers to collectively reach a consensus. Furthermore, the underlying data should be made available to independent bodies for thorough analysis and validation of the scale’s effectiveness and fairness.
Measuring research impact is undeniably a complex undertaking with no single perfect solution. Institutions like the ICMR are presented with a unique opportunity – and a profound responsibility – to tackle this challenge through creative and collaborative approaches.