The Indian Council of Medical Research (ICMR) recently unveiled its innovative Impact of Research and Innovation Scale (IRIS). This new system is designed to quantify the impact of biomedical, public health, and other related research projects funded by the organization. As India’s leading body for health research grants and agenda-setting, the structure and calculations of IRIS will significantly influence the country’s medical and health research communities.
ICMR proposes to measure research impact using units called ‘publication-equivalents’ (PEs). For instance, a research paper reporting primary findings or a systematic review published in a peer-reviewed journal earns 1 PE. However, a paper cited in policies or guidelines is given 10 PEs, a patent accounts for 5 PEs, and a widely adopted commercial device brings in 20 PEs.
The Benefits of Standardization
This method of quantifying research impact offers several key advantages. Firstly, using PEs provides a standardized metric for discussing various types of research outcomes. This allows ICMR to uniformly assess the impact of diverse projects from different disciplines like biochemistry, physiology, biomedical engineering, and public health, operating at various stages from basic science to translational and population-level research.
Secondly, IRIS acknowledges that research impact extends beyond simple citations. Many medical and health researchers, especially those in academic settings, often lack incentives to pursue work that doesn’t primarily result in citable academic papers. A system like IRIS could disrupt this pattern, encouraging scholars to diversify their research portfolios to include broader forms of impact. Thirdly, by directly linking PEs and IRIS to crucial decisions about funding allocation and project prioritization, the scale ensures that impact measurement is not merely a theoretical exercise. ICMR’s initiative to pilot IRIS for evaluating existing research grant programs and institutions signals a serious commitment to action.
Potential Pitfalls in Research Evaluation
Despite its advantages, this seemingly straightforward approach to standardizing research evaluation warrants careful scrutiny for potential shortcomings. Firstly, the theoretical foundation for PEs as a universal unit of research impact, as proposed by ICMR, appears to be missing a robust rationale. The ICMR’s guidelines assign 0 PEs to commentary, perspective pieces, and narrative reviews. This means a seminal 1977 paper that introduced the biopsychosocial model of medicine, which profoundly reshaped medical and public health research globally, would, under IRIS, be considered to have no impact. Given that the articulation of new ideas and critical discourse is foundational to research and undeniably impactful, a PE-based system could inadvertently discourage Indian researchers from engaging in such vital intellectual contributions.
Secondly, IRIS risks skewing the valuation of certain research types over others. Research leading to policy changes is awarded 10 PEs, while commercial devices earn 20 PEs. This differential could lead to a situation where, for example, the RATIONS clinical trial on nutrition in tuberculosis patients or India’s groundbreaking Home-Based Neonatal Care program — both revolutionary in community health — are deemed less impactful than a commercialized robotic surgery device. Such an outlook might disincentivize basic science and academic medicine, pressuring researchers and institutions toward commercializing their discoveries.
While commercialization has its place, there’s a real danger that this shift could undermine the fundamental ethos of scientific research as a public good. The value of new proposals is deeply tied to their local context. India’s biomedical and health research landscape has historically grappled with ethical concerns, and there’s a risk that this focus on PEs could exacerbate such issues. It’s crucial for public sector institutions to uphold the principle of ‘research as a public good.’
Finally, a measurement unit and scale intended to shape the future of biomedical and health research in India demand development through rigorous research standards, complete transparency, and robust accountability mechanisms. Although the ICMR mentions an ongoing pilot and positive feedback, the development of such a critical scale should adhere to well-established study design and analysis methodologies. For example, the assignment of PEs to different indicators could be determined through a national-level Delphi study, allowing researchers to reach a consensus. Furthermore, all data must be openly shared with independent groups for thorough analysis and validation.
Ultimately, measuring research impact is an inherently complex task with no single correct answer. Institutions like the ICMR are presented with a unique opportunity and responsibility to address this challenge with creativity and collaboration, ensuring that progress benefits all.