Reviewing for the Canada Graduate Research Scholarship – Doctoral (CGRS D) program
Title
Welcome to this learning module for peer reviewers of the Canada Graduate Research Scholarship – Doctoral (CGRS D) Program.
This module pertains specifically to applications submitted to CIHR. The goal of this module is to ensure that reviewers understand the CGRS D Program and feel prepared to effectively participate in the review process.
Navigation
This course is designed to be self paced.
Use the playbar below to resume playback, navigate between slides, mute and unmute audio, and toggle closed captions. You can also browse the full table of contents, and collapse or move the playbar.
Objectives
By the end of this module, you will be able to:
- Identify key features of the CGRS D Program;
- Apply evaluation criteria and considerations to the review of applications;
- Understand the rating scale and how to write a high quality review;
- and summarize the steps in the peer review process for the CGRS D Program.
Objective 1: The CGRS D Program
In this section, you will learn about the CGRS D Program and the eligibility criteria for applicants.
The CGRS D Program
The CGRS D Program at CIHR supports and promotes research excellence in all areas of health research. This support allows scholars to more fully concentrate on their doctoral studies, to seek out the best research mentors in their chosen fields and contribute to the Canadian research ecosystem during and beyond the tenure of their awards in Canada or abroad.
Eligibility of Applicants
For the CGRS D Program, applicants must have completed no more than 36 months of full-time equivalent study toward the degree for which they are requesting funding, as of December 31st of the application year. They must also not hold or be on leave from a tenure-track or tenured faculty position.
CIHR staff and/or institutions have reviewed all applications to confirm the eligibility of the applicants – it is not the reviewer's responsibility to assess the applicant's eligibility.
If reviewers note any eligibility flags, they should bring it to the attention of the CIHR staff immediately, and must not include it as part of their review.
Formatting Requirements
To ensure that all applicants have the same amount of space to write their research proposals, applicants must adhere to the formatting requirements outlined by CIHR such as font sizes, spacing, and page limits.
Similar to the eligibility criteria, it is not the responsibility of reviewers to ensure appropriate formatting of the application. However, should a reviewer be assigned an application that may not have followed instructions, this should be brought to the attention of CIHR staff as soon as possible.
For the full list of formatting requirements, consult the Acceptable Application Formats and PDF Attachments webpage.
Objective 2: Evaluation Criteria and Considerations
In this section, you will learn about evaluation criteria and considerations used in the review of the CGRS D Program applications.
Assignment of Applications
As a reviewer for the CGRS D Program, you will primarily be assessing the candidate. As such, it is not essential for your research expertise to align directly with the research area of the application.
Reviewers are asked to apply their research expertise generally when assessing the diverse array of application assignments.
When assessing applications, all reviewers are expected to abide by CIHR's principles of peer review: confidentiality, absence of conflict of interest, fairness, and transparency.
Evaluation Criteria
Your evaluation of the application should be based on the following evaluation criteria: Research potential, and Relevant experience and achievements obtained within and beyond academia.
In the following slides you will learn more about the evaluation criteria.
Review Appendix A of the Reviewers' Guide for the CGRS D Program, for additional details on the evaluation criteria.
Indicators of Research Potential
The criterion, Research potential, accounts for 50% of the score. The indicators of research potential are:
- The quality of the research proposal and contributions
- Demonstration of ability and potential
- Sound judgement and critical thinking
- Responsible and ethical research conduct
- Research originality and initiative
- And lastly, effective communication
Note: When assessing sponsor evaluations, keep in mind that applicants have no opportunity within the application to provide justification for their choices of sponsors.
Indicators of relevant experience and achievements
The criterion, relevant experience and achievements obtained within and beyond academia accounts for the other 50% of the score. Indicators of this criterion are:
- Relevant training of the applicant; such as academic training, lived experience, and traditional teachings
- Scholarships, awards, and distinctions
- Academic record
- And lastly, professional, academic, and extracurricular activities and collaborations with supervisors, colleagues, peers, students and members of the community.
Broadening your assessment of research contributions and impacts
CIHR is a signatory of the San Francisco Declaration on Research Assessment (or DORA), which recognizes the need to improve the ways in which research is evaluated. When assessing applications, reviewers should look beyond the traditional indicators of productivity.
Apply the following when assessing productivity in reviews according to DORA.
- Do comment on other research contributions and impacts.
- Do consider an applicant's context and how it could have affected their productivity. For example, leave history, career stage, area(s) of research, experiential knowledge, diverse career paths, family responsibilities and pandemic impact.
- Don't solely comment on h-index or journal-based metrics such as impact factor. Instead, also speak to the quality and impact of the contribution as described by the applicant.
Check out our Skill Builders to practice integrating DORA principles into your reviews.
Assessing Sex and Gender Integration
When reviewing applications, reviewers must also assess the integration of sex and gender in the proposed project where applicable.
Learn more about Sex, Gender, and Health Research by exploring the resources on this slide. All links will open in new windows, so you can return to this module when you are ready.
Evaluating Indigenous Health Research
When evaluating applications that involve First Nations, Inuit and Métis Peoples, reviewers are strongly encouraged to review the following training modules and literature. These resources provide critical context, ethical frameworks, and distinctions-based guidance to ensure respectful, informed, and equitable evaluation.
Question 1
Test your knowledge!
In the application you are reviewing, the candidate has provided a 2-page document which defines their role in multi-authored publications as well as environmental factors that affected their capacity to publish. Which evaluation criteria does this apply to?
- Research potential;
- Relevant experience and achievements obtained within and beyond academics;
- Or all of the above.
Question 2
Which of the following does not apply when considering "Relevant experience and achievements obtained within and beyond academia" in your evaluation:
- The sponsor's assessment letter about candidate;
- Undergraduate academic transcripts and graduate transcripts;
- The proposed Research Project Summary;
- Or the "leaves of Absence and impact on Research" Section of the Common CV.
Objective 3: Rating Scale and Conducting Quality Reviews
In this section, you will learn about the rating scale and how to write a high-quality review.
The Rating Scale – Eligible for Funding
Reviewers should be assessing each application based on the information provided and should not be seeking additional information or outside opinion to supplement what is contained in the application.
Reviewers will be asked to rate the evaluation criteria on a scale of 0.0 to 4.9 for their assigned applications. These ratings will then be weighted automatically to produce an overall score for each application you review in ResearchNet.
Applications with an overall weighted score above 3.5 are eligible for funding and are categorized as follows:
Applications with a range of 4.5 to 4.9, excel in most or all relevant aspects and any short-comings are minimal.
Applications with a range of 4.0 to 4.4, excel in many relevant aspects, and reasonably address all others. Certain improvements are possible.
Applications with a range of 3.5 to 3.9, excel in some relevant aspects, and reasonably address all others. Some improvements are necessary.
The Rating Scale – Not Eligible for Funding
Applications with an overall weighted score of 3.4 or less are not eligible for funding and are categorized as follows:
Applications with a range of 3.0 to 3.4, broadly address relevant aspects and major revisions are required.
Applications with a range between 2.0 and 0.0, fail to provide convincing information and/or have serious inherent flaws or gaps.
Please use the full rating scale when evaluating your assigned applications.
Review Quality Expectations
For each application, reviewers must provide a concisely written assessment highlighting the strengths and weaknesses for each evaluation criterion that supports their ratings.
When writing your review, you should consider the following aspects to ensure your review is of the highest quality:
- Appropriateness: Your review comments are fair, understandable, original, confidential, and respectful.
- Robustness: Your review is thorough, complete and credible.
- Utility: Your review provides feedback that addresses the needs of reviewers, applicants and funders.
CIHR has developed a checklist and worksheet as practical tools to assist both applicants and reviewers to apply the review quality criteria to their received or submitted reviews. In addition, all reviewers are expected to complete the Conducting Quality Reviews module.
Writing Reviews
When writing reviews , you can focus your comments on the following elements:
- Use descriptors from the CIHR rating scale that align with your rating.
- Justify and provide context for each comment to ensure clarity and relevance; and
- Consider a broad range of contributions and impacts, balancing quantitative metrics with qualitative aspects to provide a comprehensive evaluation.
It is important to keep your review simple and straightforward. Avoid biased, sarcastic, flippant, or arrogant language to maintain professionalism and objectivity. Additionally, use gender-neutral pronouns such as "they" or refer to "the applicant" instead of using "he" or "she".
For additional guidance, check out our Skill Builders to help improve the quality of your written reviews.
Artificial Intelligence and Award Review
Reviewers should also be aware of CIHR's current guidance on the use of generative artificial intelligence (AI) in peer review.
With the rise of powerful AI tools, it is important to remember that peer reviewers remain responsible for reading applications assigned to them and writing fair and rigorous reviews in their own words. Inputting application information into generative AI tools, such as ChatGPT or DeepL will result in breaches of privacy and in the loss of custody of intellectual property, as these tools may store and reuse data.
Therefore, the use of publicly available generative AI tools for evaluating grant applications is strictly prohibited. Reviewers will be asked to confirm their understanding and compliance with the Use of Artificial Intelligence in the Development and Review of Research Grant Proposals guidance in ResearchNet.
Question 3
True or False. To ensure that all applications are treated equally, reviewers should not complete additional research in addition to their evaluation on the content of the application.
Question 4
Yes or no? While reviewing an application you find yourself stuck on how to express your feedback. You decide to use ChatGPT to write parts of your review and to compare sections of the research proposal with existing research in the field. Is this acceptable?
Objective 4: The Review Process
In this section, you will learn about the peer review process for the CGRS D Program.
Peer Review Process
The peer review process of the CGRS D consists of:
- Assigning applications to committees;
- Identifying conflicts of interest;
- Conducting reviews of assigned applications;
- Submitting written reviews and ratings;
- And participating in re-review, if required.
Distribution of Applications by Committee
Each application will be assigned to the committee with the scientific area that has been chosen by the applicant, which most closely aligns with the applicant's proposed research activities. Every application in the committee will be assigned to two reviewers.
The CGRS D A committee will assess candidates' potential of attaining competence as prospective researchers in biomedical and/or clinical research.
The CGRS D B committee will assess candidates' potential of attaining competence as prospective researchers in health services research and/or social, cultural, environmental and population health research.
For more information about each committee, you can refer to their mandates.
Identify Conflicts
In ResearchNet, you will first need to agree to the Conflict of Interest and Confidentiality Agreement, which includes the expanded version of the Equity, Diversity and Inclusion Self-identification Questionnaire. Completing this questionnaire is a requirement for peer review committee members, enhancing our ability to understand the degree of diversity in peer review committees.
The questionnaire is accompanied by a Privacy Notice Statement which outlines CIHR's intended purpose and uses of the self-identification data.
Next, to start conducting your reviews, click on the "Manage Conflicts/Ability to Review" task.
Identify Conflicts
This will bring you to the set of applications that have been assigned to you. Select each application by clicking on the hyperlinked application numbers.
You will have access to the relevant information to determine if you are in conflict.
Please indicate if you are in conflict with the application. You can find more information on when you might need to declare a Conflict of Interest by clicking the button at the bottom of your screen.
As previously mentioned, the assignment of applications is not based on specific expertise of reviewers. As such, the reviewers' mandate is to review each application with a generalist's perspective and assess the overall quality of the research proposed. There should be no conflict declared due to lack of expertise with an application.
Conduct Reviews
Please ensure you are familiar with the evaluation criteria for the CGRS D Program. You can now start working on your reviews by clicking on the "Conduct Reviews" task.
To assess the applications, click on the hyperlinked application numbers.
From this screen, insert your numeric rating and ensure you provide written feedback to highlight the application's strengths and weaknesses for each evaluation criterion.
The applicant will be given access to these comments. Therefore, they must be written in a constructive manner and must be a reflection of the attributed ratings.
It is advised that reviewers should save their work often in ResearchNet by selecting "Save draft copy" or copy and paste from a word processor as ResearchNet times-out often.
Submit Reviews
When you are ready to submit your reviews, you will first need to select the reviews and then click on "Submit Selected Reviews".
It is critical for reviewers to submit their written reviews and scores by the deadline specified by CIHR staff. If, at any point, you are unable to submit your review on or before the deadline, contact CIHR staff as soon as possible.
Participate in re-review, if required
When all reviews are submitted, a calculation is applied to determine which applications have received discrepant scores. In this instance, CIHR will ask these reviewers to discuss the application in order to reconcile their scores. If you are requested to do a discrepancy review on an application, you will be notified by e-mail with the coordinates of the other reviewer and your access to that specific application on ResearchNet will be reopened. The goal of this process is to have reviewers listen to and consider each other's opinion. While you are not obliged to change your score, you will be required to re-submit your score after your discussion with the other reviewer.
Once all re-reviews are submitted, CIHR will re-assess to determine if applications are still at risk of an unfair decision because of a wide spread between the 2 reviewers' scores. If the response is still "Yes", then CIHR will ask an additional reviewer to evaluate the discrepant application and the final score will be determined by taking the average of all scores. You may be asked to act as an additional reviewer at this stage to settle discrepant scores. Finally, following the submission of all the reviews, CIHR will generate ranking lists for each committee. Applications are recommended for funding starting with the highest ranked as far as the total competition budget allows.
Summary
Congratulations! You have now completed the learning module for peer reviewers of the CGRS D Program. You should now be able to:
- Identify key features of the CGRS D Program;
- Apply evaluation criteria and considerations to the review of applications;
- Understand the rating scale and how to write a high-quality review;
- And summarize the steps in the peer review process for the CGRS D Program.
Survey
Please complete the survey to help CIHR track completion and improve the quality of our learning materials.
Additional Resources
This page contains additional resources. Click on them below to view important links for peer reviewers of the CGRS D Program.
- Date modified: