Showing posts with label NIH Announcements. Show all posts
Showing posts with label NIH Announcements. Show all posts

Friday, June 23, 2023

Should critiques from humans be replaced by artificial intelligence for grant proposal reviews?

Today, NIH released Notice number NOT-OD-23-149, "The Use of Generative Artificial Intelligence Technologies is Prohibited for the NIH Peer Review Process". The Notice is linked to NOT-OD-22-044, released in December of 2021, "Maintaining Security and Confidentiality in NIH Peer Review: Rules, Responsibilities and Possible Consequences".

The latest notice was released to clarify that NIH reviewers are prohibited from using any natural language processing (NLP), large language model, or other Artificial Intelligence (AI) based technology to aid in their review and written critique of proposals. In other words, those harsh comments about your Approach from the dreaded Reviewer #2 should be written by an actual human scientific peer reviewer and not by a ChatGPT bot. (Although I do wonder how many times terms like "fishing expedition" and "overly ambitious" would be used by a model that is trained using summary statements from prior grant reviews.)

If you currently serve or will be serving on an NIH study section as a peer reviewer, you will be expected to adhere to revised Security, Confidentiality, and Non-disclosure Agreements for Peer Reviewers that are updated to incorporate the new restrictions regarding AI-based grant reviews. Any reviewer that uploads or pastes content from a grant application into a generative AI tool will be deemed in violation of NIH peer review confidentiality and integrity requirements. 

This new requirement is designed to ensure fairness during the proposal review process. As recent controversial examples in banking and sports have shown, it is relatively easy to introduce bias into an AI based model. Although peer reviewers bring their own biases to the review process, the scoring and outcome are more transparent when written critiques and summary statements are prepared by human beings who are putting forth their best effort to review proposals based on their untainted scientific opinions.

Thursday, June 1, 2023

How do I address the new NIH Data Management and Sharing Plan (DMSP) requirements?

In January of 2023, the Final NIH Policy for Data Management and Sharing was released after several years of development. The goal of this policy is "to promote the management and sharing of scientific data generated from NIH-funded or conducted research". A Data Management and Sharing Plan attachment (also called DMSP and DMS Plan) is now required for submission as part of most NIH research grant proposal mechanisms (e.g. R01, R21, R03) and not currently for training grants or fellowships. If you are not sure whether your proposal is required to comply with the policy, NIH has created a helpful page.

The Data Management and Sharing Policy requires investigators and institutions to: 

  1. Plan and budget for the managing and sharing of data
  2. Submit a Data Management and Sharing Plan for review when applying for funding
  3. Comply with the approved Data Management and Sharing plan
What does this mean for your grant proposal? First, you will need to write a Data Management and Sharing Plan that includes the specified information. A blank format page that can be used to create the document is located here. 

You will also need to consider whether the data sharing activities proposed in your application will incur any costs. These costs can be budgeted as a separate line item provided they fall into one of the allowable categories. It is important to work closely with a pre-award research administration professional at your institution to ensure these costs are budgeted appropriately and in line with any institutional requirements.

One important note: the Data Management and Sharing Plan does not replace the Resource Sharing Plan. You may still need to include both with your application.

For more information, hear directly from NIH program staff in the video below:






Tuesday, May 23, 2023

Are you guilty of using active links and hypertext in NIH grant proposals?

So much of our daily lives include links, looking up information, and hopping between websites.  Clicking on active links in documents makes things seamless and fast.  Like other aspects of grant applications (human subjects attachments, anyone?), links have unfortunately been abused by some applicants who are seeking to circumvent the rules. In an effort to level the playing field, NIH created a policy that expressly prohibits hyperlinks in grant applications.  The policy can be found in the Grant Application Guide.  It states, in part:

Hyperlinks and URLs are only allowed when specifically noted in funding opportunities and/or form field instructions. It is highly unusual for a funding opportunity to allow links in Specific Aims, Research Strategy, and other page-limited attachments.

Hyperlinks and URLs may not be used to provide information necessary to application review. Applications must be self-contained and reflect the information available at time of review. 

Reviewers are not obligated to view linked sites and are cautioned that they should not directly access a website (unless the link to the site was specifically requested in application instructions) as it could compromise their anonymity.

When allowed, you must hyperlink the actual URL text so it appears on the page rather than hiding the URL behind a specific word or phrase (hypertext).

Since applicants were either unaware or routinely ignored this policy, NOT-OD-20-174 was released as a reminder. Although this policy has been in existence for years, there are many who continue to attempt to include hyperlinks in the text. The notice "serves to remind the applicant community about existing guidance that restricts use of hypertext (e.g., hyperlinks and URLs) in NIH applications".  It also lists multiple concerns related to the use of hypertext in grant applications, including overstuffing of applications, reviewer confidentiality, and use of malware.

In summary, keep the hyperlinks out of your entire proposal.  Allow enough time to review your complete application and remove any hypertext. Do not take a chance that your proposal will be withdrawn due to noncompliance!


Tuesday, October 5, 2021

Francis Collins announced he is leaving NIH - what comes next?

 Today, Francis Collins announced that he will be stepping down from the National Institutes of Health (NIH) by the end of this year. Dr. Collins led NIH for the past 12 years, and although he is leaving his leadership role, he plans to continue his laboratory work at the National Human Genome Research Institute (NHGRI). An interim director has not yet been named.

Will the change of leadership result in a revised research agenda for NIH? The NIH director is appointed by the U.S. president and must be confirmed by the senate. In my opinion, this is unfortunately a case of politics mixing too closely with science, but at least it guarantees an open selection process. Dr. Collins has led the NIH in a way that has been non-partisan and focused on propelling science forward. Many great achievements have occurred during his tenure, and I hope that President Biden will find a suitable scientific mind to continue this great work.

Monday, June 15, 2020

New NIH Coronavirus Disease 2019 (COVID-19) Update Posted

Today, the NIH posted an updated document with revised guidance on current operations and procedures for current grantees and grant applicants.  Here is a link to a helpful presentation file.    

If you don't want to scroll through the slides (this Zoom fatigue has me feeling impatient, too!), then you can read the summary of talking points.

Monday, January 20, 2020

NIH FORMS-F applications are coming!

Are you working on a proposal that is due May 25, 2020 or later?  It seems like we were all just barely starting to make sense of all the complicated human subjects and clinical trials requirements in FORMS-E, when NIH decided to publish this notice and throw us all another curve ball.  Yes, it's time to start to familiarize yourself with the new application requirements.  There are also updates to the NIH and AHRQ research training grant, career development, and fellowship application requirements, which are outlined here.

Saturday, January 12, 2019

Is NIH closed during the government shutdown?

This question has been coming up quite a bit over the past couple of weeks.  For the record, NIH is open during the partial federal government shutdown.  Don't believe me?  Read it for yourself here. 

Although NIH is not closed, agencies such as NASA and the FDA are experiencing the effects of the shutdown.  This impacts not only research, but also the individual employees who are currently furloughed or working without pay.  My sincere hope is that both parties can come together on a compromise and that President Trump will reopen the government soon so everyone can get back to work.

Thursday, January 10, 2019

Scientific Premise, Rigor, and Reproducibility are still causing confusion

Are you working on an R01 proposal for the upcoming February 5th NIH deadline?  Like many Principal Investigators, you may have spent a lot of time crafting the perfect write-up of your scientific premise.  Unfortunately, that means you missed the recent notice (NOT-OD-18-228) regarding the changes to NIH grant application instructions that take effect this month.  The references to "scientific premise" have been replaced by an explanation of requirements to address scientific rigor and reproducibility.  Still confused?  See this handy website for guidance on rigor and reproducibility from NIH.

Monday, October 15, 2018

New NIH Proposal Videos

Are you still confused by the NIH grant peer review process?  As explained in a recent notice, NIH has released new videos to help you better understand what happens during peer review.
The video that I believe will be most helpful is "Top 10 NIH Peer Review Q&A".  The video includes 10 commonly asked questions and answers from 10 Center for Scientific Review experts. Watch it and share any other tips or questions below in the comments.

Thursday, October 4, 2018

Upcoming changes to review criteria at NIH

In a recently published notice, NIH announced that changes will be made to grant reviewer instructions beginning with applications received January 25, 2019.  Notably, the term "scientific premise" is being replaced with additional instructions surrounding rigor and reproducibility.  (From my perspective, this is an excellent change, since many a scientist has struggled with how to address the scientific premise given the limited guidance provided by NIH.)   Other changes are to inclusion reporting and protection of human subjects. One significant change is the removal of language about children.  Instead of "Inclusion of Children", there will now be a requirement to address "Inclusion Across the Lifespan", which includes children and older adults. 
Unlike the Common Rule, Forms E, and other recent NIH proposal changes, there hasn't been much buzz about these modifications.  Prepare now so you won't be caught by surprise for your winter or spring submission.

Tuesday, January 30, 2018

Training grant human subjects requirements are incorrect in Forms E instructions

According to a recently-released NIH notice, the Forms E instructions incorrectly advise that a human subjects section attachment is required for institutional training (T series) grant proposals.  The correction states that no human subjects attachments are required, even if the box to select "yes" to human subjects is checked on the grants.gov forms.

Tuesday, January 23, 2018

The shutdown is over!

Yesterday, President Trump signed a temporary funding bill that ended the shutdown.  It will keep our government open for approximately another 2 weeks, at which point hopefully an agreement can be made that prevents another shutdown.  For now at least, government employees are back at work and systems are up and running.

Sunday, January 7, 2018

Single IRB policy for multi-site trials

By Metropolitan Transportation Authority of the State of New York (Blizzard of 2015: Empty Grand Central Terminal) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons
The new policy requiring a single IRB for multi-site clinical trials goes into effect on January 25th, 2018.  All of your IRB approvals will now need to be routed through one IRB that will serve as a central hub.  For details, including links to the many previous NIH notices regarding this change, visit this page.

Friday, January 5, 2018

New changes to human subjects mean updated review criteria

NIH has released updated review criteria that will be used to evaluate grant applications proposing clinical trials.  According to NIH, the purpose of the expanded criteria is "to ensure that key pieces of clinical trial-specific information are submitted with each application and that reviewers appropriately consider this clinical trial-related information. Implementing new and more rigorous review criteria for evaluating clinical trial applications will ensure the highest likelihood of translating research results into knowledge that will improve human health."
The full description of the updated clinical trial-specific review criteria and rationale can be accessed here. 

Monday, November 27, 2017

Your final report will now be public

In a recently-released notice, NIH announced that all Project Outcomes sections from interim and final RPPR reports will be made public.  This applies to all submissions starting October 1, 2017.  The reports will be available in the NIH RePORTER.  This is an important step toward showing transparency in publicly-funded research.  It will be interesting to see how it is received by the public.  There has been considerable outcry in the past over NIH-funded research, including studies of sexy goldfish, origami condoms, and poop-throwing monkeys. Although this new policy is based on good intentions, I can't help but wonder if it will hurt scientists whose potential research impact is not obvious to those without particular scientific knowledge.  To help prevent this type of scrutiny, it is important to write your Project Outcomes section in clear language that can be easily understood by the public.

Monday, November 20, 2017

New NIH Research Program - All of Us

NIH is currently beta testing a new initiative called All of Us that is designed to engage 1 million or more volunteers in research.  The All of Us website describes the initiative as follows:
"Creating the right health approaches and care for the right person is called precision medicine. Getting the right information to make that happen is the goal of the All of Us Research Program from the National Institutes of Health (NIH).  To get there, we want to create the largest health data resource ever. By understanding people’s health, neighborhood, family, and lifestyle, researchers will have information to better understand health and disease. This information is essential to create a healthier future for generations to come."
By taking part in this initiative, participants will help to advance the field of precision medicine.  They will take part in activities such as providing blood or urine samples, answering surveys, and allowing access to medical records.  Anyone can join who is not in prison, unable to consent on their own, or younger than 18.
For more information, view the recent press release from NIH here. 

Sunday, November 5, 2017

Planning to apply to NIOSH in February or March? Not so fast...

In a series of recent notices, the Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health (NIOSH) has closed all February and March 2018 application deadlines.  It is anticipated that they will begin accepting R01 and R21 applications again in June and July of 2018.  However, if you are planning a grant proposal submission, it is important to watch this closely, monitor for any new updates, and stay in close contact with NIOSH program staff.

Monday, October 16, 2017

New NIH criteria for review of clinical trial applications

Beginning with the January 25, 2018 deadline, NIH will instruct reviewers to use additional criteria in the evaluation of grant applications that propose clinical trials.  The full policy change notice is located here.

The additional review criteria are as follows:

Significance
Are the scientific rationale and need for a clinical trial to test the proposed hypothesis or intervention well supported by preliminary data, clinical and/or preclinical studies, or information in the literature or knowledge of biological mechanisms? For trials focusing on clinical or public health endpoints, is this clinical trial necessary for testing the safety, efficacy or effectiveness of an intervention that could lead to a change in clinical practice, community behaviors or health care policy?  For trials focusing on mechanistic, behavioral, physiological, biochemical, or other biomedical endpoints, is this trial needed to advance scientific understanding?
Investigator(s) 
With regard to the proposed leadership for the project, do the PD/PI(s) and key personnel have the expertise, experience, and ability to organize, manage and implement the proposed clinical trial and meet milestones and timelines? Do they have appropriate expertise in study coordination, data management and statistics? For a multicenter trial, is the organizational structure appropriate and does the application identify a core of potential center investigators and staffing for a coordinating center?
Innovation 
Does the design/research plan include innovative elements, as appropriate, that enhance its sensitivity, potential for information or potential to advance scientific knowledge or clinical practice?
Approach 
Does the application adequately address the following, if applicable?
Study Design 
Is the study design justified and appropriate to address primary and secondary outcome variable(s)/endpoints that will be clear, informative and relevant to the hypothesis being tested? Is the scientific rationale/premise of the study based on previously well-designed preclinical and/or clinical research? Given the methods used to assign participants and deliver interventions, is the study design adequately powered to answer the research question(s), test the proposed hypothesis/hypotheses, and provide interpretable results? Is the trial appropriately designed to conduct the research efficiently? Are the study populations (size, gender, age, demographic group), proposed intervention arms/dose, and duration of the trial, appropriate and well justified?
Are potential ethical issues adequately addressed? Is the process for obtaining informed consent or assent appropriate? Is the eligible population available? Are the plans for recruitment outreach, enrollment, retention, handling dropouts, missed visits, and losses to follow-up appropriate to ensure robust data collection? Are the planned recruitment timelines feasible and is the plan to monitor accrual adequate? Has the need for randomization (or not), masking (if appropriate), controls, and inclusion/exclusion criteria been addressed? Are differences addressed, if applicable, in the intervention effect due to sex/gender and race/ethnicity?
Are the plans to standardize, assure quality of, and monitor adherence to, the trial protocol and data collection or distribution guidelines appropriate? Is there a plan to obtain required study agent(s)? Does the application propose to use existing available resources, as applicable?
Data Management and Statistical Analysis 
Are planned analyses and statistical approach appropriate for the proposed study design and methods used to assign participants and deliver interventions? Are the procedures for data management and quality control of data adequate at clinical site(s) or at center laboratories, as applicable? Have the methods for standardization of procedures for data management to assess the effect of the intervention and quality control been addressed? Is there a plan to complete data analysis within the proposed period of the award?
Environment 
If proposed, are the administrative, data coordinating, enrollment and laboratory/testing centers, appropriate for the trial proposed?
Does the application adequately address the capability and ability to conduct the trial at the proposed site(s) or centers? Are the plans to add or drop enrollment centers, as needed, appropriate?
If international site(s) is/are proposed, does the application adequately address the complexity of executing the clinical trial?
If multi-sites/centers, is there evidence of the ability of the individual site or center to: (1) enroll the proposed numbers; (2) adhere to the protocol; (3) collect and transmit data in an accurate and timely fashion; and, (4) operate within the proposed organizational structure?

Additional Review Criteria

Study Timeline
Is the study timeline described in detail, taking into account start-up activities, the anticipated rate of enrollment, and planned follow-up assessment? Is the projected timeline feasible and well justified? Does the project incorporate efficiencies and utilize existing resources (e.g., CTSAs, practice-based research networks, electronic medical records, administrative database, or patient registries) to increase the efficiency of participant enrollment and data collection, as appropriate?  Are potential challenges and corresponding solutions discussed (e.g., strategies that can be implemented in the event of enrollment shortfalls)?

Saturday, October 14, 2017

Another video to help you through the new human subjects process

This 9-minute video guides viewers through the various components of the new human subjects form.  Some of this content may also be relevant to research administration personnel.  There is a lot to learn in order to comply with the new policy.

Friday, October 13, 2017

New NIH video to help with human subjects policy changes

NIH recently released a new video tutorial to help applicants understand and comply with the new human subjects research procedures.  These new guidelines will take effect in late January, so if you are planning a proposal submission for the February R01 or other spring and summer deadlines, you must comply with the new policy.  You can view the video here.