ANSTO has agreed to participate in an Australian trial of a review of research infrastructure access proposals in which applicants to the Australian Centre for Neutron Scattering supported by the National Deuteration Facility remain anonymous to aid the removal of structural barriers to the career progression of women in STEM.
The trial, which is being coordinated by the Office of the Australian Government Women in STEM Ambassador, Prof. Lisa Harvey-Smith, seeks to establish whether unconscious bias in the Australian research sector is adversely affecting the success rate of women in science.
While ANSTO’s preliminary analysis shows that women are not disadvantaged by ANSTO’s review process as success rates are slightly higher than male colleagues, the Australian review overall will provide valuable data in relation to the role of unconscious bias in access allocation processes.
The Australian Centre for Neutron Scattering, with the support of the National Deuteration Facility for joint proposals, is participating in the review. Neutron beam and linked neutron-deuteration submissions will be subject to the Anonymised Review process. Requests to the National Deuteration Facility for deuteration only will not be subject to anonymised review.
ANSTO will be providing support to users for the changes brought about by participation in the Anonymised Review trial and the transition to the ANSTO Research Portal. Please refer to the ACNS & NDF ARP user guide.
The Anonymised Review
Isabelle Kingsley of the Office of the Women in STEM Ambassador Research Associate introduces the national research trial to anonymise research funding proposals.
Frequently asked questions
How will the process of submitting a proposal to ANSTO be different?
The main difference is that scientific merit reviewers and program advisory committee will not have access to the identity, institution or track record of applicants.
This also means that the review criteria will not include ‘track record’ as part of the scoring or ranking process.
As part of the study, a member of the Office of the Women in STEM Ambassador will observe the activity of the program advisory committees.
The process for scientific scoring of merit access proposals for the Australian Centre for Neutron Scattering, with or without the support of the National Deuteration Facility, remains unchanged.
All merit access proposals are also subject to separate technical feasibility and safety reviews.
Identification and track record information will be included at this stage, in order to adequately assess an applicant’s technical and safety track record.
Identifying information will not be included in the outcome of the technical reviews, which can be referred to by the program advisory committees who are assessing the overall application.
Guidelines on how to write a technical review without revealing identifying information will be available to reviewers.
Final approval of proposals after the scientific ranking process will still be subject to the usual considerations, such as the capacity available for the requested capability.
Examples of text that is compliant with anonymous proposal submission.
What does this mean for applicants?
The Australian Centre for Neutron Scattering and National Deuteration Facility 2022-1 round open will open in August 2021 rather than March. Portal modifications will be made in this period.
These changes are necessary for the anonymised review. User Guidelines will be available prior to round opening to aid users in the application process. The usual identifying information will be collected but won’t be made available to scientific reviewers or program advisory committee.
A 2-page experimental section of merit access proposal must be written without revealing the identity of the applicant/s. This proposal must be submitted as a PDF.
Guidelines on how to achieve this are available with examples from applications to the Australian Centre for Neutron Scattering.
Are there compliance checks?
Submissions in the anonymised review rounds will be subject to a compliance check against the ‘Anonymity Guidelines for Principal Investigators (or authors)’. Submissions found not to be compliant will be ‘flagged’ in the data provided to the study organisers. No changes will be made to any proposals received.
What data is supplied to the study organisers?
No identifying information nor specifics of the experimental plan are provided to the study research team. The study research team receives the following information:
Unique identifier (anonymised) for applicant
Applicant’s institution
Requested beamline/instrument allocation
Gender
Program Advisory Committee ranking
Program Advisory Committee recommendation for access and beamline/instrument allocation
Final recommendation for access after capacity review by ANSTO and
Approved beamline/instrument allocation
What if I have a complaint or any concerns about the research study?
If you have a complaint regarding any aspect of the study or the way it is being conducted, please contact the UNSW Human Ethics Coordinator:
Complaints Contact Position UNSW Human Research Ethics Coordinator Telephone + 61 2 9385 6222 Email: humanethics@unsw.edu.au HC Reference Number
HC200129
What should I do if I have further questions about my involvement in the research study?
If you require further information regarding this study or if you have any problems which may be related to your involvement in the study, you can contact the Principal Investigator:
Isabelle Kingsley
Chief Investigator
Position Research Associate
Telephone 02 9385 5218
Email isabelle.kingsley@unsw.edu.au"
Do you have any questions?
Can I opt-out?
Partly. All submissions for the 2022 rounds are expected to comply with the Anonymised Review Guidelines; however, the Principal Investigator may choose not to provide data associated with their proposal to the study. The ANSTO portal for submissions will contain a section where you may choose for your data not to be provided to the study.
Anonymised review process for submissions
ANSTO places a high value on equity and integrity and is participating in a review process that ensures anonymity in which the identities of the proposing team are concealed from reviewers. The goal is to enable reviewers to focus on the science, not the scientist.
Several studies have shown that a reviewer's attitude toward a submission may be affected, even unconsciously, by the identity of the principal investigator:
Reid, I. N. (2014). Gender-correlated systematics in HST proposal selection. Publications of the Astronomical Society of the Pacific, 126(944), 923
Space Telescope Science Institute (2018). Recommendations of the Working Group on Anonymizing Proposal Reviews
Irish Research Council (2016). Irish Research Council policies and practice to promote gender equality and the integration of gender analysis in research
Anonymised Review Trial Process
Anyone applying to use an ANSTO capability at the Australian Centre for Neutron Scattering or associated National Deuteration Facility (2022-1, 2022-2 rounds) is required to make their submission anonymous following the guidelines below.
Please take special care if resubmitting a proposal from a previous cycle or other submission.
Submissions must include the names and affiliations of all investigators in the usual portal fields; however, ANSTO will not include names or affiliations in the versions for assessments.
The Program Advisory Committee will examine the proposals and finalise scoring and ranking based on reviews of the anonymised submissions and their own assessments of the proposals.
After the review process, identifying information associated with a proposal is no longer restricted and is available on request if needed to finalise a ranking. Should this be done, it will be recorded in the report of the meeting.
Anonymity Guidelines for Principal Investigators (or authors)
Don’t "water down" or obscure your science, your methods, or your tools; simply write about your work in the third person, in a way that does not intentionally identify the applicants.
These guidelines will help conceal the identities of the applicant and ensure a fairer proposal evaluation process.
- Do not include author names or affiliations anywhere in the experimental text
This includes but is not limited to, page headers, footers, diagrams, figures, or watermarks. - When citing references within the proposal, use third person neutral wording
This especially applies to self-referencing. For example, replace phrases like “as we have shown in our previous work (Doe et al. 2010)” with “as Doe et al. (2010) showed...” - Do not refer to previous projects using language that reveals the identity of the applicant(s)
For instance, rather than write "we observed another cluster, similar to the one we are proposing under _____ program #XXXXX," instead write "_____ program #XXXXX has observed this target in the past..." - Use references to published work, including work citable by a DOI, without including information that that may reveal the identity of the applicant(s)
If you cite exclusive access datasets or non-public software that may reveal or strongly imply the investigators on the proposal, use language like "obtained in private communication" or "from private consultation". - Do not include acknowledgments or the source of any grant funding in the experimental section of the submission
- Team expertise and background is provided as supplementary information and should not be made available in the experimental section of the submission
Examples of compliant proposals
Platypus P1775
Kowari P1792
Echidna P2378
Examples of compliant proposals from Space Telescope Science Institute
Timeline
Date | Australian Centre for Neutron Scattering and associated National Deuteration Facility |
|---|---|
Aug 2021 | Portal and User guidelines available |
Aug 2021 | 2022-1 anonymized review round opens |
Aug 2021 - | Regular information sessions on application to anon-review@ansto.gov.au |
Sep 2021 | 2022 anonymised review round submissions extended to Feb 2022 |
Feb 2022 | 2022 round closes 15 February |
19-20 May 2022 | Program Advisory Committee Meeting |
30 May | Notifications to Users |
June 2022 | Feedback Survey on anonymsied review process and User Office support |