ANSTO has agreed to participate in an Australian trial of a review of research infrastructure access proposals in which applicants to the Australian Centre for Neutron Scattering supported by the National Deuteration Facility remain anonymous to aid the removal of structural barriers to the career progression of women in STEM.
The trial, which is being coordinated by the Office of the Australian Government Women in STEM Ambassador, Prof. Lisa Harvey-Smith, seeks to establish whether unconscious bias in the Australian research sector is adversely affecting the success rate of women in science.
While ANSTO’s preliminary analysis shows that women are not disadvantaged by ANSTO’s review process as success rates are slightly higher than male colleagues, the Australian review overall will provide valuable data in relation to the role of unconscious bias in access allocation processes.
The Australian Centre for Neutron Scattering, with the support of the National Deuteration Facility for joint proposals, is participating in the review. Neutron beam and linked neutron-deuteration submissions will be subject to the Anonymised Review process. Requests to the National Deuteration Facility for deuteration only will not be subject to anonymised review.
ANSTO will be providing support to users for the changes brought about by participation in the Anonymised Review trial and the transition to the ANSTO Research Portal. Please refer to the ACNS & NDF ARP user guide.
Information sessions will be delivered via zoom at the following dates/times:
- Wednesday 22 September 2021 – 8am-9am AEST
- Wednesday 13 October 2021 – 12.30pm-1:30pm AEDT
- Wednesday 27 October 2021 – 5pm-6pm AEDT
- Wednesday 10 November 2021 – 8am-9am AEDT
- Wednesday 24 November 2021 – 5pm-6pm AEDT
- Wednesday 8 December 2021 – 12pm-1pm AEDT
- Wednesday 22 December 2021 – 8am-9am AEDT
- Wednesday 12 January 2022 – 12pm-1pm AEDT
- Wednesday 19 January 2022 – 5pm-6pm AEDT
- Thursday 27 January 2022 – 12pm-1pm AEDT
- Wednesday 2 February 2022 - 8am-9am AEDT
- Wednesday 9 February 2022 – 5pm-6pm AEDT
Please register your attendance via email.
The Anonymised Review
Isabelle Kingsley of the Office of the Women in STEM Ambassador Research Associate introduces the national research trial to anonymise research funding proposals.
Frequently asked questions
How will the process of submitting a proposal to ANSTO be different?
The main difference is that scientific merit reviewers and program advisory committee will not have access to the identity, institution or track record of applicants.
This also means that the review criteria will not include ‘track record’ as part of the scoring or ranking process.
As part of the study, a member of the Office of the Women in STEM Ambassador will observe the activity of the program advisory committees.
The process for scientific scoring of merit access proposals for the Australian Centre for Neutron Scattering, with or without the support of the National Deuteration Facility, remains unchanged.
All merit access proposals are also subject to separate technical feasibility and safety reviews.
Identification and track record information will be included at this stage, in order to adequately assess an applicant’s technical and safety track record.
Identifying information will not be included in the outcome of the technical reviews, which can be referred to by the program advisory committees who are assessing the overall application.
Guidelines on how to write a technical review without revealing identifying information will be available to reviewers.
Final approval of proposals after the scientific ranking process will still be subject to the usual considerations, such as the capacity available for the requested capability.
Examples of text that is compliant with anonymous proposal submission.
What does this mean for applicants?
The Australian Centre for Neutron Scattering and National Deuteration Facility 2022-1 round open will open in August 2021 rather than March. Portal modifications will be made in this period.
These changes are necessary for the anonymised review. User Guidelines will be available prior to round opening to aid users in the application process. The usual identifying information will be collected but won’t be made available to scientific reviewers or program advisory committee.
A 2-page experimental section of merit access proposal must be written without revealing the identity of the applicant/s. This proposal must be submitted as a PDF.
Guidelines on how to achieve this are available with examples from applications to the Australian Centre for Neutron Scattering.
Are there compliance checks?
Submissions in the anonymised review rounds will be subject to a compliance check against the ‘Anonymity Guidelines for Principal Investigators (or authors)’. Submissions found not to be compliant will be ‘flagged’ in the data provided to the study organisers. No changes will be made to any proposals received.
What data is supplied to the study organisers?
No identifying information nor specifics of the experimental plan are provided to the study research team. The study research team receives the following information:
Unique identifier (anonymised) for applicant
Requested beamline/instrument allocation
Program Advisory Committee ranking
Program Advisory Committee recommendation for access and beamline/instrument allocation
Final recommendation for access after capacity review by ANSTO and
Approved beamline/instrument allocation
Can I opt-out?
Partly. All submissions for the 2022 rounds are expected to comply with the Anonymised Review Guidelines; however, the Principal Investigator may choose not to provide data associated with their proposal to the study. The ANSTO portal for submissions will contain a section where you may choose for your data not to be provided to the study.
Anonymised review process for submissions
ANSTO places a high value on equity and integrity and is participating in a review process that ensures anonymity in which the identities of the proposing team are concealed from reviewers. The goal is to enable reviewers to focus on the science, not the scientist.
Several studies have shown that a reviewer's attitude toward a submission may be affected, even unconsciously, by the identity of the principal investigator:
Reid, I. N. (2014). Gender-correlated systematics in HST proposal selection. Publications of the Astronomical Society of the Pacific, 126(944), 923
Space Telescope Science Institute (2018). Recommendations of the Working Group on Anonymizing Proposal Reviews
Irish Research Council (2016). Irish Research Council policies and practice to promote gender equality and the integration of gender analysis in research
Anonymised Review Trial Process
Anyone applying to use an ANSTO capability at the Australian Centre for Neutron Scattering or associated National Deuteration Facility (2022-1, 2022-2 rounds) is required to make their submission anonymous following the guidelines below.
Please take special care if resubmitting a proposal from a previous cycle or other submission.
Submissions must include the names and affiliations of all investigators in the usual portal fields; however, ANSTO will not include names or affiliations in the versions for assessments.
The Program Advisory Committee will examine the proposals and finalise scoring and ranking based on reviews of the anonymised submissions and their own assessments of the proposals.
After the review process, identifying information associated with a proposal is no longer restricted and is available on request if needed to finalise a ranking. Should this be done, it will be recorded in the report of the meeting.
Anonymity Guidelines for Principal Investigators (or authors)
Don’t "water down" or obscure your science, your methods, or your tools; simply write about your work in the third person, in a way that does not intentionally identify the applicants.
These guidelines will help conceal the identities of the applicant and ensure a fairer proposal evaluation process.
- Do not include author names or affiliations anywhere in the experimental text
This includes but is not limited to, page headers, footers, diagrams, figures, or watermarks.
- When citing references within the proposal, use third person neutral wording
This especially applies to self-referencing. For example, replace phrases like “as we have shown in our previous work (Doe et al. 2010)” with “as Doe et al. (2010) showed...”
- Do not refer to previous projects using language that reveals the identity of the applicant(s)
For instance, rather than write "we observed another cluster, similar to the one we are proposing under _____ program #XXXXX," instead write "_____ program #XXXXX has observed this target in the past..."
- Use references to published work, including work citable by a DOI, without including information that that may reveal the identity of the applicant(s)
If you cite exclusive access datasets or non-public software that may reveal or strongly imply the investigators on the proposal, use language like "obtained in private communication" or "from private consultation".
- Do not include acknowledgments or the source of any grant funding in the experimental section of the submission
- Team expertise and background is provided as supplementary information and should not be made available in the experimental section of the submission
Examples of compliant proposals
Australian Centre for Neutron Scattering and associated National Deuteration Facility
Portal and User guidelines available
2022-1 anonymized review round opens
Aug 2021 -
Regular information sessions on application to email@example.com
2022 anonymised review round submissions extended to Feb 2022
2022 round closes 15 February
Program Advisory Committee Meeting
Feedback Survey on anonymsied review process and User Office support