Skip to main content
Anonymity
User Portal

Anonymised Review

ANSTO has agreed to participate in an Australian trial of a review of research infrastructure access proposals in which applicants to the Australian Centre for Neutron Scattering supported by the National Deuteration Facility remain anonymous to aid the removal of structural barriers to the career progression of women in STEM.

The trial, which is being coordinated by the Office of the  Australian Government Women in STEM Ambassador, Prof. Lisa Harvey-Smith,  seeks to establish whether unconscious bias in the Australian research sector is adversely affecting the success rate of women in science.

While ANSTO’s preliminary analysis shows that women are not disadvantaged by ANSTO’s review process as success rates are slightly higher than male colleagues, the Australian review overall will provide valuable data in relation to the role of unconscious bias in access allocation processes.

The Australian Centre for Neutron Scattering, with the support of the National Deuteration Facility for joint proposals, is participating in the review. Neutron beam and linked neutron-deuteration submissions will be subject to the Anonymised Review process. Requests to the National Deuteration Facility for deuteration only will not be subject to anonymised review.

ANSTO will be providing support to users for the changes brought about by participation in the Anonymised Review trial and the transition to the ANSTO Research Portal.   Please refer to the ACNS & NDF ARP user guide.

Information sessions will be delivered via zoom at the following dates/times:

  • Wednesday 22 September 2021 – 8am-9am AEST
  • Wednesday 13 October 2021 – 12.30pm-1:30pm AEDT
  • Wednesday 27 October 2021 – 5pm-6pm AEDT
  • Wednesday 10 November 2021 – 8am-9am AEDT
  • Wednesday 24 November 2021 – 5pm-6pm AEDT
  • Wednesday 8 December 2021 – 12pm-1pm AEDT
  • Wednesday 22 December 2021 – 8am-9am AEDT
  • Wednesday 12 January 2022 – 12pm-1pm AEDT
  • Wednesday 19 January 2022 – 5pm-6pm AEDT
  • Thursday 27 January 2022 – 12pm-1pm AEDT
  • Wednesday 2 February 2022 - 8am-9am AEDT
  • Wednesday 9 February 2022 – 5pm-6pm AEDT

Please register your attendance via email.

The Anonymised Review

Isabelle Kingsley of the Office of the Women in STEM Ambassador Research Associate introduces the national research trial to anonymise research funding proposals.

 

Frequently asked questions

How will the process of submitting a proposal to ANSTO be different?

The main difference is that scientific merit reviewers and program advisory committee will not have access to the identity, institution or track record of applicants.  

This also means that the review criteria will not include ‘track record’ as part of the scoring or ranking process.

As part of the study, a member of the Office of the Women in STEM Ambassador will observe the activity of the program advisory committees.

The process for scientific scoring of merit access proposals for the Australian Centre for Neutron Scattering, with or without the support of the National Deuteration Facility,  remains unchanged.

All merit access proposals are also subject to separate technical feasibility and safety reviews. 

Identification and track record information will be included at this stage, in order to adequately assess an applicant’s technical and safety track record. 

Identifying information will not be included in the outcome of the technical reviews, which can be referred to by the program advisory committees who are assessing the overall application.

Guidelines on how to write a technical review without revealing identifying information will be available to reviewers. 

Final approval of proposals after the scientific ranking process will still be subject to the usual considerations, such as the capacity available for the requested capability.

Examples of text that is compliant with anonymous proposal submission.

What does this mean for applicants?

The Australian Centre for Neutron Scattering and National Deuteration Facility 2022-1 round open will open in August 2021 rather than March.   Portal modifications will be made in this period.

These changes are necessary for the anonymised review.  User Guidelines will be available prior to round opening to aid users in the application process. The usual identifying information will be collected but won’t be made available to scientific reviewers or program advisory committee.

A 2-page experimental section of merit access proposal must be written without revealing the identity of the applicant/s. This proposal must be submitted as a PDF.

Guidelines on how to achieve this are available with examples from applications to the Australian Centre for Neutron Scattering.

Are there compliance checks?

Submissions in the anonymised review rounds will be subject to a compliance check against the ‘Anonymity Guidelines for Principal Investigators (or authors)’.  Submissions found not to be compliant will be ‘flagged’ in the data provided to the study organisers.  No changes will be made to any proposals received.

What data is supplied to the study organisers? 

No identifying information nor specifics of the experimental plan are provided to the study research team.  The study  research team receives the following information: 

  • Unique identifier (anonymised) for applicant 

  • Applicant’s institution 

  • Requested beamline/instrument allocation 

  • Gender

  • Program Advisory Committee ranking 

  • Program Advisory Committee recommendation for access and beamline/instrument allocation 

  • Final recommendation for access after capacity review by ANSTO and

  • Approved beamline/instrument allocation 

Do you have any questions?

Can I opt-out?

Partly.  All submissions for the 2022 rounds are expected to comply with the Anonymised Review Guidelines; however, the Principal Investigator may choose not to provide data associated with their proposal to the study. The ANSTO portal for submissions will contain a section where you may choose for your data not to be provided to the study.

Anonymised review process for submissions

ANSTO places a high value on equity and integrity and is participating in a review process that ensures anonymity in which the identities of the proposing team are concealed from reviewers. The goal is to enable reviewers to focus on the science, not the scientist.  

Several studies have shown that a reviewer's attitude toward a submission may be affected, even unconsciously, by the identity of the principal investigator:

Reid, I. N. (2014). Gender-correlated systematics in HST proposal selection. Publications of the Astronomical Society of the Pacific, 126(944), 923
Space Telescope Science Institute (2018). Recommendations of the Working Group on Anonymizing Proposal Reviews
Irish Research Council (2016). Irish Research Council policies and practice to promote gender equality and the integration of gender analysis in research

Anonymised Review Trial Process 

Anyone applying to use an ANSTO capability at the Australian Centre for Neutron Scattering or associated National Deuteration Facility  (2022-1, 2022-2 rounds) is required to make their submission anonymous following the guidelines below.  

Please take special care if resubmitting a proposal from a previous cycle or other submission. 

Submissions must include the names and affiliations of all investigators in the usual portal fields; however, ANSTO will not include names or affiliations in the versions for assessments. 

The Program Advisory Committee will examine the proposals and finalise scoring and ranking based on reviews of the anonymised submissions and their own assessments of the proposals.  

After the review process, identifying information associated with a proposal is no longer restricted and is available on request if needed to finalise a ranking.  Should this be done, it will be recorded in the report of the meeting. 

Anonymity Guidelines for Principal Investigators (or authors)

Don’t "water down" or obscure your science, your methods, or your tools; simply write about your work in the third person, in a way that does not intentionally identify the applicants.

These guidelines will help conceal the identities of the applicant and ensure a fairer proposal evaluation process.

  1. Do not include author names or affiliations anywhere in the experimental text
    This includes but is not limited to, page headers, footers, diagrams, figures, or watermarks.
  2. When citing references within the proposal, use third person neutral wording
    This especially applies to self-referencing. For example, replace phrases like “as we have shown in our previous work (Doe et al. 2010)” with “as Doe et al. (2010) showed...
  3. Do not refer to previous projects using language that reveals the identity of the applicant(s)
    For instance, rather than write "we observed another cluster, similar to the one we are proposing under _____ program #XXXXX," instead write "_____ program #XXXXX has observed this target in the past..."
  4. Use references to published work, including work citable by a DOI, without including information that that may reveal the identity of the applicant(s)
    If you cite exclusive access datasets or non-public software that may reveal or strongly imply the investigators on the proposal, use language like "obtained in private communication" or "from private consultation".
  5. Do not include acknowledgments or the source of any grant funding in the experimental section of the submission
  6. Team expertise and background is provided as supplementary information and should not be made available in the experimental section of the submission

Examples of compliant proposals 

Platypus P1775

Kowari P1792

Echidna P2378

Examples of compliant proposals from Space Telescope Science Institute 

Calender

Timeline

Date
Australian Centre for Neutron Scattering and associated National Deuteration Facility 
Aug 2021
Portal and User guidelines available
Aug 2021
2022-1 anonymized review round opens
Aug 2021 -
Feb 2022
Regular information sessions on application to anon-review@ansto.gov.au
Sep 2021
2022 anonymised review round submissions extended to Feb 2022
Feb 2022
2022 round closes 15 February
Apr 2022
Program Advisory Committee Meeting
May 2022
Feedback Survey on anonymsied review process and User Office support