Guidance material for using AI-assisted recruitment tools
AI-assisted recruitment myths
With the increase of technology, Artificial Intelligence (AI) and automation have found a home within some recruitment processes. Here we take a look at some of the myths surrounding the use of AI and automation.
Myth 1: All AI-assisted and automated tools on the market have been thoroughly tested.
Artificial Intelligence-assisted (AI-assisted) and automated recruitment technologies have been tested in different ways and to varying degrees. There are limited national and international guidelines on the development of AI-assisted and automated recruitment tools, meaning that the quality of AI-assisted assessments can vary significantly.
Myth 2: AI-assisted tools are guaranteed to be completely unbiased.
AI can reproduce the bias of the developers. For example, developers may only test the AI on certain population demographics, meaning that the tool may disadvantage diversity cohorts. AI-assisted tools can also contain algorithmic bias that does not reflect the true suitability of a candidate for the role. For example, if an AI is based on a dataset of interviews where all successful candidates cough during the interview, the AI may determine that coughing makes a candidate suitable for the position.
Myth 3: Agencies are not accountable for the decisions that AI makes.
Agencies are accountable for ensuring their recruitment processes follow the merit principle as outlined in section 10A of the Public Service Act 1999 (Public Service Act).Agencies must be able to demonstrate the effectiveness of the AI-assisted tool in assessing candidates in accordance with criteria relevant to the position description. Those criteria must reflect the work-related qualities genuinely required to perform the relevant duties of the role to meet the merit principle.
AI and automation in recruitment
Advances in technology and a tightening labour market have led Australian Public Sector (APS) agencies to increasingly utilise AI-assisted and automated recruitment tools in their recruitment processes. For the purposes of this guidance material 'AI-assisted and automated tools' will refer to any recruitment tools that aim to minimise or remove direct human input. Types of AI-assisted and automated tools include resume scanners, video interviews or psychometric tests which are reviewed by AI or an automated process.
Of 66 APS agencies surveyed in 2022, 15 (23%) responded that they had used AI-assisted and automated tools in their recruitment processes in the last 12 months. As AI-assisted and automated recruitment tools are expected to become more prevalent in the future, providing guidance on how best to use these tools is crucial in order to ensure that recruitment processes continue to meet APS employment principles, particularly merit.
AI-assisted and automated recruitment tools may enable agencies to increase the efficiency of their recruitment processes whilst mitigating some forms of recruitment bias. However, there are also a number of risks associated with AI-assisted and automated recruitment tools which may impact on the ultimate fairness and effectiveness of a recruitment process.
In particular, incorrect or negligent use of AI-assisted and automated recruitment tools can impede the operation of a merit-based recruitment process. Agencies should exercise care when engaging these tools, in order to uphold the merit principle. There should be a clear demonstrated connection between the candidate's qualities being assessed and the qualities required to perform the duties of the job.
This guidance material advises agencies on engaging AI-assisted and automated tools to:
- mitigate risks
- maximise benefits; and
- ensure the merit principle outlined in section 10A of the Public Service Act is upheld.
Should I use AI in my agency's recruitment process?
AI tools are designed to minimise human input by replicating human-levels of decision-making. Commonly used AI-assisted recruitment tools use machine learning models. Supervised machine learning models are trained on data that has been assigned value, i.e. 'labelled data' (for example, positive and negative values). The AI stores this as knowledge and then interprets new information based on the assigned values in the base data. An 'algorithm' is the unique way in which information is processed and determines the way in which the AI makes decisions. Understanding how AI works can take years of training and expertise. Recruiters should note the risks outlined below, as well as the benefits associated with utilising AI-assisted recruitment tools.
Benefits of AI-assisted and automated recruitment tools
The use of AI in recruitment processes has the potential to:
- Substantially reduce the time and resources required to complete a recruitment process.
- Reduce various types of bias associated with traditional recruitment tools.
- Provide more consistent assessments of all candidates across the entire recruitment process.
- Enhance the candidate's experience of the recruitment process.
Among APS agencies that used AI in recruitment, the reduction in time spent on recruitment processes was the most common benefit cited for using these tools.
Risks of AI-assisted and automated recruitment tools
If used incorrectly, AI-assisted recruitment processes may:
- Assess candidates on something other than merit due to poorly developed algorithms or poor-quality data.
- Raise ethical and legal concerns on issues such as transparency and data privacy.
- Lead to automation bias, when recruitment teams over-rely on automated decisions and disregard information which contradicts AI results.
- Produce biased results due to:
- Poor quality data, for example, having AI trained on data which excludes or under represents particular demographic groups. As representative data reflects societal inequalities, AI may reproduce this inequality when making decisions.
- Algorithms reflecting the biases of their developers
- Cause statistical bias when characteristics of candidates such as socioeconomic background are erroneously deemed as indicative of candidate success.
Among APS agencies that discontinued use of AI tools,most indicated 'tools not working as intended' as the primary reason.
Procuring AI-assisted and automated tools: choosing the right provider
Please take into account the following considerations when comparing an AI-assisted or automated tool, or provider.
Questions to consider:
- Have you engaged with several prospective AI providers?
- How does the information from each provider compare to others on the market?
It is important that you engage with several AI providers to compare the information you are given from each, to find an AI-assisted tool which has been thoroughly tested and is appropriate for the recruitment process in question. You have to be satisfied that this selection technique will reliably assess the particular candidate qualities you are wanting to assess.
Note that much of the immediately accessible research on AI-assisted tools is sponsored by the provider themselves. Such research should be carefully reviewed and scrutinised. You should also consider reviewing research from independent sources, such as peer-reviewed academic articles or independent reviews from government jurisdictions.
Consider the transparency of a provider
Questions to consider:
- Does the AI-assisted tool provider whose services are being considered have an in-depth guide available which explains the criteria being assessed?>/li>
- Why is that criteria being assessed for the role? How does the AI make decisions?
Outsourcing elements of the recruitment process also means outsourcing parts of the decision-making process. You must therefore ensure that you understand how recruitment decisions are being made and are confident the selection techniques selected are assessing on the basis of merit and not some other criteria. Although outsourced, your agency remains responsible for ensuring the recruitment is compliant with merit, as defined in the Public Service Act.
Assess the effectiveness of the tool/s
Questions to consider:
- Can the provider give long-term evidence, over a variety of contexts, that the AI-assisted or automated tool measures what it claims to measure?
- Does this translate to long-term success, and how does the provider measure success?
AI-assisted tools have emerged from recent technological innovation and some tools may not have been scientifically validated to the same extent as more traditional recruitment techniques.
Consider whether your AI provider can provide long-term evidence that the AI-assisted tool measures accurately for the key criteria you aim to assess. Agencies should consider, particularly with first and early use, developing an internal monitoring and evaluation process to assess the effectiveness of your AI-assisted tools, including whether your tools may disadvantage diversity cohorts.
Assess how the tool/s address the risk of bias
Questions to consider:
- Has the tool been tested with various diversity cohorts in an Australian audience?
For example, has it been tested with people with disability? First Nations people?
AI is trained on a combination of labels and data. If the AI developers are biased, and/or if the data is unrepresentative, the AI will produce biased results. If the tool has not been tested on certain cohorts, these cohorts may be disadvantaged by the AI algorithms.
Understand how the tool has been tested on diversity groups, and seek information on whether diversity groups are disadvantaged by the tool.
Ensure that the tool/s can be tailored
- To what extent can the tool be tailored to the specific needs of your agency and the role being advertised?
It is important that an AI-assisted or automated tool assesses the relevant criteria for a particular position. Consider the extent that you can collaborate with the provider to create a tailored assessment system that measures the relevant criteria. It is also important that the AI-assisted or automated tools are applicable to a public-sector context and meets APS standards.
Investigate the privacy policies of the tool/s
Using AI-assisted and automated tools
Please take into account the following considerations when using an AI-assisted or automated tool in your recruitment.
- Does your team have access to the skills needed to fully answer the questions under the 'procuring AI-assisted and automated tools' section?
When using AI-assisted tools, it is recommended that agencies have in-house subject matter expertise. A data science background may be required to fully understand how the AI has been trained, the data the AI has been trained on, and how the AI makes decisions. A background in organisational psychology may be needed to assess whether the criteria is relevant to the vacant position. Also consider hiring a third party to evaluate the AI-assisted or automated tools used by your agency.
Educate staff on the risk of bias
- Does everyone involved in the recruitment process have a basic understanding of the risks of AI?
AI can produce biased recruitment decisions. For example, AI can reproduce the bias of developers if the tool has only been tested on a limited demographic range. AI can also contain algorithmic bias, such as picking up on similarities between successful candidates that don't necessarily correlate with job performance.
Educate staff on the risks of these technologies to help make sure employees don't over-rely on automated processes to make decisions. Education may also encourage staff to take steps to mitigate biased decisions.
Minimise the use of hard screening decisions
- Have you considered using AI as one part of a holistic assessment process?
Minimise the use of hard screening decisions, where candidates are screened out on the outcomes of a single AI-assisted assessment. Using a number of different tools, including AI-assisted and automated tools, as well as more traditional tools to assess candidates and give a holistic ranking will further reduce the risks of AI or automation. Having humans actively involved in decisions at different stages to review the AI's recruitment choices is beneficial.
Protect candidates' right to privacy
- How are candidates' information stored?
Before using an AI-assisted tool, ensure candidate consent is obtained and that storage and collection of data is minimised to necessary information and compliant with privacy obligations.
- How are you evaluating the effectiveness of the AI-assisted or automated tool?
AI-assisted and automated tools should be evaluated to ensure they are working effectively. When a new AI-assisted recruitment tool is introduced to a recruitment process, it can be run alongside previous tools to determine if the tool is producing reliable outcomes. Consider evaluating the long-term effectiveness of the tool in determining suitable candidates.
If you are considering using AI-assisted tools in your next recruitment round, consider the following key questions:
- Have I researched a variety of AI providers?
- How transparent are the AI-assisted tools that I am planning to use?
- Does the AI-assisted tool effectively assess the criteria that I hope to measure?
- Has the AI-assisted tool been tested with diverse cohorts?
- How can I support my staff to use AI effectively?
- How can I use AI in a way that minimises the risks of AI while maximising the benefits?
- Is the collection of information compliant with privacy obligations?
If you have questions regarding the application of the merit in recruitment, please contact the office of the Merit Protection Commissioner at email@example.com.
The Merit Protection Commissioner would like to thank each of the 2022 graduates of Australian Public Service Commission's graduate program, for all their hard work in researching and developing this guidance
Looking for more guidance?
Our Guidance section contains guidance from all aspects of the Merit Protection Commission's work.