When AI plays favourites: How algorithmic bias shapes the hiring process
By Mehnaz Rafi, PhD Candidate, Haskayne School of Business, University of Calgary,
2 days ago
A public interest group filed a U.S. federal complaint against artificial intelligence hiring tool, HireVue , in 2019 for deceptive hiring practices. The software, which has been adopted by hundreds of companies, favoured certain facial expressions, speaking styles and tones of voice, disproportionately disadvantaging minority candidates.
Similarly, Amazon stopped using its AI recruitment tool , as reported in 2018, after discovering it was biased against women . The algorithm, trained on male-dominated resumes submitted over 10 years, favoured male candidates by downgrading applications that included the word “women’s” and penalizing graduates of women’s colleges. Engineers tried to address these biases, but could not guarantee neutrality, leading to the project’s cancellation.
As companies increasingly rely on algorithms to make critical hiring decisions, it’s crucial to be aware of the following ways AI can create bias in hiring :
1. Bias in training data. AI systems rely on large datasets — referred to as training data — to learn patterns and make decisions, but their accuracy and fairness are only as good as the data they are trained on. If this data contains historical hiring biases that favour specific demographics, the AI will adopt and reproduce those same biases . Amazon’s AI tool, for example, was trained on resumes from a male-dominated industry, which led to gender bias.
2. Flawed data sampling. Flawed data sampling occurs when the dataset used to train an algorithm is not representative of the broader population it’s meant to serve. In the context of hiring, this can happen if training data over-represents certain groups —typically white men — while under-representing marginalized candidates.
For example, AI might disproportionately value graduates from prestigious universities, which have historically been attended by people from privileged backgrounds . Or, it might prioritize work experiences that are more common among certain demographics.
4. Lack of transparency. Many AI systems function as “ black boxes ,” meaning their decision-making processes are opaque. This lack of transparency makes it difficult for organizations to identify where bias might exist and how it affects hiring decisions.
Without insight into how an AI tool makes decisions, it’s difficult to correct biased outcomes or ensure fairness. Both Amazon and HireVue faced this issue; users and developers struggled to understand how the systems assessed candidates and why certain groups were excluded.
5. Lack of human oversight. While AI plays an important role in many decision-making processes, it should augment, rather than replace, human judgment. Over-reliance on AI without adequate human oversight can lead to unchecked biases. This problem is exacerbated when hiring professionals trust AI more than their own judgment, believing in the technology’s infallibility.
Overcoming algorithmic bias in hiring
To mitigate these issues, companies must adopt strategies that prioritize inclusivity and transparency in AI-driven hiring processes. Below are some key solutions for overcoming AI bias:
2. Conduct regular bias audits. Frequent and thorough audits of AI systems should be conducted to identify patterns of bias and discrimination. This includes examining the algorithm’s outputs, decision-making processes and its impact on different demographic groups.
3. Implement fairness-aware algorithms. Use AI software that incorporates fairness constraints and is designed to consider and mitigate bias by balancing outcomes for underrepresented groups. This can include integrating fairness metrics such as equal opportunity, modifying training data to show less bias and adjusting model predictions based on fairness criteria to increase equity.
4. Increase transparency. Seek AI solutions that offer insight into their algorithms and decision-making processes to make it easier to identify and address potential biases. Additionally, make sure to disclose any use of AI in the hiring process to candidates to maintain transparency with your job applicants and other stakeholders.
5. Maintain human oversight. To maintain control over hiring algorithms, managers and leaders must actively review AI-driven decisions, especially when making final hiring choices. Emerging research highlights the critical role of human oversight in safeguarding against the risks posed by AI applications. However, for this oversight to be effective and meaningful, leaders must ensure that ethical considerations are part of the hiring process and promote the responsible, inclusive and ethical use of AI.
Bias in hiring algorithms raises serious ethical concerns and demands greater attention toward the mindful, responsible and inclusive use of AI. Understanding and addressing the ethical considerations and biases of AI-driven hiring is essential to ensuring fairer hiring outcomes and preventing technology from reinforcing systemic bias.
Mehnaz Rafi does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.
Get updates delivered to you daily. Free and customizable.
It’s essential to note our commitment to transparency:
Our Terms of Use acknowledge that our services may not always be error-free, and our Community Standards emphasize our discretion in enforcing policies. As a platform hosting over 100,000 pieces of content published daily, we cannot pre-vet content, but we strive to foster a dynamic environment for free expression and robust discourse through safety guardrails of human and AI moderation.
Comments / 0