Reliable Evaluation of Automated Resume Screening Through Leakage Control, Information Retrieval, and Bias Auditing
DOI:
https://doi.org/10.64137/31079911/IJMST-V2I4P101Keywords:
Resume Screening, Leakage Control, Information Retrieval, Text Mining, TF-IDF, BM25, Latent Semantic Analysis, Sentence-BERT, NDCG, Bias Auditing, Counterfactual Fairness Audit, ReproducibilityAbstract
Resume screening software is widely used when employers receive far more applications than people can read, yet much of the published evidence for it is hard to trust. Models are often scored against labels derived from the very text they read, ranking quality is rarely measured, and a system is called fair without ever being tested. This paper evaluates resume screening along three lines: leakage control, information retrieval and bias auditing. We use the public Kaggle Resume Dataset (2,481 resumes in 24 occupational categories after cleaning) and compare five lexical classifiers and one Sentence-BERT classifier under stratified five-fold cross-validation, fitting the vectoriser on training folds only. The title line at the top of each resume turns out to leak the label. With it removed, the accuracy of the lexical models falls between 0.571 and 0.652 and macro-averaged F1 never exceeds 0.594. A stacked ensemble is not significantly better than plain logistic regression, and Sentence-BERT with logistic regression is lower still (0.595 accuracy, 0.560 macro-averaged F1). For retrieval we score 24 job-description queries against graded relevance. Latent semantic analysis gives the best NDCG@10 (0.746), ahead of BM25, TF-IDF cosine and Sentence-BERT, but none of these gaps survives correction for multiple comparisons. In a counterfactual audit, swapping male and female names and pronouns changes 0.23% of predictions on average. Because the dataset carries no demographic labels, this cannot establish fairness, and we spell out what a proper audit requires. Our figures sit far below the near-perfect scores often reported on rule-labelled data and offer a more honest baseline for later work.
References
[1] J. Sánchez-Monedero, L. Dencik, and L. Edwards, “What does it mean to ‘solve’ the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems,” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, 2020, pp. 458–468, Jan 2020, doi: 10.1145/3351095.3372849
[2] European Parliament and Council of the European Union, “Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act),” Official Journal of the European Union, L series, Jul. 12, 2024. [Online]. Available: http://data.europa.eu/eli/reg/2024/1689/oj
[3] New York City Department of Consumer and Worker Protection, “Automated Employment Decision Tools (AEDT): Local Law 144 of 2021.” [Online]. Available: https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page
[4] J. Dastin, “Amazon scraps secret AI recruiting tool that showed bias against women,” Reuters, 2018. [Online]. Available: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G/
[5] M. Bogen and A. Rieke, “Help wanted: An examination of hiring algorithms, equity, and bias,” Upturn, Washington, DC, USA, Dec. 2018. [Online]. Available: https://www.upturn.org/work/help-wanted/
[6] M. Raghavan, S. Barocas, J. Kleinberg, and K. Levy, “Mitigating bias in algorithmic hiring: Evaluating claims and practices,” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, vol. 1, no. 1, pp. 469–481, 2020, doi: 10.1145/3351095.3372828
[7] S. Kaufman, S. Rosset, C. Perlich, and O. Stitelman, “Leakage in data mining: Formulation, detection, and avoidance,” ACM Transactions on Knowledge Discovery from Data, vol. 6, no. 4, pp. 1–21, 2012, doi: 10.1145/2382577.2382579
[8] S. Kapoor, and A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science,” Patterns, vol. 4, no. 9, pp. 100804–100804, Aug. 2023, doi: 10.1016/j.patter.2023.100804.
[9] S. Bhawal, “Resume Dataset,” Kaggle. [Online]. Available: https://www.kaggle.com/datasets/snehaanbhawal/resume-dataset
[10] T. Joachims, “Text Categorization With Support Vector Machines: Learning With Many Relevant Features,” Machine Learning: ECML-98, vol. 1398, pp. 137–142, 1998, doi: 10.1007/bfb0026683
[11] J. D. M. Rennie, L. Shih, J. Teevan, and D. R. Karger, “Tackling the poor assumptions of naive Bayes text classifiers,” ICML'03: Proceedings of the Twentieth International Conference on International Conference on Machine Learning, Washington, DC, USA, pp. 616–623, 2003.
[12] L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324
[13] M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Information Processing and Management, vol. 45, no. 4, pp. 427–437, 2009, doi: 10.1016/j.ipm.2009.03.002
[14] G. Salton and C. Buckley, “Term-weighting approaches in automatic text retrieval,” Information Processing and Management, vol. 24, no. 5, pp. 513–523, 1988, doi: 10.1016/0306-4573(88)90021-0
[15] S. Robertson and H. Zaragoza, “The probabilistic relevance framework: BM25 and beyond,” Found. Trends Inf. Retr., vol. 3, no. 4, pp. 333–389, 2009, doi: 10.1561/1500000019
[16] S. E. Robertson, and S. Walker, “Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval,” 17th Annual Internatinal ACM SIGIR Conference Research and Development in Information Retrieval, Dublin, Ireland, pp. 232–241, 1994.
[17] C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval, Cambridge, U.K.: Cambridge Univ. Press, 2008.
[18] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” Journal of the American Society for Information Science, vol. 41, no. 6, pp. 391–407, 1990, doi: 10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9
[19] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, pp. 4171–4186, 2019, doi: 10.18653/v1/N19-1423.
[20] N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 3982–3992, 2019, doi: 10.18653/v1/D19-1410
[21] K. Song, X. Tan, T. Qin, J. Lu, and T.-Y. Liu, “MPNet: Masked and permuted pre-training for language understanding,” Advances in Neural Information Processing Systems, vol. 33, 2020, doi: 10.48550/arXiv.2004.09297
[22] N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych, “BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models,” Proceedings of 35th Conference Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, arxiv, Nov 2021, doi: 10.48550/arXiv.2104.08663
[23] D. H. Wolpert, “Stacked generalization,” Neural Network, vol. 5, no. 2, pp. 241–259, 1992, doi: 10.1016/S0893-6080(05)80023-1
[24] Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages,” Psychometrika, vol. 12, no. 2, pp. 153–157, 1947, doi: 10.1007/BF02295996
[25] T. G. Dietterich, “Approximate statistical tests for comparing supervised classification learning algorithms,” Neural Computers, vol. 10, no. 7, pp. 1895–1923, 1998, doi: 10.1162/089976698300017197
[26] Janez Demšar, “Statistical comparisons of classifiers over multiple data sets,” Journal of Machine Learning Research (JMLR), vol. 7, no. 1, pp. 1–30, 2006.
[27] F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics Bulletin, vol. 1, no. 6, pp. 80–83, 1945, doi: 10.2307/3001968
[28] Sture Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, vol. 6, no. 2, pp. 65–70, 1979. [Online]. Available: https://www.jstor.org/stable/4615733
[29] B. Efron, “Bootstrap methods: Another look at the jackknife,” Breakthroughs in Statistics, Springer Series in Statistics. Springer, NY vol. 7, no. 1, pp. 1–26, 1979, doi: 10.1214/aos/1176344552
[30] M. D. Smucker, J. Allan, and B. Carterette, “A comparison of statistical significance tests for information retrieval evaluation,” Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, Lisbon, Portugal, pp. 623–632, Nov 2007, doi: 10.1145/1321440.1321528
[31] K. Järvelin and J. Kekäläinen, “Cumulated gain-based evaluation of IR techniques,” ACM Transactions on Information Systems, vol. 20, no. 4, pp. 422–446, 2002, doi: 10.1145/582415.582418
[32] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Computing Surveys, vol. 54, no. 6, pp. 1–35, July 2021, doi: 10.1145/3457607
[33] M. De-Arteaga et al., “Bias in bios: A case study of semantic representation bias in a high-stakes setting,” FAT* '19: Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA, pp. 120-128, 2019, doi: 10.1145/3287560.3287572
[34] A. Caliskan, J. J. Bryson, and A. Narayanan, “Semantics derived automatically from language corpora contain human-like biases,” Science, vol. 356, no. 6334, pp. 183–186, Apr 2017, doi: 10.1126/science.aal4230
[35] M. Bertrand and S. Mullainathan, “Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination,” American Economic Review, vol. 94, no. 4, pp. 991–1013, 2004, doi: 10.1257/0002828042002561.
[36] M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian, “Certifying and removing disparate impact,” Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia, pp. 259–268, 2015, doi: 10.1145/2783258.2783311.
[37] M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” Advances in Neural Information Processing Systems, vol. 29, 2016, doi: 10.48550/arXiv.1610.02413
[38] S. Barocas, M. Hardt, and A. Narayanan, Fairness and Machine Learning: Limitations and Opportunities, Cambridge, MA, USA: MIT Press, 2023. [Online]. Available: https://fairmlbook.org
[39] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. [Online]. Available: https://jmlr.org/papers/v12/pedregosa11a.html
