Study evaluates Banca d'Italia's hiring exams for ability and gender bias
A Banca d'Italia working paper proposes a new method to assess the effectiveness of high-stakes selection exams in identifying high-ability candidates. Applied to the Bank of Italy's hiring process, the study finds the exam generally selects able candidates and explores potential improvements.
Unpacking exam effectiveness with unobserved ability
A new working paper from Banca d'Italia introduces a method to model candidate performance in high-stakes selection exams, aiming to assess their effectiveness in selecting high-ability individuals and identify implicit discrimination.
The methodology extends Item Response Theory by modeling correct, wrong, or missing answers through a system of equations, accounting for individual unobserved heterogeneity like the propensity to answer and underlying ability.
This approach allows for estimating a candidate's ability based on their exam answers and simulating how structural changes to the exam could enhance the average ability of selected candidates.
The method was applied to the Bank of Italy's three-stage hiring exam, which includes a preselective test, a written exam, and an oral exam.
Findings suggest that unobserved ability is a crucial determinant of exam outcomes, with the probability of passing each stage increasing with a candidate's ability level.
The study confirms that the exam generally succeeds in selecting candidates with higher ability.
Simulating fairer and sharper selection
The paper also delves into the aspect of gender discrimination in hiring, noting that the selection rate for women at Banca d'Italia is lower than for men.
The author distinguishes between self-selection into application and actual discrimination.
While some preselective test questions were found to be gender-biased, their impact on the overall gender composition of selected candidates is negligible.
Simulations reveal that implementing a gender quota at the test stage could increase female hirings in male-majority exams but would lead to a predicted drop in the average ability of all hired candidates.
Conversely, the study suggests that increasing the difficulty of test or written exam questions, or removing penalization for wrong answers, could potentially raise the average ability of selected candidates for both genders.
This research extends existing literature by integrating individual unobserved ability and gender-specific differences into the assessment of exam scores.
Beyond intuition: Data-driven hiring
This paper offers a sophisticated, data-driven framework for evaluating high-stakes selection exams, moving beyond traditional assessments.
It provides valuable insights for central banks seeking to optimize their hiring processes, particularly regarding the nuanced interplay between ability selection and gender representation.
The findings underscore the importance of continuous methodological refinement to ensure both efficiency and fairness in public sector recruitment.