AI and Fairness: Elke Rundensteiner and Students Develop Algorithms to Address Bias in Automated Rankings

Making sure that combined rankings fairly distribute resources—such as jobs, loans, or awards—to different groups of people with “protected attributes,” however, can be tricky. Protected attributes are personal characteristics such as race, gender, and age, that cannot be used as a reason for discrimination.

Caitlin Kuhlman '20, PhD, computer science, previously worked with Rundensteiner to develop algorithms to generate fair rankings that consider a single protected attribute, such as gender, among candidates.

Rundensteiner, Computer Science Associate Professor Lane Harrison, and PhD candidate Kathleen Cachel have gone on to develop a series of fairness metrics and novel algorithms to tackle the problem of “intersectional bias” that can occur in rankings when candidates possess more than one protected attribute.

“A lot of research makes the critical assumption that protected attributes are binary—man or woman, white or non-white,” Cachel said. “But the reality is that humans belong to lots of different groups. We need algorithms to handle multiple categories. Our innovation starts with recognizing that humans are complex and asking how we can ensure fairness with respect to all parts of their identity.”

Cachel, Rundensteiner, and Harrison have described their advance in a paper scheduled for publication at the IEEE International Conference on Data Engineering 2022. In addition, the researchers and a team including PhD candidate Hilson Shrestha are developing interactive visual analytics tools to operationalize fair AI technology, so that decision makers could choose and interactively explore the desired level of fairness in a ranking.

“With AI tools, it is important to design easily interpretable and rich notions of fairness that can be utilized for auditing—beyond simply a fair or not fair diagnosis,” Cachel says. “We want to give users more authority over that decision, to tune the degree of fairness they want to attain.”

The issue of fairness is one of society’s critical problems as companies and organizations increasingly rely on automating data-intensive activities with AI, according to Rundensteiner.

“We’ve focused on this specific problem of fairness in consensus rankings, but the bigger context of our effort is that all areas of society are being impacted by AI,” she says. “Solving this issue of fairness in AI is thus paramount to the future of society, with AI-based technologies making so many critical decisions about our lives.”