When Fear of the Unknown Becomes Real: Dealing with Bias in Artificial Intelligence

Brought to you by: Sabin Selimi Zenebe Uraguchi
Technology & digitalization

In our earlier blog, we talked about why the discussion on artificial intelligence (AI) should not be based on fear, but on our readiness to manage the innovation and put it to better use. AI enhances making a quick decision — about criminal justice, health care, job application, loans, and so on. But the decision may not necessarily be fair. Human decision making can also be flawed, shaped by individual and societal biases that are often unconscious. Humans are prone to error.


As AI thinks a lot like its creators (humans), then how can it be that different? Will AI systems be less biased than humans? Or, will it worsen the problems?


What is the problem?


Increasingly, AI systems and machine learning are used to substitute work done by humans. AI is a growing force in certain industries, increasing productivity and reducing the need for people to perform routine tasks. We fear that one day machines will ‘replace’ us. This is the fear of the unknown.


What is rather certain and genuinely concerning, is the lack of fairness that emerges from the output of a computer program. This can be a racial bias, age discrimination, gender bias, to name a few. Designers create prototypes. Data engineers gather data. There are also data scientists who execute the models. These are all real people. Some of them, unintentionally, unconsciously, or otherwise, have stereotypes or biases. It, therefore, makes sense to say that AI isn’t born biased, but it’s taught to be so.


The bias in AI systems isn’t entirely new. Back in 1988, the UK Commission for Racial Equality found a British medical school guilty of discrimination. The computer program it was using to make a decision on which student applicants would be invited for interviews was determined to be biased against women. It was also biased against those applicants with non-European names. But the program had been developed to match human admissions decisions, doing so with more than 90% accuracy. The algorithm doesn’t cure biased human decision making, but neither returning to human decision-makers does solve the problem.


Three decades later, algorithms have grown complex, but we continue to face the same problem. An AI system can help identify and reduce the impact of human biases. However, it can also make the problem worse.


For instance, a criminal justice algorithm used in the state of Florida, USA, mislabeled African-American defendants as ‘high risk’ more so than white defendants. Another example shows how the Apple card algorithm determined to offer smaller lines of credit to women than to men. Amazon also stopped using a hiring algorithm after finding that it favored male applicants more so than female ones.


Biased algorithms are easier to fix than biased people


AI systems learn to make decisions based on training data. Those data can include biased human decisions or reflect social or historical inequalities, even if sensitive variables like race and gender are removed. Another source of bias is flawed data sampling, in which a minority group, for example, is underrepresented in the training data.


study found out that facial analysis technologies had higher error rates for minority women, potentially due to unrepresentative training data. Bias hurts those discriminated against and it also hurts everyone by reducing people’s ability to participate in the economy. Bias fuels mistrust in societies, worsening existing inequalities, and exclusion.


There are no quick fixes for addressing bias in AI. One of the most complex steps is understanding and measuring ‘fairness’. How should we codify definitions of fairness? Researchers have developed technical ways of defining fairness, but usually cannot be satisfied at the same time. Still, even as fairness definitions evolve, researchers have also made progress on a wide variety of techniques that ensure the computer programs can meet them, by processing data beforehand or incorporating fairness definitions into the training process itself.


In the mid- to long-term, solutions are possible through ensuring that the AI systems we use to improve on human decision making. One way is training engineers and data scientists in understanding cognitive bias, as well as how to ‘combat’ it, is important. This also means taking more responsibility to encourage progress on research and standards that will reduce bias in algorithms.


These improvements will help. However, other challenges require more than technical ways of solving the problem. For instance, how to determine when an AI system is fair enough to be released? In which situations AI-powered decision making should be permissible at all? These questions require multi-disciplinary perspectives, including from social scientists, AI ethicists, and other thinkers in the humanities.


AI is no longer just about advanced economies


We may think that AI is more for advanced economies, and less for emerging economies (developing countries). This isn’t entirely correct.


AI is still finding its footing in emerging markets. However, certain applications have emerged and are now widely used.


recent study by MIT showed that countries that face the risk of automation are those with many jobs that involve manual or routine tasks (e.g. sorting, lifting). For example, 42% and 35% of jobs in Ghana and Sri Lanka, respectively, have an estimated automation likelihood greater than 70%.


AI also has other benefits in emerging or developing economies. For example, the United Nations Office for the Coordination of Humanitarian Affairs used Artificial Intelligence for Disaster Response (AIDR) to gather all information about the 2015 earthquake in Nepal and its damage, emergency needs, and disaster response.


In healthcare, considering that the emerging economies are short of doctors, an AI system could assist healthcare workers to make better decisions. The system can also train medical personnel. In education, an AI system can be designed to support teachers in delivering content better. And in finance, an AI system can support people who traditionally lack access to credit. Also in agriculture, farmers are using AI to inform their decisions when to sow by considering data like weather patterns, production, and sowing area. Adopting AI in emerging economies will have transformative potential.


We, development practitioners, need to embrace AI to the extent required to reallocate resources that would help emerging economies to embrace its potential. AI is based on the processing of data, and the data for emerging economies isn’t yet on the radar of the creators. We need to focus on developing AI systems that are suited to driving systemic transformation in emerging economies.

Sabin Selimi

Sabin Selimi works for Helvetas as Communication Manager in RECONOMY. Before, he worked in communication advisory roles in the public sector and internationally-funded projects, with experience in the Balkans. As a Chevening scholar, he obtained an MSc in International Public Policy from University College London. As a recipient of Presidential Scholarship, he received a bachelor’s degree in Economics and International Studies from American University in Washington, DC.

Zenebe Uraguchi

Zenebe Uraguchi is a development economist with multi-country experience (Asia, North America, Eastern Europe and Africa). He holds MA degrees in International Relations and Political Economy, as well as a PhD in Development Economics. His experience originates from working for a multinational private company, an international development bank and a research institute. Zenebe is the Programme Manager for a regional initiative in inclusive economic development, covering 12 countries in Eastern Europe (Belarus, Moldova, Ukraine), South Caucasus (Armenia, Azerbaijan, Georgia), and Western Balkans (Albania, Bosnia & Herzegovina, Kosovo, Montenegro, N. Macedonia, Serbia). His areas of expertise are in the design, management and evaluation of private, public and non-profit development initiatives focusing on employment and income.