70% of customers expect organizations to provide AI interactions

Artificial Intelligence (AI) has the power to positively transform society and the environment, and to harness that power to its full potential, organizations need to focus on addressing the ethical challenges. As per a report from the Capgemini Research Institute, “AI and the ethical conundrum: How organizations can build ethically robust AI systems and gain trust,” which found that the share of customers who believe organizations are being fully transparent about how they are using their personal data has fallen from 76% in 2019 to 62% today. The report also finds that only 53% of organizations have a leader who is responsible for ethics of AI systems. Additionally, governance and accountability for AI, and deploying pragmatic tools and frameworks for AI systems to be developed and used, is coming at a high cost for organizations.

The report notes that while organizations are more ethically aware, progress in implementing ethical AI has been inconsistent. For example, nearly the same share of executives in 2019 said that they have taken steps on building “fairness” (65% in 2020 vs. 66% in 2019) and “auditability” (45% in 2020 vs. 46% in 2019) dimensions of their AI systems; also, transparency has dropped from 73% to 59%, despite the fact that 58% of businesses say they have been building awareness amongst employees about issues that can result from the use of AI. The research also reveals that 70% of customers want a clear explanation of results and expect organizations to provide AI interactions that are transparent and fair.

Secondly, as public and private organizations increasingly continue to deploy a range of AI technologies, it is critical for them to uphold customer trust by furthering positive relationships between AI and consumers. However, reports show that datasets collected for healthcare and the public sector are subjected to social and cultural bias.

Capgemini’s research found that 65% of executives said they were aware of the issue of discriminatory bias with AI systems. Further, close to 60% of organizations have attracted legal scrutiny and 22% have faced a customer backlash in the last two to three years because of decisions reached by AI systems. In fact, 45% of customers noted they will share their negative experiences with family and friends and urge them not to engage with an organization, 39% will raise their concerns with the organization and demand an explanation, and 39% will switch from the AI channel to a higher-cost human interaction. Over a quarter of consumers (27%) say they would cease dealing with the organization altogether.

It is crucial to establish leadership at the top to ensure these issues receive due priority from top management and to create ethically robust AI systems. In addition, leaders in both business and technology functions must be fully accountable for the ethical outcomes of AI applications. Our research shows that only half said they had a confidential hotline or ombudsman to enable customers and employees to raise ethical issues with AI systems. According to the report, this is indicative of current circumstances brought about by COVID-19, as well as societal and environmental needs, growing complexity of AI models, and a change in consumer behavior, which have all disrupted the functionalities of the existing AI models. New factors, including a preference of safety, bulk buying, and a lack of training data for similar situations from the past, has meant that organizations are redesigning their systems to suit a new normal; however, this has led to less transparency.