Artificial Intelligence

We let more and more of our daily life be influenced by AI technology. But can we trust the technology to handle this influence with care?
Trustworthy Plum

Browse research content

These are samples of research projects, papers or blogs written by our researchers.

More about Trustworthy AI

Suppose you have to make a difficult decision, maybe about buying a house or about whether or not to undergo a medical treatment. Who would you ask for advice in such a situation? That would be somebody you trust. But what does it mean to consider somebody trustworthy? It should be a person that is reliable: you want to ask somebody who gives good advice. You expect your advisor to handle your data with care and not, after giving sound advice, to talk about your medical background with their friends. Thus, privacy matters. You want somebody who is objective, unbiased and doesn’t treat cases depending on some criteria that really shouldn’t matter, like nationality when it comes to medical advice. In other words, the advice should be fair. And asking that person for advice should also be safe: You don’t want to go to a doctor that gives good advice, but runs malicious side studies on you behind your back. Trustworthiness is also about a person showing you that they can be trusted; to be open, transparent about their operating and to be able to explain their choices. Finally, it is about accountability in case something goes wrong.

We have spelled this out here for a single human you ask for advice, but that holds also for trust in institutions, like a bank or insurance company: You want institutions to operate reliably, safely, fairly and to care about your privacy. To assess these criteria, you want their way of operating to be transparent and explainable and you hold the institution accountable for their actions. And these are also the criteria that matter for trust in technology. Think, for instance, about your toaster. Your toaster, too, should be reliable and safe. And even though you might not be able to explain how exactly your toaster functions, you know its functionality can be understood and explained, and that fosters your trust in its reliability and safety. Sure, as long as your toaster doesn’t come with a microphone and camera, privacy might not so much be an issue. But even fairness can become relevant: Imagine your toaster is fine in a climate between 10 and 25 degrees Celsius, but spontaneously combusts when operating at 30. However, nobody noticed this problem during safety checks, because the tests took place in Sweden. In that case testing has been biased and, as a consequence, the toaster unjustly discriminates between users based on where they live.

AI technology is not so different from a toaster: It is just a piece of technology. The problem of making sure that a new type of technology can be trusted is not new, it goes back to when humans started shaping knives and arrows. And we can build on past experience and knowledge when dealing with AI technology. We know, for instance, that more goes into making technology safe than just improving the technology. Maybe we need to implement rule systems, like those that we have for cars to make driving safe. Maybe we need to ban certain types of technology, like we have done for gene modification. The challenge is to find out what is needed for the different types of AI technology that we are dealing with.

Apart from the fact that this is still a serious challenge, there are a couple of particularities of AI technology that makes the issue of trust more complex in this case than for a toaster.  When reading through the examples above you might have noticed that AI is already involved in all three of them. There is an enormous amount of technology that already operates with AI, institutions apply AI systems to a steadily growing extent and AI systems are used as advisor, assistant or coach in all kinds of contexts. AI is special in terms of the extent to which it gets involved in all facets of our life. It is not just one new piece of technology getting on the market, but a tsunami of new applications for which trust needs to be established. A related issue is the speed with which AI technology is released to the masses. Pressure is high to be the first one bringing out the newest and best chat-system and there is little time for extensive testing. And the side effects of whatever technology is put on the market can immediately affect millions of users. Finally, some types of AI technology, especially those making use of machine learning, are not transparent in the way they operate, which makes it hard to explain their behaviour. This is also known as the “blackbox problem” of AI.

To sum up, there is an enormous need to develop methods warranting that we can trust in the AI technology we are using. But trust is complex and involves various other concepts such as bias, fairness, transparency and verifiability. Our goal at CERTAIN is to find methods that, on the one hand, allow us to assess and measure how trustworthy AI technology is, and, on the other hand, to use these measures to develop new systems that we can trust.

Projects within this research theme​

Explainable AI for Fraud Detection

In this project, we study and develop new generation AI systems for fraud detection which are not only accurate, but also explainable for different stakeholders and adaptive to certain business rules.
Trustworthy White
Explainable White

Can NLP bias measures be trusted?

van der Wall, O., Bachmann, D., Leidinger, A., van Maanden, L. Zuidema, W. & Schulz, K.

Automating the Analysis of Matching Algorithms

Endriss, U.

Participatory budgeting

Improving Language Model bias measures

Explainability in Collective Decision Making