Can we really understand and trust AI?
As we become accustomed to living with AI products and services, ugly aspects start to appear underneath the marketing and hype: Unsettling Bing Chat AI sessions and ChatGPT hallucinations, Tesla cars crashing, image classifications labelling people as animals, AI bots spouting racist and nasty comments, recruitment tools introducing bias against women or automated systems unjustly accusing 20 000 minority families of tax fraud. The list goes on.
With more and more sensible topics such as medical, financial, and legal matters getting the AI treatment, governments and societies have started to demand guardrails around AI. This has resulted in regulations such as the EU’s AI Act, USA’s Executive Order 13960 , the AI bill of rights and the OECD AI Recommendations, as well as frameworks such as the NIST AI risk management frameworks. The World Economic Forum maintains a long list of resources for AI fairness as well. While a good step forward, these are still proposals or recommendations, and have little to no teeth to be enforced, and not easy to apply in practice.
Proving that an AI/ML model is trustworthy is very hard in practice, especially for large models like GPT. By trustworthy we mean valid and reliable, safe, fair and non-biased, secure and resilient, explainable and interpretable, and respecting privacy. This means it is expensive, complex, and requires quite a lot of time (and knowledge) to get right.
To put things in perspective: just ensuring non-bias is a tough call, with many types of bias to account for (societal, statistical, mathematical, etc.) and tradeoffs between bias and fairness: between group/individual, fairness, and utility, etc. Add ethical topics to the mix and things get trickier: should an autonomous car protect the passenger or a pedestrian in case of unavoidable collision? (Check the MIT moral machine and test your own choices!).
We cannot yet fix all the issues, but thanks to quite a bit of research, industry collaboration and the release of various open-source projects by the AI community, there are solutions coming available and being taken into use right now.
In the area of explainable AI (XAI), initiatives such as the Linux Foundation’s Trusted AI , Microsoft’s InterpretML (including their Explainable Boosting Machines), Independent work such as the explainer dashboard as well as the implementation of techniques such as LiME (Local Interpretable Model Agnostic Explanations) and SHAP (SHapley Additive exPlanations) into the most popular Machine learning frameworks and libraries, provide tools to understand models better and provide visualizations to both educate your C-Suite and the non-expert users too.
For Privacy, TensorFlow Privacy, Opacus for PyTorch and other libraries implement privacy-maintaining methods such as Differential Privacy (DPSGD- Differentially Private Stochastic Gradient Descent), or PATE (Private Aggregation of Teacher Ensembles). Federated learning is also supported, allowing a limited exposure of data by doing Machine Learning training across multiple decentralized devices holding only local data.
In the security side, you can use the Adversarial toolbox from the Linux Foundation, CleverHans or FoolBox to make AI models more robust against adversarial attacks such as Evasion (e.g. forcing improper outputs, such as masking a traffic signal to fool an autonomous car), Poisoning (adding bad data samples to the training set), Extraction (guessing the parameters of the model to later do evasion), and Inference (leaking private information).
For bias and fairness checks, from the Linux Foundation there is the AI fairness 360, TensorFlow’s Responsible AI toolkit (including fairness indicators), or FairLearn (including fairness metrics, mitigation algorithms, and visualizations).
For Large language models (LLMs), there are attention score and internal and low-level pattern visualization tools like Bertviz, and LLM models like GopherCite (by Google Deepmind) that aim to provide source citation and references for its outputs by using Google search. But it´s still early days towards looking inside these large black boxes.
In summary, unless adopting best practices is made easy, convenient, and practical, with working code readily available for the AI community to use and practitioners contributing to the common pool, hundreds of whitepapers, government frameworks, position papers and recommendations won’t move the needle towards having trustworthy AI by default.
Unless recommendations become laws with strong teeth and require compliance, making this a risk management and compliance exercise instead. But who wants to work under a big stick? I rather focus on having the carrot.
Read more about how we are working with Responsible AI:
Mikel is a senior business and technology leader with broad experience in helping global customers develop and ship next-generation digital products and services. His passion is to collaborate and combine business, technology, and software to create value. At Tietoevry Create, he is responsible for driving technology leadership across the organization and with customers, including technology excellence for solutions, assets and capabilities.