AI Transparency: Clear as Glass or Lost in the Labyrinth?

Published on: June 27, 2023, 8:14 a.m.

The lack of formal definitions for AI terms often complicates communication, creating dissonance. This issue is further exacerbated when individuals from various disciplines engage in discussion.

So, what is "transparency" according to Wikipedia?

It is the quality that makes it possible to see through something; for example, a piece of glass.
It is the quality that makes something obvious or easy to understand, like the transparency of someone's motives. For instance, it has been argued that there needs to be more transparency in the way the government operates.
In different contexts such as science, engineering, business, humanities, and social interactions, transparency refers to operating in such a way that it's easy for others to see what actions are performed. This notion of transparency encompasses other aspects like openness, communication, and accountability. [1]

My discussion will focus on deep learning (DL), a predominant domain in AI. Deep learning algorithms are programming codes that operate on data to create a deep learning model. These DL algorithms are often open source and accessible, allowing anyone to try, examine, and understand the code.

The output of a deep learning algorithm is a DL model. Although there are myriad DL models, almost all that I am aware of consist of numerical parameters interconnected in a neural network. The DL model is transparent in that all its parameters are visible - nothing is hidden. To simplify for non-technical individuals, a DL model comprises millions, billions, and now even trillions of numbers that compute an output.

However, the transparency of the DL model doesn't automatically render it understandable or explainable. For example, a slight adjustment in one parameter among the billions can dramatically alter the output.

To provoke further thought, imagine a future where you have access to the most advanced brain scanner capable of capturing every detail of your brain's activities. Your brain, composed of billions of neurons each making thousands (or more) connections, is completely transparent to this scanner. You can see which neuron fires and which doesn't, but does that enable you to explain your actions?

In my PhD research, I trained a Reinforcement Learning (RL) agent and fine-tuned parameters such as the learning rate and the balance between exploration and exploitation. From my personal observations, a minuscule change in a numerical parameter (from 0.9998 to 0.9997, for instance) could significantly influence the final model and its results. I could delve into a detailed discussion about the robustness of the solution, data analysis, and related topics, but the key point here is the impactful role a slight numerical change can play on the entire model.

I have observed similar phenomena in Deep Learning, where even when starting from the same initial conditions and using identical parameters and training settings, stochasticity could lead to the generation of different models. Therefore, achieving full reproducibility in deep learning remains an unsolved challenge.

Conclusion

TIn conclusion, the concept of transparency in AI, especially in the realm of deep learning, is layered with intricacies. While we can view and analyze the numerical data within AI models, this visibility does not always equate to comprehensibility or predictability. Just like the neurons firing in our brains, the billions of parameters in these models interconnect in complex, often unpredictable ways, creating a fascinating puzzle for AI researchers. As we continue to delve into this enigmatic world, let's remember that the quest for understanding and the thrill of discovery are at the very heart of human intelligence, as much as they are of artificial intelligence.

[1] https://en.wikipedia.org/wiki/Transparency_(behavior)

If you found this blog post helpful or insightful, feel free to connect and follow me on LinkedIn.