Speech Abstract and Seminar Recap
Title: Should Open-Source Models be Governed More or Less Strictly than Closed-Source Models?
Speaker: Dr. Elliot MCKERNON
Abstract:
Open-source software is understandably beloved by many. It's collaborative, transparent, allows users to identify and fix security vulnerabilities, and there tend to be fewer profit-hungry corporations involved to muck things up. Many even argue that open-sourcing AI models would be a fantastic tool in ensuring safety from dangerous frontier AI. However, if we take the threats and dangers of AI models seriously, we should also consider whether it's responsible to promote truly open-source AI models, especially at the frontier. In this talk, we'll discuss both side of the argument, how open-source models currently compare to closed-source models, and look at how governments and policy experts are currently treating open-source AI models.
Title: Navigating the Ethical Horizon: Pioneering Responsible AI with the Generative AI Commons
Speaker: Ms. Anni LAI
Abstract:
Join me to explore Responsible AI's vital role in shaping technology ethically. We'll navigate ethical dilemmas and societal impacts, emphasizing the urgency for frameworks prioritizing human well-being. At the core is the Responsible AI Framework by Generative AI Commons, guiding developers, researchers, and policymakers. Through transparency, fairness, accountability, and inclusivity, it empowers stakeholders to uphold ethical standards across the AI lifecycle. Let's journey towards an AI-powered future that's not just innovative but also ethically responsible.
In our pursuit of innovation, it is imperative that we steer the course of GenAI with a compass calibrated not only for progress but also for ethical integrity. Join me in an exploration of the critical imperative of Responsible AI and the pivotal role it plays in shaping the future of technology and society.
In this talk, we will delve into the multifaceted dimensions of Responsible AI, unraveling its significance in ensuring that AI technologies are developed and deployed ethically and equitably. We will examine the ethical dilemmas and societal implications that arise from AI applications, highlighting the urgent need for frameworks that prioritize human well-being and societal benefit.
At the heart of our discussion lies the Responsible AI Framework, a groundbreaking initiative spearheaded by the Generative AI Commons community. This framework serves as a guiding light for developers, researchers, and policymakers, offering a comprehensive approach to navigating the complex terrain of ethical AI development. Through principles of transparency, fairness, accountability, and inclusivity, the Responsible AI Framework empowers stakeholders to uphold ethical standards and mitigate potential harms throughout the AI lifecycle.
Together, let us embark on a journey towards a future where AI technologies are not only innovative but also responsible stewards of human values and dignity. Join the movement, embrace the Responsible AI Framework, and together, let us forge a path towards a more ethical and inclusive AI-powered world.
Video
Title: Explainable AI: Explain to Whom, When to Explain, What to Explain and How?
Speaker: Prof. Xin YAO
Abstract:
Explainable Artificial Intelligence (XAI) has been an active research topic for some time. Numerous papers and books have been published on this topic. However, it is not always clearly what the stakeholders are when XAI is discussed. Answers to some crucial questions are not always clear: (a) To whom do we want to explain? AI researchers? AI model developers? Model operators/users? Regulators? ... (b) When do we want to explain? During the data pre-processing engineering stage? Model training stage? Model validation/testing stage? Model operation stage? ... (c) What do we want to explain? Reasons why a decision is made, why a decision not made, what if? ... (d) How do we explain? Through weights of attributes, visualisation, rules, trees, knowledge graphs, chains of thought, etc.? This talk attempts to clarify rich connotations of XAI along the four major axes, which enable us to formulate our research questions more precisely and help to develop more appropriate metrics to evaluate XAI technique. The talk will also give specific examples of XAI techniques where multiple metrics are used to guide the development of more explainable AI models.
Video
Title: Designing AI Governance: Lessons from Five Years of Practice-Engaged Research
Speaker: Prof. Matti MÄNTYMÄKI
Abstract:
As AI systems become increasingly embedded in organizational processes, the need to govern their use responsibly has grown more urgent, particularly for organizations that deploy AI technologies in practice. A central challenge lies in translating abstract governance ideals, such as regulatory requirements and ethical principles, into concrete decision-making and accountability structures. This “principles-to-practice” gap has been a focal point of our research over the past five years. Drawing on a multi-project, design-oriented research program conducted in collaboration with industry partners, we have developed and tested a practical AI Governance Framework tailored to the needs of deploying organizations. I will present key insights from this work, including the framework’s evolution, implementation experiences, and lessons learned across multiple real-world contexts. The presentation reflects both the conceptual challenges and organizational realities of building responsible AI governance capacity in diverse institutional environments.
Video
Title: Risk Amplification in Agentic AI: Why Safeguarding AI Matters
Speaker: Ms. Rebekka GÖRGE
Abstract:
Agentic AI promises to transform industries through autonomous interaction and decision-making capabilities. However, as individual AI models are combined into complex agentic systems, their risks and errors can become amplified across the process chain. This talk explores the emerging challenges of Agentic AI and GenAI, emphasizing that safeguarding (Gen)AI is essential for building trustworthy AI agents. We examine how regulatory requirements and technical safeguards target trustworthiness of (Gen)AI—laying the foundation for risk-aware, trustworthy agentic systems.
Video
Title: Risks and Governance of Agentic and Physical AI
Speaker: Ms. Emanuela GIRARDI
Abstract: Addressing the Challenge of Governing Autonomous AI Systems
The emergence of agentic AI and physical AI, systems that act autonomously in digital and physical environments, poses fundamental governance challenges. These technologies operate with unprecedented independence, making decisions and acting with minimal human supervision, yet they exceed traditional regulatory mechanisms designed for human-operated systems.
These autonomous systems introduce multifaceted risks: agentic AI can pursue goals through unexpected pathways, potentially causing unintended harm or manipulating users through sophisticated psychological techniques. Physical AI systems present additional safety concerns, from autonomous vehicles to household robots operating in unpredictable human environments, where failures can result in physical harm or property damage.
This presentation addresses the persistent tension between technological capability and institutional readiness. Whether examining psychological manipulation through AI companions or autonomous robotics deployment in industrial and domestic settings, each case reveals governments and institutions struggling to govern technologies that evolve faster than policy frameworks can adapt.
The governance challenge is urgent: current global AI regulations are fragmented and uncoordinated. While numerous countries have approved ethical principles and charters, binding frameworks remain absent. Even technical standards capable of implementing responsible AI principles into practical developer guidelines have yet to be globally adopted.
What concrete steps can we take at European and global levels to build effective AI governance? We need frameworks that address the unique characteristics of autonomous systems while ensuring alignment with human values and meaningful oversight as technological capabilities accelerate.
Video
Title: Compliance Procedures and Risk Models for Agentic AI in the EU AI Act
Speaker: Prof. Antonino ROTOLO
Abstract:
To be provided.
Video
Title: AI Agents Should be Regulated Based on the Extent of Their Autonomous Operations
Speaker: Dr. Takayuki OSOGAMI
Abstract:
In this talk, I will argue that AI agents should be regulated by the extent to which they operate autonomously. AI agents with long-term planning and strategic capabilities can pose significant risks of human extinction and irreversible global catastrophes. While existing regulations often focus on computational scale as a proxy for potential harm, such measures are insufficient for assessing the risks posed by agents whose capabilities arise primarily from inference-time computation. To support this position, I will discuss relevant regulations and recommendations from scientists regarding existential risks, as well as the advantages of using action sequences -- which reflect the degree of an agent's autonomy -- as a more suitable measure of potential impact than existing metrics that rely on observing environmental states.
Video
Title: Governance Principles for the Agentic Web
Speaker: Mr. Tomer Jordi CHAFFER
Abstract:
As AI agents move beyond tools to become autonomous actors, they introduce profound governance challenges. Current frameworks, such as the EU AI Act, remain focused on general-purpose AI models, without explicit mention of agentic AI systems, leaving a governance deficit. This talk will explore why governance principles must be embedded directly into the technical architecture of the Agentic Web—a vision for the internet of agents. Drawing on recent scholarship, industry developments, and my own work in this area, I will highlight the risks, safeguards, and pathways toward an open, interoperable, and trustworthy Agentic Web.
Video
Title: Integrating Complexity Modeling and AI Testing to Uncover Biases and Emerging Strategic Behaviors of LLM Agents
Speaker: Dr. Daniele PROVERBIO
Abstract:
Complex AI models are known to display biases and inconsistencies altering they outputs and behaviors. Still, little is known as of whether emerging biases may be brought about by systems of multiple strategic AI agents. In this regard, several complex systems methods developed over the last decades, from the multi-agent systems paradigm to game theory, can be fruitfully leveraged to gain useful insights.
In this talk, I will first discuss the taxonomy of known biases, distinguishing between the associated level of complexity and feasibility of human intervention. Then, I will discuss the concept of ”emerging biases” in multi-LLM agent scenarios, drawing from the experience of multi-agent systems and from application examples. Finally, I will introduce the use of FAIRGAME (a Framework for AI Agents Bias Recognition using Game Theory, that will be presented in details in a dedicated talk during ECAI) as a tool to integrate game theory and computer science in search for biases and inconsistencies in multiple strategic AI agents. I will introduce recent results and the use of quantitative metrics to effectively benchmark and compare different LLM models in various tasks, to inform the creation of a generic governance framework for the development and deployment of safe LLMs. I will conclude by discussing recent results on game-theoretic scenarios investigating trust dynamics towards the adoption of trustworthy AI.
Video
Title: Modeling Decision-making in Societies of Humans and AI Agents
Speaker: Prof. Mirco MUSOLESI
Abstract:
The analysis and modeling of decision-making processes are of interest to economics, game theory, biology, psychology, and computer science, just to name a few. Mathematical and computational models have been developed to extract insights into the underlying mechanisms and emerging behavioral patterns. Recently, there has been a surge of interest in studying societies composed of artificial agents (and possibly humans) that learn strategies through interaction. At the same time, the AI landscape has undergone a revolution with the advent of Large Language Models (LLMs) and Foundation Models. These models possess capabilities that were unimaginable just a few years ago. An important open research area involves the integration of LLMs and Foundation Models into the design of autonomous agents and decision-support systems.
In this talk, I will give an overview of the work of my lab in modeling societies of artificial learning agents. I will discuss the design and evaluation of different decision-making architectures based on reinforcement learning and generative models. Finally, I will discuss open challenges and research questions in this fascinating emerging field.
Video
Title: Towards Human-Centric AI – Current Challenges and Opportunities in Ethics and Governance of AI
Speaker: Prof. Dr. Christoph LÜTGE
Abstract:
The rapid advancement of AI has brought about transformative opportunities across industries, from healthcare to education, while simultaneously raising profound ethical and governance challenges. This talk explores the ethical challenges and opportunities of AI, focusing on key issues such as bias and fairness, transparency, accountability, privacy, and the societal impact of autonomous systems, and also highlighting frameworks for ethical design, governance models, and regulatory approaches that balance innovation with ethics. Case studies from the TUM Institute for Ethics in AI will be provided.
Video
Title: Trustworthy AI: How to Design AI within Legal, Ethical and Societal Constraints?
Speaker: Ms. Catelijne MULLER
Abstract:
To be provided.
Title: Towards a Safe AGI
Speaker: Prof. Fabrizio RIGUZZI
Abstract:
As we get closer to Artificial General Intelligence (AGI), many researchers have raised the problem of control: how can we retain absolute power over machines that are more powerful than us? In this talk, I will discuss the problem of control and present pathways to its solution that have been proposed in the literature.