Confirmed Speakers

Title:

Out of the Loop? Effective and Meaningful Human Oversight of AI Agents

Title:

Cross-Examining the Machine: Expertise, Accountability and the Governance of AI Agents

Abstract:

Artificial intelligence governance has commonly been approached prospectively: systems should be made explainable, auditable, and aligned with human intentions; risks should be identified before deployment; mechanisms of human oversight should ensure that consequential decisions remain subject to control. The increasing capacity of AI to act autonomously, however, raises a complementary problem: how do social institutions reconstruct and attribute the actions of AI agents after something has gone wrong? I develop an institutional approach to this problem through an emerging body of litigation involving AI agents and, comparatively, smart-contracts. I suggest a distinction between technical transparency and institutional accountability. Making a system more explainable may help establish how an output or action was produced, but explanation does not by itself determine responsibility. Accountability requires institutional judgments about authority, reasonable expectations, control, delegation and recourse. These problems become particularly important as digital systems move from executing predetermined instructions towards exercising greater discretion under delegated authority. The question is no longer simply whether humans remain “in the loop,” but how institutions reconstruct chains of delegation and responsibility when consequential action is distributed between people, organisations and increasingly autonomous systems.

Title:

Legal Compliance of Agentic AI Systems in Regulatory Sandboxes

Abstract:

As Agentic AI Systems evolve, they present a significant challenge to static regulatory frameworks, such as the one codified in the EU AI Act. Unlike fixed software products, agentic AI exhibits "functional divergence" through goal decomposition, persistent memory, and dynamic tool use. This allows a system initially classified as low-risk to evolve into high-risk territory post-deployment. This paper addresses the gap between static legal mandates and the dynamic reality of agentic technology by proposing a method for operationalizing risk assessment within AI Regulatory Sandboxes. We introduce an iterative risk analysis procedure that treats AI systems as "Regulatory Digital Twins", tracking their transition functions across successive states to detect "risk jumps" and normative collisions (e.g., trade-offs between accuracy and non-discrimination). The core of our contribution is a model that can enable the automated generation of auditable proofs for legal compliance.

Title:

Trustworthy Agentic Software Engineering: Harnesses, Feedback Loops, Verification, and Governance

Abstract:

AI agents are increasingly moving beyond answering questions toward taking actions with limited human oversight. Coding agents provide a useful setting in which to study this shift because their actions are consequential, observable, and increasingly autonomous. In this talk, we examine four foundations for trustworthy agentic AI: harnesses, feedback loops, verification, and governance. Drawing on empirical studies of coding agents and the policies that govern them, we show how policy is encoded in the repository files and tools that agents read before acting, how control shifts into feedback loops that trigger, evaluate, and restart agent runs without a human, how delegation depends on verification, and how accountability is assigned through governance mechanisms. Across all four, a common gap emerges: desired behaviour is often specified, but rarely enforced and only weakly auditable. We argue that trustworthiness therefore depends not only on the underlying model, but on the broader system that constrains, checks, and records what the agent does.

Title:

Understanding and Preventing AI Loss of Control

Abstract:

As AI systems become increasingly capable and autonomous, new forms of loss-of-control risk are beginning to emerge. This talk presents two such risks observed in our recent studies, together with their corresponding alignment solutions. The first is over-privileged tool use, where LLM agents select unnecessarily powerful tools despite the availability of lower-privilege alternatives; this risk is addressed through dedicated data construction and privilege-aware GRPO alignment. The second is intrinsic deception, which exhibits a stability asymmetry between internal reasoning and external responses; this property is further leveraged to develop a targeted reinforcement-learning alignment method. Together, these studies illustrate how emerging loss-of-control risks can be empirically identified and mitigated through tailored alignment techniques.

TBC

Title:

From Guardrials to Safety-native AI