Prof Ji Ma Explored “Can Machines Think Like Humans” in the “Collaborative Research Postgraduate Students' Seminar Series”
The School of Graduate Studies of Lingnan University and The Education University of Hong Kong co-hosted the "Collaborative Research Postgraduate Students' Seminar Series" in the 2024/25 academic year. The fourth online seminar, held on 20 December 2024, featured a compelling presentation by Prof Ji Ma from The University of Texas at Austin on "Can Machines Thinks like Humans? A Behavioural Evaluation of LLM-Agents in Dictator Games". The seminar attracted around 60 online participants, including postgraduate students and academics from various institutions.
Prof Ji Ma is an Assistant Professor at the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin. He is also an affiliated faculty member of the Centre for East Asian Studies and the School of Information at UT Austin. He focuses on the nexus of technology and society, and his publications appear in leading journals across a broad range of disciplines, including non-profit studies, sociology, political science, public administration and policy, and data science.
In this seminar, Prof Ma introduced cutting-edge AI technology, to simulate interactions between humanoids and humans in a futuristic setting. This provided a unique perspective on how large language model (LLM) agents' prosocial behaviour can be induced by different personas and benchmarked against human behaviours. By examining the underlying mechanisms that drive the generosity of LLM-Agents, Prof Ma presented a behavioural approach to explore how various personas and experimental framings affect these AI agents' altruistic behaviour in dictator games. The study compared their behaviours within the same LLM family, across different families, and with human behaviours.
The findings revealed significant variations and inconsistencies among LLMs, as well as notable differences compared to human behaviours. Merely assigning a human-like identity to LLMs does not necessarily result in human-like behaviours. Despite being trained on extensive human-generated data, these AI agents cannot accurately predict human decisions. LLM agents are unable to fully capture the internal processes of human decision-making, and their alignment with human behaviour is highly variable and dependent on specific model architectures and prompt formulations. Moreover, such dependence does not follow a clear pattern.
The study concluded with a discussion on the inconsistencies in LLM behaviour and the fundamental dilemma of determinism versus human-like uncertainty in AI decision-making. This highlights the challenges in achieving AI behaviour that mirrors human unpredictability and the complexities involved in developing AI systems that can make decisions in a manner akin to human cognition.
The “Collaborative Research Postgraduate Students' Seminar Series” will continue throughout January to May 2025.
About the Collaborative Research Postgraduate Students' Seminar Series
The collaborative seminar series aims to provide a platform for research postgraduate students to share their work, engage in insightful discussions, and foster interdisciplinary collaborations across the two universities. By bringing together scholars and students from different backgrounds, the series encourages the exchange of ideas and the advancement of knowledge in key research areas. Since this semester, the seminar series is also supported by the Section on Chinese Public Administration (SCPA) of the American Society for Public Administration (ASPA).
