Computational and Text Analysis Methods

Neueste Paper

• Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluation

  Antonela Tommasel, Markus Schedl · 2. Oktober 2026

  Researchers increasingly use human-derived bias constructs to study Large Language Models (LLMs), including social-cognitive constructs such as implicit bias and stereotype activation, and cognitive biases such as anchoring, framing effects, and confirmation bias. Such approaches offer alternatives …

• Evaluating LLM-Generated Preference Distributions

  Fan Huang, Minsuk Kim, C. Tyler Diggans, Filippo Radicchi · 2. Oktober 2026

  Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settings where real-world data are unavailable. Yet, the structure and reliability of the distributions they produce remain understudied. Here, we systema…

• Who Judges the Frame? Auditing Multimodal LLM Judges for News Framing Across Event-Level Perspectives

  Antonela Tommasel, Markus Schedl · 2. Oktober 2026

  News coverage of major world events is shaped not only by what is reported, but also by how events are framed through text, images and their combination. At the same time, Large Language Models (LLMs), including multimodal LLMs, are increasingly used as scalable instruments for analysing framing, se…

• Who Warmed the Archives? LLMs Overestimate Historical Warmth

  Claudiu Creanga, Liviu P. Dinu · 1. Oktober 2026

  Historical archives are an under-used source for extending the instrumental climate record backward in time, and LLMs offer a way to extract the indices climatologists derive by hand. Beyond measuring how well systems extract this signal, we check whether their errors are safe to use for cross-centu…

• Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation

  Thomas Reiter, Christoph Kern, Fedor Miasnikov, Sofiia Nikolenko, Rob Chew, Stephanie Eckman, Frauke Kreuter · 1. Oktober 2026

  Large language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task designs, three independent runs, and 3,000 tweets labeled for offensive language and hate speech. Repeating the same mode…

• Tracing mechanisms of sycophantic agreement in language models

  Sixing Chen, Zhuofan Josh Ying, Logan Riggs Smith, Jeremy Wertheimer, Natalie Shapira · 1. Oktober 2026

  Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of factual accuracy. Although it is widely recognized as an alignment failure, its underlying mechanisms remain poorly understood. In this work, we use causal…

• Who Owns That? Evaluating Ownership Intuitions in Large Language Models

  Xizhi Xiao, Yue Wu, Shan Xu, Jia Liu · 1. Oktober 2026

  Ownership establishes rights over the use, control, and transfer of objects. Understanding these relations is essential for AI systems to interact appropriately with people and their resources. Yet how large language models (LLMs) attribute ownership under competing claims remains unclear. We introd…

• Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

  Thilo Tamme, Anton Hantel, Bijan Khosrawi-Rad · 1. Oktober 2026

  Organizations increasingly route employee feedback to leaders through large language model (LLM) summaries, an unaudited layer that silences already-spoken voice. We introduce a Voice Retention / Representation Ratio metric for representational bias in summarization and apply it to a bilingual (Engl…

• Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions

  Meng-Chen Wu, Qipin Chen, Ansh Jain, Tess Wood, Zhe Du, Si-Chi Chin · 1. Oktober 2026

  Large language models (LLMs) are increasingly used in culturally sensitive settings, where alignment requires representing diverse preferences within populations. Yet existing methods model populations at coarse demographic or community levels and overlook within-group variation. We introduce Demogr…

• AI Agents are Vulnerable to Radicalization

  Ozgur Can Seckin, Shalmoli Ghosh, Alessandro Flammini, Kristina Lerman, Maria Elizabeth Grabe, Filippo Menczer · 1. Oktober 2026

  Large language models (LLMs) can influence people's beliefs, yet little is known about whether and how they can manipulate each other. To investigate this, we simulate conversations between two agents: a target LLM that role-plays a human persona based on demographic and psychological attributes, an…

• Framing the Narrative: Ideological Mimicry in Large Language Models

  Olivia Macmillan-Scott, Michael Jacobs, Nils Metternich, Mirco Musolesi · 1. Oktober 2026

  Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's stance as a relatively stable property. Real users, however, communicate political signals through their terminology, assumptions, and personal conte…

• From Normative Frameworks to Alignment Data: Constructing and Evaluating SFT and Preference Data

  Husrev Taha Sencar, Rezart Beka, Danish Naeem, Seda Ozalkan, Majd Hawasly, Ji Lucas, Ala AlFuqaha, Mohamed Abdallah, Recep Senturk · 30. September 2026

  Aligning language models with a specified normative framework requires translating abstract principles into concrete examples and preference signals from which models can learn. We present an expert-driven methodology for constructing such alignment data and apply it to a normative framework grounde…

• One Model Is Not a Crowd: Multi-LLM and Aspect-Conditioned Diverse Comment Generation

  Nafis Irtiza Tripto, Delvin Ce Zhang, Mahjabin Nahar, Dongwon Lee · 30. September 2026

  Human communication on the internet is shaped by diverse perspectives, most visibly expressed in online comment spaces. As large language model (LLM)based AI agents begin to inhabit these spaces, a key question arises: whether synthetic comment threads can capture the diversity inherent in human dis…

• LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems

  Liron Soffer, Ravid Shwartz-Ziv, Chen Shani · 30. September 2026

  Large language models (LLMs) are increasingly deployed in multi-agent settings, where agents observe and influence one another, making social influence a key dimension of AI behavior and safety. We investigate whether LLMs' responses depend on the social identity of other agents, beyond the effect o…

• Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions

  Yuxiang Zheng, Lin Tian, Marian-Andrei Rizoiu · 30. September 2026

  Large language model (LLM) assistants can now search the web and consult external sources while completing user requests. These sources can provide useful evidence, but they can also introduce additional content into the model's context. Can such passive exposure steer a decision even when the added…

• Calibrated to Whom? Persona and Language Effects on Cultural Values in JEV

  Bushra Asseri, Abdulaziz Asseri · 30. September 2026

  Decision-only language models return a probability for every answer option instead of generating text, which makes them attractive as survey respondents and as judges. We audit the cultural values of one such model, TypeSafe's JEV, with the Values Survey Module 2013. We asked it the 24 items as 12 m…

• Beyond Keywords: Leveraging Generative LLMs and Label Aggregation to Classify Economic Policy Uncertainty in News Articles

  Paul Trust · 30. September 2026

  This research describes the adaptation of Large Language Models (LLMs) for economic monitoring in the public sector to automatically determine whether an article discusses Economic Policy Uncertanity (EPU) and to identify its specific type. Previous studies either rely on keywords, which often resul…

• Alignment Forecasting: Predicting Misalignment From Training Data

  Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky, Tomek Korbak · 30. September 2026

  Training a language model on data with a narrow flaw can sometimes make the model broadly misaligned. Inspecting the data at face value often does not settle whether it will emerge, and today it is caught only after training, by auditing the resulting model. To complement post-hoc audits, we introdu…

• Gender bias across LLMs is common and highly heterogenous

  Edoardo Bolzoni, Valerio Capraro · 30. September 2026

  Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneo…

• Population Fidelity: Evaluating Population Representativeness in LLMs

  Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang, Daniel Silver, Matt Ratto, Thiago H. Silva · 30. September 2026

  Large language models (LLMs) show considerable potential in simulating human attitudes and preferences. Prior work finds that LLM-generated responses can compress the range of attitudes found within populations and misrepresent particular subgroups in ways that vary across models and topics. We intr…

• Narrowing the Horizon: Quantifying Topic Saliency Shifts in Generative Monoculture

  Oriane Peter, Elena Simperl, Kate Devlin · 29. September 2026

  As Large Language Models (LLMs) become central to how we access and share information, they play an increasingly powerful role in shaping global knowledge. However, as these models evolve, their outputs risk converging into a \textit{generative monoculture}, where the diversity of perspectives they …

• The Argument and the Letterhead: Source-Position Coherence in AI Evaluation

  Michele Loi · 29. September 2026

  An argument can be surprising coming from a particular speaker without being a bad argument. Do AI evaluators keep these judgments apart? Two preregistered descriptive studies and a later Jev supplement collected 2,976 usable evaluations of six fixed texts about US AI policy, Germany's debt brake an…

• What Drives Citations in Production Large Language Models? An Observational Multi-Method Study of Two Million AI Citations Across Ten Thousand Web Pages

  Ben Moore, Liam Dunne · 29. September 2026

  Production large language models retrieve and cite web pages alongside generated answers, yet the page-level features that predict citation frequency remain poorly characterised. We present an observational study of approximately 2 million LLM citations from four commercial engines (ChatGPT, Claude,…

• Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Models

  Ji Huang, Mengfei Li, Shuai Shao · 29. September 2026

  Large language models are increasingly used to simulate response distributions in social surveys. Prior work has achieved accurate population-level simulation for individual questions. Real questionnaires, however, ask each respondent a sequence of related questions. A simulated respondent should sh…

• Large Language Models Substantially Compress Well-Being Inequality but Largely Preserve Its Socioeconomic Structure

  Nattavudh Powdthavee · 29. September 2026

  Research using large language models (LLMs) to generate synthetic populations has repeatedly shown that model outputs compress the diversity of human experience. This has raised doubts about whether LLM-generated data can capture meaningful differences within populations. We show that such compressi…