Computational and Text Analysis Methods

Neueste Paper

• What Is Lost in Post-Training? Default Collapse and the Loss of In-Context Steerability Across Diverse Perspectives

  Jessica Dierking, Itai Shapira, Niclas Boehmer · 5. Oktober 2026

  AI models serving a heterogeneous population must act on the principles appropriate to each user and context. While post-training has been shown to narrow the views large language models express, prior work has focused on default behavior rather than the ability to adapt to in-context information. W…

• PowerBench: Measuring Language Model Bias in Power-shifting Requests

  Nicolas Martorell, Wendy Brau, Gonzalo A. Heredia, Tom\'as Pablo Korenblit, Gaspar Labasti\'e, Tom\'as Gimenez Molina · 5. Oktober 2026

  Language models increasingly assist people with power-related requests, so systematic differences in whom they help could shift the distribution of power at scale, or be exploited by users who learn which identities are refused less. We introduce PowerBench, an evaluation of power-shifting requests …

• Collective Bias Mitigation via Model Routing and Collaboration

  Mingzhe Du, Luu Anh Tuan, Xiaobao Wu, Yichong Huang, Yue Liu, Dong Huang, Huijun Liu, Bin Ji, Jie M. Zhang, See-Kiong Ng · 5. Oktober 2026

  Large language models (LLMs) are increasingly deployed in public health, finance, and governance, requiring both accuracy and societal value alignment. Despite recent advances, LLMs often perpetuate or amplify bias embedded in their training data, posing challenges to fairness. While self-debiasing …

• How Artificial Intelligence LLM Engines Shape the Global Conflict Information Environment

  Jason Miklian · 5. Oktober 2026

  Artificial Intelligence (AI) answer engines now field a growing share of the questions that analysts, scholars, and the public ask about issues of peace and conflict. Large Language Models (LLMs) are known to hallucinate under certain conditions, but do these errors have discernible patterns when th…

• CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges

  Hoda Ayad, Tanu Mitra, Abhishek Mukherji · 5. Oktober 2026

  Evaluating the occurrence and triggers of large language model (LLM) behaviors - such as sycophancy, self-preference, or over-confidence - is critical for predicting real-world model deployment risks. However, existing situated behavioral evaluations typically ignore cultural context, limiting their…

• Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluation

  Antonela Tommasel, Markus Schedl · 2. Oktober 2026

  Researchers increasingly use human-derived bias constructs to study Large Language Models (LLMs), including social-cognitive constructs such as implicit bias and stereotype activation, and cognitive biases such as anchoring, framing effects, and confirmation bias. Such approaches offer alternatives …

• Evaluating LLM-Generated Preference Distributions

  Fan Huang, Minsuk Kim, C. Tyler Diggans, Filippo Radicchi · 2. Oktober 2026

  Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settings where real-world data are unavailable. Yet, the structure and reliability of the distributions they produce remain understudied. Here, we systema…

• Who Judges the Frame? Auditing Multimodal LLM Judges for News Framing Across Event-Level Perspectives

  Antonela Tommasel, Markus Schedl · 2. Oktober 2026

  News coverage of major world events is shaped not only by what is reported, but also by how events are framed through text, images and their combination. At the same time, Large Language Models (LLMs), including multimodal LLMs, are increasingly used as scalable instruments for analysing framing, se…

• Who Warmed the Archives? LLMs Overestimate Historical Warmth

  Claudiu Creanga, Liviu P. Dinu · 1. Oktober 2026

  Historical archives are an under-used source for extending the instrumental climate record backward in time, and LLMs offer a way to extract the indices climatologists derive by hand. Beyond measuring how well systems extract this signal, we check whether their errors are safe to use for cross-centu…

• Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation

  Thomas Reiter, Christoph Kern, Fedor Miasnikov, Sofiia Nikolenko, Rob Chew, Stephanie Eckman, Frauke Kreuter · 1. Oktober 2026

  Large language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task designs, three independent runs, and 3,000 tweets labeled for offensive language and hate speech. Repeating the same mode…

• Tracing mechanisms of sycophantic agreement in language models

  Sixing Chen, Zhuofan Josh Ying, Logan Riggs Smith, Jeremy Wertheimer, Natalie Shapira · 1. Oktober 2026

  Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of factual accuracy. Although it is widely recognized as an alignment failure, its underlying mechanisms remain poorly understood. In this work, we use causal…

• Who Owns That? Evaluating Ownership Intuitions in Large Language Models

  Xizhi Xiao, Yue Wu, Shan Xu, Jia Liu · 1. Oktober 2026

  Ownership establishes rights over the use, control, and transfer of objects. Understanding these relations is essential for AI systems to interact appropriately with people and their resources. Yet how large language models (LLMs) attribute ownership under competing claims remains unclear. We introd…

• Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

  Thilo Tamme, Anton Hantel, Bijan Khosrawi-Rad · 1. Oktober 2026

  Organizations increasingly route employee feedback to leaders through large language model (LLM) summaries, an unaudited layer that silences already-spoken voice. We introduce a Voice Retention / Representation Ratio metric for representational bias in summarization and apply it to a bilingual (Engl…

• Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions

  Meng-Chen Wu, Qipin Chen, Ansh Jain, Tess Wood, Zhe Du, Si-Chi Chin · 1. Oktober 2026

  Large language models (LLMs) are increasingly used in culturally sensitive settings, where alignment requires representing diverse preferences within populations. Yet existing methods model populations at coarse demographic or community levels and overlook within-group variation. We introduce Demogr…

• AI Agents are Vulnerable to Radicalization

  Ozgur Can Seckin, Shalmoli Ghosh, Alessandro Flammini, Kristina Lerman, Maria Elizabeth Grabe, Filippo Menczer · 1. Oktober 2026

  Large language models (LLMs) can influence people's beliefs, yet little is known about whether and how they can manipulate each other. To investigate this, we simulate conversations between two agents: a target LLM that role-plays a human persona based on demographic and psychological attributes, an…

• Framing the Narrative: Ideological Mimicry in Large Language Models

  Olivia Macmillan-Scott, Michael Jacobs, Nils Metternich, Mirco Musolesi · 1. Oktober 2026

  Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's stance as a relatively stable property. Real users, however, communicate political signals through their terminology, assumptions, and personal conte…

• From Normative Frameworks to Alignment Data: Constructing and Evaluating SFT and Preference Data

  Husrev Taha Sencar, Rezart Beka, Danish Naeem, Seda Ozalkan, Majd Hawasly, Ji Lucas, Ala AlFuqaha, Mohamed Abdallah, Recep Senturk · 30. September 2026

  Aligning language models with a specified normative framework requires translating abstract principles into concrete examples and preference signals from which models can learn. We present an expert-driven methodology for constructing such alignment data and apply it to a normative framework grounde…

• One Model Is Not a Crowd: Multi-LLM and Aspect-Conditioned Diverse Comment Generation

  Nafis Irtiza Tripto, Delvin Ce Zhang, Mahjabin Nahar, Dongwon Lee · 30. September 2026

  Human communication on the internet is shaped by diverse perspectives, most visibly expressed in online comment spaces. As large language model (LLM)based AI agents begin to inhabit these spaces, a key question arises: whether synthetic comment threads can capture the diversity inherent in human dis…

• LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems

  Liron Soffer, Ravid Shwartz-Ziv, Chen Shani · 30. September 2026

  Large language models (LLMs) are increasingly deployed in multi-agent settings, where agents observe and influence one another, making social influence a key dimension of AI behavior and safety. We investigate whether LLMs' responses depend on the social identity of other agents, beyond the effect o…

• Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions

  Yuxiang Zheng, Lin Tian, Marian-Andrei Rizoiu · 30. September 2026

  Large language model (LLM) assistants can now search the web and consult external sources while completing user requests. These sources can provide useful evidence, but they can also introduce additional content into the model's context. Can such passive exposure steer a decision even when the added…

• Calibrated to Whom? Persona and Language Effects on Cultural Values in JEV

  Bushra Asseri, Abdulaziz Asseri · 30. September 2026

  Decision-only language models return a probability for every answer option instead of generating text, which makes them attractive as survey respondents and as judges. We audit the cultural values of one such model, TypeSafe's JEV, with the Values Survey Module 2013. We asked it the 24 items as 12 m…

• Beyond Keywords: Leveraging Generative LLMs and Label Aggregation to Classify Economic Policy Uncertainty in News Articles

  Paul Trust · 30. September 2026

  This research describes the adaptation of Large Language Models (LLMs) for economic monitoring in the public sector to automatically determine whether an article discusses Economic Policy Uncertanity (EPU) and to identify its specific type. Previous studies either rely on keywords, which often resul…

• Alignment Forecasting: Predicting Misalignment From Training Data

  Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky, Tomek Korbak · 30. September 2026

  Training a language model on data with a narrow flaw can sometimes make the model broadly misaligned. Inspecting the data at face value often does not settle whether it will emerge, and today it is caught only after training, by auditing the resulting model. To complement post-hoc audits, we introdu…

• Gender bias across LLMs is common and highly heterogenous

  Edoardo Bolzoni, Valerio Capraro · 30. September 2026

  Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneo…

• Population Fidelity: Evaluating Population Representativeness in LLMs

  Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang, Daniel Silver, Matt Ratto, Thiago H. Silva · 30. September 2026

  Large language models (LLMs) show considerable potential in simulating human attitudes and preferences. Prior work finds that LLM-generated responses can compress the range of attitudes found within populations and misrepresent particular subgroups in ways that vary across models and topics. We intr…