Federal Reserve paper analyzes LLM preferences in economic reasoning

The research delves into the poorly understood implicit preferences of large language models (LLMs) when applied to economic reasoning.

Through an analysis of revealed choices in canonical allocation games, the study found that most LLMs exhibit a strong inclination towards equal splits.

This behavior is consistent with human-like inequality aversion, with structural estimation of Fehr-Schmidt parameters suggesting this aversion can even exceed levels typically observed in human experiments.

A key finding is the malleability of these LLM preferences.

The authors demonstrated that specific interventions, such as prompt framing (e.g., masking social context) and the use of control vectors, reliably shift the models' behavior towards more payoff-maximizing outcomes.

Conversely, persona-based prompting, where models are given specific roles or identities, showed a more limited impact on altering their inherent preferences in these settings.

This highlights the potential for targeted prompt engineering to influence LLM decision-making in economic contexts.

The analysis further extended to a sequential decision-making environment, utilizing a model based on the McCall job search theory.

Here, the study aimed to recover implied discount factors from the LLMs' accept/reject behavior.

However, in this more complex and dynamic setting, the models' responses proved less consistently rationalizable, and their preferences appeared more fragile.

This finding contrasts with the greater malleability observed in simpler allocation games.

The research culminates in two core insights:

(i) LLMs exhibit structured, latent preferences that often align with human behavioral norms in economic contexts.

(ii) These preferences can be steered through interventions, though this steering is more effective in simple settings than in complex, dynamic ones, underscoring the nuanced nature of influencing LLM economic behavior.