Published Papers:
Banerjee, A., & Urminsky, O. (2022). What you are getting and what you will be getting: Testing whether verb tense affects intertemporal choices. Journal of Experimental Psychology: General.
https://doi.org/10.1037/xge0001192
Prior research has shown that the way information is communicated can impact decisions, consistent with some forms of the Sapir-Whorf hypothesis that language shapes thought. In particular, language structure—specifically the form of verb tense in that language—can predict savings behaviors among speakers of different languages. We test the causal effect of language structure encountered during financial decision-making, by manipulating the verb tense (within a single language) used to communicate intertemporal tradeoffs. We find that verb tense can significantly shift choices between options, owing to tense-based inferences about timing. However, the spontaneous use of verb tense when making choices occurs only in the complete absence of other timing cues and is eliminated if even ambiguous or nondiagnostic time cues are present, although prompted timing inferences persist. We test between multiple competing accounts for how verb tense differentially impacts timing inferences and choices. We find evidence for a cue-based account, such that the presence of other cues blocks the spontaneous use of verb tense in making intertemporal decisions, consistent with the “Good Enough” proposal in psycholinguistics.
(Data publicly available here)
“The Language That Drives Engagement: A Systematic Large-scale Analysis of Headline Experiments. ”
Banerjee, A., & Urminsky, O. (2024). The language that drives engagement: A systematic large-scale analysis of headline experiments. Marketing Science.
https://doi.org/10.1287/mksc.2021.0018
We use a large-scale data set of thousands of field experiments conducted on Upworthy.com, an online media platform, to investigate the cognitive, motivational, affective, and grammatical factors implementable in messages that increase engagement with online content. We map from textual cues measured with text-analysis tools to constructs implied to be relevant by a broad range of prior research literatures. We validate the constructs with human judgment and then test which constructs causally impact click-through to articles when implemented in headlines. Our findings suggest that the use of textual cues identified in previous research and industry advice does impact the effectiveness of headlines overall, but the prior research and industry advice does not always provide useful guidance as to the direction of the effects. We identify which textual characteristics make headlines most effective at motivating engagement in our online news setting.
(Materials publicly available here)
Papers under Revision/Review:
Banerjee, A. & Urminsky, O.
(Invited third round at Journal of Consumer Psychology)
Language is pervasive and hence a common factor in people’s decision making. Prior research has mostly studied the effects of comprehensible language, language that communicates a literal meaning to consumers – on behavior and attitudes. In this paper, we investigate the potential for language that is incomprehensible to a given consumer to nevertheless impact willingness to pay and choice. In particular, we propose that potentially meaningfully incomprehensible language can convey associations beyond literal meaning. We demonstrate that adding text in a foreign language unreadable to the consumer to a known native language description of foreign food significantly increases perceptions of authenticity, uniqueness, and quality, resulting in higher valuations and greater likelihood of choice, while holding the country of origin constant. Thus, we show that, contrary to prior accounts, an incomprehensible cue creates consumer value by instilling feelings of intangible experiences and that those feelings impact decisions. We test our framework using secondary field data as well as experiments, including with consequential choices.
(Data publicly available here)
“Confident judgments of (mis)information veracity are more, rather than less, accurate."
Banerjee, A., & Rocklage, M., Mosleh, M., Rand, D.
(Under third round review at PNAS Nexus)
Does confidence help or hinder the recognition of misinformation? Prior work has reached opposing conclusions, in part because it has not separated confidence in a specific judgment from confidence in one's abilities in general. Here, we separate these constructs and test them side-by-side. In a large, pre-registered study on Lucid (N=503) where participants judged the accuracy of news headlines, confidence in specific judgments predicted greater accuracy. By contrast, general confidence predicted greater inaccuracy. In a pre-registered Prolific replication (N=498), the confidence-in-judgment effect replicated, whereas the confidence-in-general effect was less consistent. Together, these results indicate that when people say they are confident in a specific judgment, they tend to be right - but people who are generally confident in themselves are no better at spotting misinformation, and may even be worse.
(Materials publicly available here)
“Systematic AI Coding of Psychological Constructs in Language: Development and Validation of a Scalable Open-Source NLP Tool for Consumer Research”
Chen, Y., Banerjee, A., & Urminsky, O.
(Reject and Resubmit at Journal of Marketing Research)
We introduce an open-source, transparent approach to measuring psychologically meaningful language constructs at scale by converting large language models (LLMs) into phrase-aware dictionaries with continuous scoring. Rather than deploying a live LLM to label each document, we use LLMs once—up front—to score words and multi-word expressions for 48 theory-grounded constructs and freeze those scores as versioned artifacts that can be applied deterministically to new texts. We first validate the dictionaries with human judgment and then apply them to a large headline A/B test dataset, using experiment fixed effects (by mean-centering variables at the experiment level) to estimate the effects of constructs on click-through rates. Empirically, we find that the LLM-assisted dictionaries outperform both vetted, traditional Natural Language Processing tools, and even one-shot end-to-end LLM document coding, in terms of model fit. Divergences across models are also informative, highlighting constructs for targeted phrase expansion or rubric refinement. Our intended contribution is methodological and infrastructural: developing reproducible, interpretable, and modular measurement that can evolve with improving LLM models without tying inference to any single proprietary system.
(Materials publicly available here)
“The Confidence – Quality Mismatch: Assertive Language Signals Lower-Quality News ”
Banerjee, A., Aghamohammadi, Z., Rocklage, M., Rand, D., Mosleh, M.
(Submitted to Marketing Science Frontiers)
Misinformation on social media threatens democracy, public health, and civic discourse. In fast- paced feeds, users often rely on quick cues when deciding what to attend to, believe, and share. One particularly consequential linguistic cue is confidence—widely treated as a signal of knowledge and credibility. Yet, we show that in social media news contexts, confidence is counterintuitively anti-diagnostic: it signals lower-quality information while also predicting greater diffusion. Using computational linguistics and machine learning, we analyze over 20 million posts across eight major online platforms. Posts expressed with greater confidence are more likely to link to lower-quality, more misinformative news domains, even controlling for user baselines, toxicity, and political lean. At the same time, confident posts also receive more engagement. Together, these findings show that confidence—often treated as a cue to knowledge—acts as an anti-diagnostic signal of information quality while simultaneously predicting the diffusion of misinformation.
“Asymmetric Variety Seeking in Hierarchical Choices”
Winet, Y.* & Banerjee, A.*
(Submitted to Journal of Consumer Research)
People frequently navigate hierarchical decision-making environments, where higher- stages choices contain the options available at lower stages (e.g., choosing a restaurant entails another choice among dishes). While variety-seeking is a well-established phenomenon in consumer research, little is known about how a choice’s position within a hierarchical choice structure might influence variety-seeking behavior. Across eleven pre-registered experiments (N = 3,907), we document a robust asymmetry in variety-seeking across hierarchical stages: people simultaneously seek more variety at higher stages of a choice structure while seeking less variety (i.e., concentrating their choices) at lower stages . We rule out various alternative accounts, such as seeking an optimal stimulation level, and determine that hierarchical variety seeking is a goal- directed behavior: by default, consumers often hold a goal of finding the best higher-stage option, and they use differential variety seeking to do so. When seeking the best lower-stages option, however, their allocation of variety seeking changes to become more symmetrical. Together, these findings demonstrate that hierarchical positioning is not merely a contextual feature of decision environments but it also fundamentally shapes the underlying psychology of how people explore variety versus concentrate their choices across different stages.
Ongoing Projects:
Banerjee, A., & Urminsky, O.
Negative language is widely believed to boost engagement online—but does it always work? Across six studies combining over 150,000 headline field experiments and five pre-registered experiments (N = 8,369), we find that the effects of negative tone and emotion vary widely and depend critically on the reader’s goals. In large-scale headline A/B tests across 398 news platforms from around the world, negatively worded headlines showed no average effect on engagement, but revealed stark heterogeneity across platforms. Experiments testing this heterogeneity do not support differences in content or reader’s site-specific expectations, but instead identified a motivational distinction: whether readers seek credibility or enjoyment from news. Crucially, this motivational orientation appears to be a characteristic of the audiences that different platforms attract—rather than a structural feature of the platforms themselves. In controlled experiments, we show that under credibility-seeking goals, negative language triggers inferences of manipulative intent, suppressing engagement even for societally important topics. By contrast, under enjoyment goals, the same negative language may increase appeal to readers. In political contexts, we further find that while negative emotion heightens affective polarization, negative tone—when paired with credibility goals—can reduce partisan favoritism. These findings challenge the prevailing assumption that “negativity sells,” and instead show that the effects depend on goal alignment. In high-stakes domains like health, science, and politics, emotional negativity in headlines can backfire—undermining trust, engagement, and civic discourse. Goal-sensitive communication strategies are essential in a fragmented digital environment.
(Materials publicly available here)
“Does Big-Data Correlational Analysis Predict Causal Effects of Language on Decisions?”
Banerjee, A. & Urminsky, O.
A substantial research literature has used large-scale correlational text analysis to determine how characteristics of language encountered by people predicts (and presumably causes) their subsequent behavior, such as online engagement or donation decisions. These approaches have high external validity, by analyzing large-scale real- world data and behavior, but are causal interpretations of the findings internally valid? Using a novel large-scale dataset containing thousands of headline experiments, we compare the results of correlational analyses (excluding the experimental variation) with causal analyses (solely using experimental variation) to test what factors increase the likelihood of click-through in an online news setting. We find that not only does experimental data provide higher statistical power, but the correlational findings differ in magnitude, and sometimes even in direction, from the causal findings. Our results suggest that big-data correlational analyses may provide poor predictions of causal effects, underscoring the need for marketers to conduct experiments.
“Choosing Oleander Over Zanthoxylum: How Consumer Inferences of Chemicalness from Linguistic Cues in Non- Comprehended Ingredients Influence Product Choice”
Banerjee, A., Chen, S., Urminsky, O.
We explore how consumers integrate incomprehensible but meaningful linguistic cues into decision making, finding that they use linguistic cues to categorize unknown ingredients as chemical-sounding or natural-sounding. Our findings reveal that consumers are more likely to favor products with natural-seeming names, often perceiving them as less harmful and more desirable, despite lacking a full understanding of the ingredients. This research highlights the significant impact of associations drawn from incomprehensible labels on product choice and consumer decision-making.
“Information Mis-Creation: The Role of Story-Telling and Curiosity Gaps”
Banerjee, A., Wadhwani, R., Loretizo, J., Aribarg, A.
How content is written—not just what it says—shapes the way people engage with it online. Across Reddit data and a large lab study, we show that story-like writing and curiosity gaps (i.e., missing information that prompts questions) influence how users generate speculative or polarizing content in response. In political discussions, story-like writing and curiosity gaps each increase speculation, but their combination can reduce it by providing a more complete narrative. In short-form or headline-driven posts, however, curiosity gaps fuel speculation by inviting users to fill in missing details. In non-political forums, these same features spark speculation without increasing disagreement. These findings suggest that speculation and polarization don’t only result from misinformation exposure—they can also emerge from the structure of otherwise accurate content. By highlighting how linguistic features drive reader inference, our research underscores the need for platform-level interventions that address not only misinformation itself, but also the ways people fill in its gaps.