News & Events

Rethinking LLM Behaviors: From Uncertainty and Priors to Financial Applications

Hangfeng He

Humanities Center Room D

As Large Language Models (LLMs) become prevalent, understanding their limitations is critical for effective deployment. This talk highlights three critical issues identified in recent research. First, we investigate the influence of input biases on uncertainty quantification in LLMs, revealing that biases disproportionately distort epistemic uncertainty estimates, often leading to overconfidence. To address this, we propose a bias mitigation strategy designed to yield more accurate and robust uncertainty measurements. Second, we analyze inductive reasoning mechanisms in LLMs, demonstrating that model-generated hypotheses predominantly reflect inherent priors rather than adapting effectively to provided demonstrations. This inherent rigidity limits the potential benefits of standard prompting techniques, necessitating novel strategies to better utilize or override these strong priors. Lastly, we examine LLM applications in finance, specifically focusing on earnings call transcripts. We identify that current approaches predominantly capture superficial ticker identity rather than the nuanced semantic details essential for insightful financial predictions. Collectively, these findings emphasize the necessity of refining current LLM approaches and outline valuable directions for future improvements.

This event will take place in person and via zoom. If participating online, please register in advance:

https://rochester.zoom.us/meeting/register/tJcsd-iuqD8tG9SEvQTZ9m9LcikHj4bJCYHE