A Survey of Large Language Models in Finance (FinLLMs)

A curated list of resources of LLMs in Finance (FinLLMs) including their history, techniques, evaluation, and opportunities and challenges. It's based on our survey paper: A Survey of Large Language Models in Finance (FinLLMs). This survey will be actively updated including further evaluation of advanced Financial NLP tasks, a collection of financial datasets, and sharing FinLLM use-cases. Please stay tuned!🔥

Table of Contents

• Evolution : from General LMs to Financial LMs
  • General-domain LMs
  • Financial-domain LMs
• Techniques : from FinPLMs to FinLLMs
  • Continual Pre-training : FinBert-19
  • Domain-Specific Pre-training from Scratch : FinBERT-20
  • Mixed-Domain Pre-training : FinBERT-21, FLANG
  • Mixed-Domain LLM with Prompt Engineering : BloombergGPT
  • Instruction Fine-tuned LLM with Prompt Engineering : FinMA, InvestLM, FinGPT
• Benchmark Tasks and datasets
  • Sentiment Analysis
  • Text Classification
  • Named Entity Recognition
  • Question Answering
  • Stock Movement Prediction
  • Text Summarization
• Advanced Tasks and datasets
  • Relation Extraction
  • Event Detection
  • Causality Detection
  • Numerical Reasoning
  • Structure Recognition
  • Multimodal Understanding
  • Machine Translation
  • Market Forecasting
• Other Useful Resources
• Citation

Evolution

Selected papers associated with the above figure.

General-domain LMs

• GPT-1 : Improving Language Understanding by Generative Pre-Training. 2018. Paper
• GPT-2 : Language Models are Unsupervised Multitask Learners. 2019. Paper
• GPT-3 : Language Models are Few-Shot Learners. NeurIPS 2020. Paper
• GPT-4 : GPT-4 Technical Report. 2023. Paper
• BERT : BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL 2018. Paper
• T5 : Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR 2019. Paper
• ELECTRA : ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. 2020. Paper
• BLOOM : BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. 2022. Paper
• LLaMA : LLaMA: Open and Efficient Foundation Language Models. 2023. Paper
• LLaMA2 : Llama 2: Open foundation and fine-tuned chat models. 2023. Paper

Financial-domain LMs

• FinBert-19 : FinBERT: Financial Sentiment Analysis with Pre-trained Language Models. 2019. Paper | Github | HuggingFace
• FinBert-20 : FinBERT: A Pretrained Language Model for Financial Communications. 2020. Paper | Github | HuggingFace
• FinBert-21 : FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining. IJCAI special track 2021. Paper
• FLANG : When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain. EMNLP main 2022. Paper | Github | HuggingFace
• BloombergGPT : BloombergGPT: A Large Language Model for Finance, 2023, Paper
• FinMA : PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance. NeurIPS datasets and benchmarks track 2023. Paper | Github | Leaderboard
• InvestLM : InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning. 2023. Paper | Github
• FinGPT : FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets. NeurIPS Workshop 2023. Paper | Github| HuggingFace

Techniques

• [FinPLMs] Continual Pre-training : FinBert-19

• [FinPLMs] Domain-Specific Pre-training from Scratch : FinBERT-20
• [FinPLMs] Mixed-Domain Pre-training : FinBERT-21, FLANG
• [FinLLMs] Mixed-Domain LLM with Prompt Engineering : BloombergGPT
• [FinLLMs] Instruction Fine-tuned LLM with Prompt Engineering : FinMA, InvestLM, FinGPT

Benchmark Tasks and datasets

Evaluation summary of representative models across 6 financial NLP tasks

• models : FinPLMs (FLANG), FinLLMs (BloombergGPT, FinMA), LLMs (ChatGPT, GPT-4), and task-specific state-of-the-art models (SOTA) models across 6 financial NLP tasks.
• FinPLMs did not perform experiments on the more complex tasks such as Hybrid QA, SMP or Summ; hence, these data points are not included.
• Results are referenced from original or analysis research, and SOTA results from task-specific models.

Task	Dataset	Paper	Venue	Link	Data Link
Sentiment Analysis (SA)	Financial PhraseBank (FPB)	Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts	JASIST 2014	Paper	HuggingFace
	FiQA-SA	WWW'18 Open Challenge: Financial Opinion Mining and Question Answering	WWW Workshop 2018	Paper	HuggingFace
	SemEval-2017	SemEval-2017 Task 5: Fine-Grained Sentiment Analysis on Financial Microblogs and News	SemEval 2017	Paper	Bitbucket
	StockEmotions	StockEmotions: Discover Investor Emotions for Financial Sentiment Analysis and Multivariate Time Series	AAAI Bridge 2023	Paper	Github
Text Classification (TC)	Headline	Impact of News on the Commodity Market: Dataset and Results	FICC 2021	Paper	Kaggle
	FedNLP	FedNLP: An interpretable NLP System to Decode Federal Reserve Communications	SIGIR 2021	Paper	Github
	FOMC	Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis	ACL 2023	Paper	Github
	Baking77	Efficient Intent Detection with Dual Sentence Encoders	NLP4ConvAI Workshop 2020	Paper	HuggingFace
Named Entity Recognition (NER)	FIN	Domain Adaption of Named Entity Recognition to Support Credit Risk Assessment	ALTA Workshop 2015	Paper	HuggingFace
	FiNER-139	FiNER: Financial Numeric Entity Recognition for XBRL Tagging	ACL 2022	Paper	Github
Question Answering (QA)	FiQA-QA	WWW'18 Open Challenge: Financial Opinion Mining and Question Answering	WWW Workshop 2018	Paper	HuggingFace
	FinQA	FinQA: A Dataset of Numerical Reasoning over Financial Data	EMNLP 2021	Paper	Github
	ConvFinQA	ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering	EMNLP 2022	Paper	Github
	TAT-QA	TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance	ACL 2021	Paper	Github
	PACIFIC	PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance	EMNLP 2022	Paper	Github
Stock Movement Prediction (SMP)	StockNet	Stock Movement Prediction from Tweets and Historical Prices	ACL 2018	Paper	Github
	CIKM18	Hybrid Deep Sequential Modeling for Social Text-Driven Stock Prediction	CIKM 2018	Paper	Github
	BigData22	Accurate Stock Movement Prediction with Self-supervised Learning from Sparse Noisy Tweets	IEEE BigData 2022	Paper	Github
Text Summarization (Summ)	ECTSum	ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts	EMNLP 2022	Paper	Github
	MultiLing 2019	MultiLing 2019: Financial Narrative Summarisation	RANLP 2019	Paper	Website

Evaluation details across 6 financial NLP tasks

• Sentiment Analysis (SA)

- Text Classification (TC)

- Named Entity Recognition (NER)

- Question Answering (QA)

- Stock Movement Prediction (SMP)

- Text Summarization (Summ)

Advanced Tasks and datasets

Task	Dataset	Paper	Venue	Link	Data Link
Relation Extraction (RE)	FinRED	FinRED: A dataset for relation extraction in financial domain	FinWeb Workshop 2022	Paper	Github
Event Detection (ED)	EDT	Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading	ACL Findings 2021	Paper	Github
Causality Detection (CD)	FinCausal20	The Financial Document Causality Detection Shared Task (FinCausal 2020)	FNP-FNS Workshop 2020	Paper	Github
Numerical Reasoning (NR)	FiNER-139, FinQA, ConvFinQA, TAT-QA, PACIFIC	refer to NER, QA tasks
Structure Recognition (SR)	FinTabNet	Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context	WACV 2021	Paper	Website
Multimodal Understanding (MM)	MAEC	Maec: A multimodal aligned earnings conference call dataset for financial risk prediction	CIKM 2020	Paper	Github
	MONOPOLY	Monopoly: Financial prediction from monetary policy conference videos using multimodal cues	MM 2022	Paper	Github
Machine Translation (MT)	MINDS-14	Multilingual and Cross-Lingual Intent Detection from Spoken Data	EMNLP 2021	Paper	HuggingFace
	MultiFin	MultiFin: A Dataset for Multilingual Financial NLP	EACL Findings 2023	Paper	Github
Market Forecasting (MF)	StockEmotions, EDT, MAEC, MONOPOLY	refer to SA, ED, MM tasks

Other Useful Resources

We include several workshops and programs for financial NLP. The link directs to the most recent workshop website.

• [FNP] Financial Narrative Processing Workshop 5th
• [FinNLP] Financial Technology and Natural Language Processing Workshop 5th
• [ECONLP] Economics and Natural Language Processing Workshop 4th
• [AiFinBridge] AAAI bridge - AI for Financial Services 2th
• [MUFFIN] Multimodal AI for Financial Forecasting Workshop 2th
• [KDF] Knowledge Discovery from Unstructured Data in Financial Services 4th

Citation

    @article{lee2024survey,
        title={A Survey of Large Language Models in Finance (FinLLMs)}, 
        author={Lee, Jean and Stevens, Nicholas and Han, Soyeon Caren and Song, Minseok},
	journal={arXiv preprint arXiv:2402.02315},
        year={2024}
    }