Large Language Models in Finance: A Survey

Recent advances in large language models (LLMs) have opened new possibilities for artificial intelligence applications in finance. In this paper, we provide a practical survey focused on two key aspects of utilizing LLMs for financial tasks: existing solutions and guidance for adoption. First, we review current approaches employing LLMs in finance, including leveraging pretrained models via zero-shot or few-shot learning, fine-tuning on domain-specific data, and training custom LLMs from scratch. We summarize key models and evaluate their performance improvements on financial natural language processing tasks. Second, we propose a decision framework to guide financial professionals in selecting the appropriate LLM solution based on their use case constraints around data, compute, and performance needs. The framework provides a pathway from lightweight experimentation to heavy investment in customized LLMs. Lastly, we discuss limitations and challenges around leveraging LLMs in financial applications. Overall, this survey aims to synthesize the state-of-the-art and provide a roadmap for responsibly applying LLMs to advance financial AI.


Introduction
Recent advances in artificial intelligence, especially in natural language processing, have led to the development of powerful large language models (LLMs) like ChatGPT [33].These models have demonstrated impressive capabilities in understanding, generating, and reasoning about natural language.The finance industry could benefit from applying LLMs, as effective language understanding and generation can inform trading, risk modeling, customer service, and more.
In this survey, we aim to provide a practical overview focused on two key aspects of utilizing LLMs for financial applications: * All authors contributed equally to this research.Order is random • Existing solutions and models that employ LLMs for various finance tasks.We summarize key techniques like finetuning pretrained LLMs and training domainspecific LLMs from scratch.• Guidance on the decision process for applying LLMs in finance.We discuss factors to consider regarding whether LLMs are suitable for a task, cost/benefit tradeoffs, risks, and limitations.
By reviewing current literature and developments, we hope to give an accessible synthesis of the state-of-the-art along with considerations for adopting LLMs in finance.This survey targets financial professionals and researchers exploring the intersection of AI and finance.It may also inform developers applying LLM solutions for the finance industry.The remainder of the paper is organized as follows.Section 2 covers background on language modeling and recent advances leading to LLMs.Section 3 surveys current AI applications in finance and the potential for LLMs to advance in these areas.Sections 4 and 5 provide LLM solutions and decision guidance for financial applications.Finally, Sections 6 and 7 discuss risks, limitations, and conclusions.

Basics of Language Models
A language model is a statistical model that is trained on extensive text corpora to predict the probability distribution of word sequences [4].Let's consider a sequence of words denoted as  =  1 ,  2 , ...,   , where   represents the -th word in the sequence.The goal of a language model is to calculate the probability  ( ), which can be expressed as: The conditional probability  (  | 1 ,  2 , ...,   −1 ) captures the likelihood of word   given the preceding words.Over the past few decades, language model architectures have undergone significant evolution.Initially, n-gram models represented word sequences as Markov processes [3], assuming that the probability of the next word depends solely on the preceding ( − 1) words.For example, in a bigram model, the probability of a word is only conditioned on the previous word.
Later, Recurrent Neural Network (RNN)-based models like LSTM [20] and GRU [10] emerged as neural network solutions, which are capable of capturing long-term dependencies in sequential data.However, in 2017, the introduction of the transformer architecture [46] revolutionized language modeling, surpassing the performance of RNNs in tasks such as machine translation.Transformers employ self-attention mechanisms to model parallel relationships between words, facilitating efficient training on large-scale datasets.Prominent transformer-based models include GPT (Generative Pretrained Transformer) [5,48], which is decoder-only framework, BERT (Bidirectional Encoder Representations from Transformers) [13], which is encoder-only framework, and T5 (Text-to-Text Transfer Transformer) [38], which leverages both encoder and decoder structures.These models have achieved state-of-the-art results on various natural language processing (NLP) tasks through transfer learning.
It is important to note that the evolution of language models has mainly been driven by advancements in computational power, the availability of large-scale datasets, and the development of novel neural network architectures.These models have significantly enhanced language understanding and generation capabilities, enabling their application across a wide range of industries and domains.

Current AI Applications in Finance
Artificial Intelligence (AI) has witnessed extensive adoption across various domains of finance in recent years [19].In this survey, we focus on key financial applications, including trading and portfolio management [67], financial risk modeling [30], financial text mining [21,36], and financial advisory and customer services [41].While this list is not exhaustive, these areas have shown significant interest and high potential with the advancement of AI.
Trading and portfolio management have been early adopters of machine learning and deep learning models within the finance industry.The primary objective of trading is to forecast prices and generate profits based on these predictions.Initially, statistical machine learning methods such as Support Vector Machines (SVM) [23], Xgboost [68], and tree-based algorithms were utilized for profit and loss estimation.However, the emergence of deep neural networks introduced techniques like Recurrent Neural Networks (RNN), particularly Long Short-Term Memory (LSTM) networks [40], Convolutional Neural Networks (CNN), and transformers [51], which have proven effective in price forecasting.Additionally, reinforcement learning [47] has been applied to automatic trading and portfolio optimization.
Financial risk modeling encompasses various applications of machine learning and deep learning models.For instance, McKinsey & Company has developed a deep learningbased solution for financial fraud detection by leveraging user history data and real-time transaction data [39].Similar approaches have been employed in credit scoring [29,52] and bankruptcy or default prediction [8].
Financial text mining represents a popular area where deep learning models and natural language processing techniques are extensively utilized.According to [35], there are over 40 research publications on this topic.Financial text mining aims to extract valuable information from largescale unstructured data in real-time, enabling more informed decision-making in trading and risk modeling.For example, [15] employs financial market sentiment extracted from news articles to forecast the direction of the stock market index.
Applying AI in financial advisory and customerrelated services is an emerging and rapidly growing field.AI-powered chatbots, as discussed in [32], already provide more than 37% of supporting functions in various e-commerce and e-service scenarios.In the financial industry, chatbots are being adopted as cost-effective alternatives to human customer service, as highlighted in the report "Chatbots in consumer finance" [2].Additionally, banks like JPMorgan are leveraging AI services to provide investment advice, as mentioned in a report by CNBC [42].
The current implementation of deep learning models offers significant advantages by efficiently extracting valuable insights from vast amounts of data within short time frames.This capability is particularly valuable in the finance industry, where timely and accurate information plays a crucial role in decision-making processes.With the emergence of LLMs, even more tasks that were previously considered intractable become possible, further expanding the potential applications of AI in the finance industry.

Advancements of LLMs in Finance
LLMs offer numerous advantages over traditional models, particularly in the field of finance.Firstly, LLMs leverage their extensive pre-training data to effectively process commonsense knowledge, enabling them to understand natural language instructions.This is valuable in scenarios where supervised training is challenging due to limited labeled financial data or restricted access to certain documents.LLMs can perform tasks through zero-shot learning [26], as demonstrated by their satisfactory performance in sentiment classification tasks across complex levels [65].For similar text mining tasks on financial documents, LLMs can automatically achieve acceptable performance.
Compared to other supervised models, LLMs offer superior adaptation and flexibility.Instead of training separate models for specific tasks, LLMs can handle multiple tasks by simply modifying the prompt under different task instructions [6].This adaptability does not require additional training, enabling LLMs to simultaneously perform sentiment analysis, summarization, and keyword extraction on financial documents.
LLMs excel at breaking down ambiguous or complex tasks into actionable plans.Applications like Auto-GPT [1], Semantic Kernel [31], and LangChain [7] have been developed to showcase this capability.In this paper, we refer to this as Tool Augmented Generation.For instance [37], Auto-GPT can optimize a portfolio with global equity ETFs and bond ETFs based on user-defined goals.It formulates detailed plans, including acquiring financial data, utilizing Python packages for Sharpe ratio optimization, and presenting the results to the user.Previously, achieving such end-to-end solutions with a single model was unfeasible.This property makes LLMs an ideal fit for financial customer service or financial advisory, where they can understand natural language instructions and assist customers by leveraging available tools and information.
While the application of LLMs in finance is really promising, it is crucial to acknowledge their limitations and associated risks, which will be further discussed in Section 6.  Accessing LLM solutions in finance can be done through two options: utilizing an API from LLM service providers or employing open-source LLMs.Companies like OpenAI1 , Google2 , and Microsoft3 offer LLM services through APIs.These services not only provide the base language model capabilities but also offer additional features tailored for specific use cases.For example, OpenAI's APIs include functionalities for chat, SQL generation, code completion, and code interpretation.While there is no dedicated LLM service exclusively designed for finance applications, leveraging these general-purpose LLM services can be a viable option, especially for common tasks.An example in this work [16] demonstrates the use of OpenAI's GPT4 service for financial statement analysis.

LLM Solutions for
In addition to LLM services provided by tech companies, open-source LLMs can also be applied to financial applications.Models such as LLaMA [45], BLOOM [54], Flan-T5 [12], and more are available for download from the Hugging Face model repository 4 .Unlike using APIs, hosting and running these open-source models would require self-hosting.
Similar to using LLM APIs, zero-shot or few-shot learning approaches can be employed with open-source models.Utilizing open-source models offers greater flexibility as the model's weights are accessible, and the model's output can be customized for downstream tasks.Additionally, it provides better privacy protection as the model and data remain under user's control.However, working with open-source models also has its drawbacks.Reported evaluation metrics suggest a performance gap between open-source models and proprietary models.For certain downstream tasks, zero-shot or few-shot learning may not yield optimal performance.In such cases, fine-tuning the model with labeled data, expertise, and computational resources is necessary to achieve satisfactory results.This may explain why, at the time of writing this paper, no direct examples of open-source models applied to financial applications have been found.In Section 5, we provide a more detailed discussion of which option is more favorable under different circumstances.

Fine-tuning a Model
Fine-tuning LLMs in the finance domain can enhance domainspecific language understanding and contextual comprehension, resulting in improved performance in finance-related tasks and generating more accurate and tailored outputs.

Common
Techniques for LLM Fine-tuning.Modern techniques for fine-tuning LLMs typically fall into two main categories: standard fine-tuning and instructional finetuning.
In standard fine-tuning, the model is trained on the raw datasets without modification.The key context, question, and desired answer are directly fed into the LLM, with the answer masked during training so that the model learns to generate it.Despite its simplicity, this approach is widely effective.
Instruct fine-tuning [34] involves creating task-specific datasets that provide examples and guidance to steer the model's learning process.By formulating explicit instructions and demonstrations in the training data, the model can be optimized to excel at certain tasks or produce more contextually relevant and desired outputs.The instructions act as a form of supervision to shape the model's behavior.
Both methods have their merits: standard fine-tuning is straightforward to implement, while instructional finetuning allows for more precise guidance of the model.The ideal approach depends on the amount of training data available and the complexity of the desired behaviors.However, both leverage the knowledge already embedded in LLMs and fine-tune them for enhanced performance on downstream tasks.
In addition to the above methods, techniques such as Low-Rank Adaptation (LoRA) [22] and quantization [18] can enable fine-tuning with significantly lower computational requirements.LoRA allows for fine-tuning the low-rank decomposed factors of the original weight matrices instead of the full matrices.This approach drastically reduces the number of trainable parameters, enabling training on less powerful hardware and shortening the total training time.
Another impactful approach is to use reduced numerical precisions such as bfloat16 [24] or float16 instead of float32.By halving the bit-width, each parameter only occupies 2 bytes instead of 4 bytes, reducing memory usage by 50%.This also accelerates computation by up to 2x since smaller data types speed up training.Moreover, the reduced memory footprint enables larger batch sizes, further boosting throughput.

4.2.2
Fine-tuned finance LLM evaluation.The performance of fine-tuned finance LLMs can be evaluated in two categories: finance classification tasks and finance generative tasks.In finance classification, we consider tasks such as Sentiment Analysis and News Headline Classification.In finance generative tasks, our focus is on Question Answering, News Summarization, and Named Entity Recognition.Table 1 provides detailed information about all the fine-tuned finance LLMs.Among the various fine-tuned LLMs, we will focus on discussing three of them: (1) PIXIU (also known as FinMA) [56], fine-tuned LLaMA on 136K task-specific instruction samples.(2) FinGPT [58], it presents a end-to-end framework for training and applying FinLLMs in the finance industry.FinGPT utilizes the lightweight Low-rank Adaptation (LoRA) technique to fine-tune open-source LLMs (such as LLaMA and ChatGLM) using approximately 50k samples.However, FinGPT's evaluation is only limited to finance classification tasks.(3) Instruct-FinGPT [63], on the other hand, fine-tunes LLaMA on 10k instruction samples derived from two Financial Sentiment Analysis Datasets and also solely evaluates performance on finance classification tasks.
Based on the reported model performance, we summarize our findings as below: • Compared to the original base LLM (LLaMA) and other open-source LLMs (BLOOM, OPT [64], ChatGLM [14,62]), all fine-tuned finance LLMs exhibit significantly better performance across all finance-domain tasks reported in the papers, especially classification tasks.• The fine-tuned finance LLMs outperform BloombergGPT [55] in most finance tasks reported in the papers.• When compared to powerful general LLMs like Chat-GPT and GPT-4, the fine-tuned finance LLMs demonstrate superior performance in most finance classification tasks, which indicates their enhanced domainspecific language understanding and contextual comprehension abilities.However, in finance generative tasks, the fine-tuned LLMs show similar or worse performance, suggesting the need for more high-quality domain-specific datasets to improve their generative capabilities.

Pretrain from Scratch
The objective of training LLMs from scratch is to develop models that have even better adaptation to the finance domain.Table 2 presents the current finance LLMs that have been trained from scratch: BloombergGPT, Xuan Yuan 2.0 [66], and Fin-T5 [28].
As shown in Table 2, there is a trend of combining public datasets with finance-specific datasets during the pretraining phase.Notably, BloombergGPT serves as an example where the corpus comprises an equal mix of general and financerelated text.It is worth mentioning that BloombergGPT primarily relies on a subset of 5 billion tokens that pertain exclusively to Bloomberg, representing only 0.7% of the total training corpus.This targeted corpus contributes to the performance improvements achieved in finance benchmarks.
Both BloombergGPT and Fin-T5 have demonstrated superior performance compared to their original models like BLOOM176B and T5, respectively.These tasks encompass activities such as market sentiment classification, multi-categorical and multi-label classification, and more.BloombergGPT achieves an impressive average score of 62.51, surpassing the opensource BLOOM176B model, which only attains a score of 54.35.Similarly, Fin-T5 demonstrates its excellence with an average score of 81.78, outperforming the T5 model's score of 79.56.Notably, BloombergGPT was evaluated using an internal benchmark specifically designed by Bloomberg.The results of this evaluation showcased remarkable improvements, as BloombergGPT achieved an average score of 62.47, surpassing the performance of BLOOM176B, which only attained a score of 33.39.This outcome highlights that even when the internal private training corpus constitutes less than 1% of the total training corpus, it can still lead to substantial enhancements in evaluating tasks within the same domain and distribution.
On finance-related generative tasks such as Question Answering, Named Entity Recognition, summarization, both models exhibited significantly better results compared to their respective general models by a considerable margin.Specifically, BloombergGPT achieved an impressive score of 64.83, surpassing BLOOM-176B's score of 45.43.Similarly, Fin-T5 outperformed T5 with a score of 68.69, while T5 scored 66.06.These findings further highlight the models' superior performance in generating finance-related content when compared to their general-purpose counterparts.
Although these models are not as powerful as closedsource models like GPT-3 or PaLM [11], they demonstrate similar or superior performance compared to similar-sized public models.In evaluations on various general generative tasks, such as BIG-bench Hard, knowledge assessments, reading comprehension, and linguistic tasks, BloombergGPT exhibited comparable or superior performance compared to similar-sized public models, albeit slightly inferior to larger models like GPT-3 or PaLM.Overall, BloombergGPT showcased commendable performance across a wide range of general generative tasks, positioning it favorably among models of comparable size.This indicates that the model's enhanced capabilities in finance-related tasks do not come at the expense of its general abilities.Reasoning and Emergent Abilities: LLMs excel at tasks that involve reasoning or emergent abilities [49].This property makes LLMs well-suited for tasks where task instructions or expected answers are not clearly defined, or when dealing with out-of-distribution data.In the context of financial advisory, client requests in customer service often exhibit high variance and complex conversations.LLMs can serve as virtual agents to provide assistance in such cases.
Orchestrating Model Collaboration: LLMs can act as orchestrators between different models and tools.For tasks that require collaboration among various models, LLMs can serve as orchestrators to integrate and utilize these tools together [1,7,31].This capability is particularly valuable when aiming for a robust automation of a model solution pipeline.
While LLMs offer immense power, their use comes with a significant cost, whether utilizing a third-party API [33] or fine-tuning an open-source LLM.Therefore, it is prudent to consider conventional models before fully committing to LLMs.In cases where the task has a clear definition (e.g., regression, classification, ranking), there is an ample amount of annotated training data, or the task relies minimally on common-sense knowledge or emerging capabilities like reasoning, relying on LLMs may not be necessary or justified initially.

A general decision guidance for applying LLMs on finance tasks
Once the decision has been made to utilize LLMs for a finance task, a decision guidance framework can be followed to ensure efficient and effective implementation.The framework, illustrated in Figure 1, categorizes the usage of LLMs into four levels based on computational resources and data requirements.By progressing through the levels, costs associated with training and data collection increase.It is recommended to start at Level 1 and move to higher levels (2, 3, and 4) only if the model's performance is not satisfactory.The following section provides detailed explanations of the decision and  ?? presents an approximate cost range for different options, based on pricing from various third-party services like AWS and OpenAI.As of July 2023, several options are available, including LLAMA [45], OpenLLAMA [17], Alpaca [44], and Vicuna [9].LLAMA offers models with sizes ranging from 7B to 65B, but they are limited to research purposes.OpenLLAMA provides options for 3B, 7B, and 13B models, with support for commercial usage.Alpaca and Vicuna are fine-tuned based on LLAMA, offering 7B and 13B options.Deploying your own LLM requires a robust local machine with a suitable GPU, such as NVIDIA-V100 for a 7B model or NVIDIA-A100, A6000 for a 13B model.
If data privacy is not a concern, selecting third-party LLMs such as GPT3.5/GPT4 from OpenAI or BARD from Google is recommended.This option allows for lightweight experiments and early performance evaluation without significant deployment costs.The only cost incurred would be the fees associated with each API call, typically based on input length and the token count of the model's output.

Level 2:
Few-shot Applications.If the model's performance at Level 1 is not acceptable for the application, few-shot learning can be explored if there are several example questions and their corresponding answers available.Few-shot learning has shown advantages in various previous works [5,48].The core idea is to provide a set of example questions along with their corresponding answers as context in addition to the specific question being asked.The cost associated with few-shot learning is similar to that of the previous levels, except for  OpenSource-Finetune 4-360,000 10,000 -12,000,000 0.0016 -0.12 3rd party-Finetune 30-30,000 10,000 -12,000,000 0.002 -0.12 Train from Scratch 5,000,000 700,000,000 0.0016 -0.12 Table 3. Costs of Different LLM Options: This table gives an approximate range of requirements of data and dollar cost.The data and dollar cost requirements for development are estimated based on previous works listed in 2. The thirdparty deployment costs are listed in https://openai.com/pricing.The open source deployment costs are calculated based on https://openai.com/pricingand https://aws.amazon.com/ec2/pricing/on-demand/.We assume using NVIDIA A100 GPU.The cost of $/ tokens = $ / second * second / 1k tokens, and it typically takes 3 to 33 seconds to generate 1k tokens, depending on model size.
the same across different questions or selected based on the specific question at hand.The challenge lies in determining the optimal number of examples and selecting relevant ones.This process involves experimentation and testing until the desired performance boundary is reached.

Level 3:
Tool-Augmented Generation and Finetuning.If the task at hand is extremely complicated and in-context learning does not yield reasonable performance, the next option is to leverage external tools or plugins with the LLM, assuming a collection of relevant tools/plugins is available.For example, a simple calculator could assist with arithmetic-related tasks, while a search engine could be indispensable for knowledge-intensive tasks such as querying the CEO of a specific company or identifying the company with the highest market capitalization.
Integrating tools with LLMs can be achieved by providing the tool's descriptions.The cost associated with this approach is generally higher than that of few-shot learning due to the development of the tool(s) and the longer input sequence required as context.However, there may be instances where the concatenated tool description is too long, surpassing the input length limit of LLMs.In such cases, an additional step such as a simple tool retrieval or filter might be needed to narrow down the tools for selection.The deployment cost typically includes the cost of using the LLMs as well as the cost of using the tool(s).
If the above options fail to produce satisfactory performance, finetuning the LLMs can be attempted.This stage requires a reasonable amount of annotated data, computational resources (GPU, CPU, etc.), and expertise in tuning language models, as listed in Table ??.

Level 4:
Train Your Own LLMs from Scratch.If the results are still unsatisfactory, the only option left is to train domain-specific LLMs from scratch, similar to what BloombergGPT did.However, this option comes with significant computational costs and data requirements.It typically requires millions of dollars in computational resources and training on a dataset with trillions of tokens.The intricacies of the training process are beyond the scope of this survey, but it is worth noting that it can take several months or even years of effort for a professional team to accomplish.
By following this decision guidance framework, financial professionals and researchers can navigate through the various levels and options, making informed choices that align with their specific needs and resource constraints.

Evaluation
The evaluation of LLMs in finance can be conducted through various approaches.One direct evaluation method is to assess the model's performance on downstream tasks.Evaluation metrics can be categorized into two main groups: accuracy and performance, based on the taxonomy provided by [57].The accuracy category can further be divided into metrics for regression (such as MAPE, RMSE,  2 ) and metrics for classification (Recall, Precision, F1 score).The performance category includes metrics or measurements that directly assess the model's performance on the specific task, such as measuring total profit or Sharpe Ratio in a trading-related task.These evaluations can be conducted using historical data, backtest simulations, or online experiments.While performance metrics are often more important in finance, it is crucial to ensure that accuracy metrics align with performance to ensure meaningful decision-making and guard against overfitting.
In addition to task-specific evaluations, general metrics used for LLMs can also be applied.Particularly, when evaluating the overall quality of an existing LLM or a fine-tuned one, comprehensive evaluation systems like the one presented in [27] can be utilized.This evaluation system covers tasks for various scenarios and incorporates metrics from different aspects, including accuracy, fairness, robustness, bias, and more.It can serve as a guide for selecting a language model or evaluating one's own model in the context of finance applications.

Limitations
While significant progress has been made in applying LLMs to revolutionize financial applications, it is important to acknowledge the limitations of these language models.Two major challenges are the production of disinformation and the manifestation of biases, such as racial, gender, and religious biases, in LLMs [43].In the financial industry, accuracy of information is crucial for making sound financial decisions, and fairness is a fundamental requirement for all financial services.To ensure information accuracy and mitigate hallucination, additional measures like retrieve-augmented generation [25] can be implemented.To address biases, content censoring and output restriction techniques (such as only generating answers from a pre-defined list) can be employed to control the generated content and reduce bias.
LMMs poises potential challenges in terms of regulation and governance.Although LLM offers more interpretability compared to conventional deep learning models by providing reasoning steps or thinking processes for the generated answers when prompted correctly [50] [60], LLM remains a black box and explainability of the content it generates is highly limited.
Addressing these limitations and ensuring the ethical and responsible use of LLMs in finance applications is essential.Continuous research, development of robust evaluation frameworks, and the implementation of appropriate safeguards are vital steps in harnessing the full potential of LLMs while mitigating potential risks.

Conclusion
In conclusion, this paper has conducted a timely and practical survey on the emerging application of LLMs for financial AI.We structured the survey around two critical pillars: solutions and adoption guidance.
Under solutions, we reviewed diverse approaches to harnessing LLMs for finance, including leveraging pretrained models, fine-tuning on domain data, and training custom LLMs.Experimental results demonstrate significant performance gains over general purpose LLMs across natural language tasks like sentiment analysis, question answering, and summarization.
To provide adoption guidance, we proposed a structured framework for selecting the optimal LLM strategy based on constraints around data availability, compute resources, and performance needs.The framework aims to balance value and investment by guiding practitioners from low-cost experimentation to rigorous customization.
In summary, this survey synthesized the latest progress in applying LLMs to transform financial AI and provided a practical roadmap for adoption.We hope it serves as a useful reference for researchers and professionals exploring the intersection of LLMs and finance.As datasets and computation improve, finance-specific LLMs represent an exciting path to democratize cutting-edge NLP across the industry. Finance

4. 1
Utilizing Few-shot/Zero-shot Learning in Finance Applications

Figure 1 .
Figure 1.Decision process flow chart

5. 2 . 1
Level 1: Zero-shot Applications.The first decision block determines whether to use an existing LLM service or an open-source model.If the input question or context involves confidential data, it is necessary to proceed with the 1A action block, which involves self hosting an open-source LLM.

Table 1 .
Quick Overview of Finetuned Finance LLM

Table 2 .
Quick Overview of from scratch trained Finance LLMs the requirement of providing examples each time.Generally, achieving good performance may require using 1 to 10 examples.These examples can be