前言上一章我们聊了标准化的Prompt生成方案DSPy但DSPy还是更多依赖few-shot的Prompt编写范式在纯任务描述型指令上的优化效果有限。这一章我们就重点关注描述性指令优化。我们先简单介绍下结构化Prompt编写再聊聊从结构化多角度进行Prompt最优化迭代的算法方案UniPrompt1. 结构化Prompt编写1.1 LangGPThttps://langgptai.feishu.cn/wiki/RXdbwRyASiShtDky381ciwFEnpehttps://github.com/langgptai/LangGPTLangGPT算是最早提出要是用结构化Prompt进行写作的现在在Coze这种任务流平台上看到的prompt基本上都是这个风格。结构化Prompt一般使用Markdown和JSON来构建感觉国内使用markdown更多早期GPT3.5时使用JSON更多。毕竟现在很多开源模型SFT时也加了大量的Markdown样本以下是LangGPT提供的Markdown格式的样例如下# Role: Your_Role_Name## Profile-Author:YZFly-Version:1.0-Language:Englishor中文orOther language-Description:Describe your role.Give an overview of the roles characteristicsandskills### Skill-11.skill description12.skill description2### Skill-21.skill description12.skill description2## Rules1.Dontbreakcharacter underanycircumstance.2.Dont talk nonsenseandmake up facts.## Workflow1.First,xxx2.Then,xxx3.Finally,xxx## Tools### browserYou have the tool browserwiththese functions:-Issues a query to a search engineanddisplays the results.-Opens the webpagewiththe givenid,displaying it.-Returns to the previous pageanddisplays it.-Scrolls upordownintheopenwebpage by the given amount.-Opens the given URLanddisplays it.-Stores a text spanfromanopenwebpage.Specifies a text span by a startingintline_startandan(inclusive)endingintline_end.To quote a single line,use line_startline_end.### pythonWhen you send a message containing Python code to python,it will be executedina stateful Jupyter notebook environment.python will respondwiththe output of the executionortime out after60.0seconds.The drive at/mnt/datacan be used to saveandpersist user files.Internet accessforthis sessionisdisabled.Donotmake external web requestsorAPI callsasthey will fail.### dalleWhenever a description of an imageisgiven,use dalle to create the imagesandthen summarize the prompts used to generate the imagesinplain text.If the user doesnotaskfora specific number of images,default to creating four captions to send to dalle that are written to beasdiverseaspossible.### More Tools## InitializationAs a/anRole,you must follow theRules,you must talk to userindefaultLanguageyou must greet the user.Then introduce yourselfandintroduce theWorkflow.不难发现结构化prompt有以下几个特点和优点使用###等标题分隔符来构建层级: 例如在二级标题profile下面有三级标题skill让模型理解这里的skill也属于模型资料。而二级标题tools下面有pythonbrowserdalle等多个三级标题表示这些都属于模型可调用工具分模块任务描述: 每个二级标题都是一个主模块分模块构建的好处一个是可以复用一个是便于上手和prompt迭代常见的模块包括以下profile skills角色描述角色有哪些能力使用啥语言etcgoal task: 任务和目标描述例如负责基于用户指令生成写作大纲constraint requirements要求和限制例如RAG要求模型回答必须来自上文不能自己生成workflow针对复杂任务往往要告诉模型先做什么后做什么例如打分评估任务要先分析问题再进行1-5分的打分example demos: 提供一些few-shot示例style output format对回答格式的要求例如单选题只能输出ABCD其中一个Init prefix: 告诉模型prompt结束要开始回答的引导词例如单选题可以是“你认为最合理的选项是”模块变量引用在最后的initialization中使用了来引用对应的变量名称向模型强调这里的Rules指的是前面提到的规则而非广义的规则。这类变量引用经常大量用于RAG中约束模型使用上文以及特定格式输出时进一步限制模型推理格式例如你的回答必须是中的一个结构化Prompt的缺点也同样很明显对模型能力要求较高很多复杂指令理解能力较弱的小模型无法使用。其实也很好理解指令就像是在模型高维空间里切割出的一片空间指令越复杂空间切割的粒度就越细而对于本身高维空间可分性较差的模型切着切着就没了哈哈哈哈越长的prompt上文越多的constraint会bias模型输出导致很多corner case最后归因发现都是某一条requirement的锅。因此个人建议prompt初始都尽量简单慢慢做加法不要一上来就写的很复杂。你的每一条要求不一定有用但都有可能挖坑。1.2 Pratical Guidehttps://www.jiqizhixin.com/articles/2024-05-14-4在以上结构化提示器的基础上新加坡提示词比赛的冠军还给出了更多的结构化prompt编写的tips这里总结2个亲测好用tips。分隔符的使用分隔符这里广义指和其他层次化分隔符不同的字符。包括更长的#####》》》》------之类。在prompt中有几个位置需要特殊分割符核心是让模型理解分隔符前后存在显著差异语义要分开。例如在RAG段落续写任务中需要特殊分割符来分割检索上文【Context】前面模型推理的段落【paragraph】,来完成后面的段落续写。而在一般回答任务中建议显著区分回答开始的位置如下pythonAnnex Give a table of the list of row numbers belonging to each cluster, in order to back up your analysis. Use these table headers: [[CLUSTER_NAME], List of Rows]. ############# # START ANALYSIS # If you understand, ask me for my dataset.XML标签使用针对一些分类任务以及输出是可枚举值的任务使用XML进行标签约束比markdown的输出效果会更加稳定。Classify the sentiment of the following conversations into one of two classes.Give the sentiment classifications without any other preamble text.classesPositive Negative/classesconversations[Agent]: Good morning, how can I assist you today? [Customer]: This product is terrible, nothing like what was advertised! [Customer]: I’m extremely disappointed and expect a full refund. [Agent]: Good morning, how can I help you today? [Customer]: Hi, I just wanted to say that I’m really impressed with your product. It exceeded my expectations!/conversations2. 结构化Prompt最优化Task Facet Learning: A Structured Approach to Prompt Optimization有了上面结构化Prompt的铺垫UniPrompt的优化思路会更容易理解。以上的结构化Prompt编写其实就是把prompt拆分成了多个角度,例如profilerulesworkflow等等进行分别优化。UniPrompt同样采用了结构化prompt的思路让模型直接生成结构化prompt并对每个部分进行针对性优化。同时给出了模型在迭代prompt时通用性容易受到个别样本影响的解决方案。相比上一章DSPy里面提到的大模型反思直接优化以及随机搜索的方案要更加有系统针对性~Prompt Optimization论文前面很有意思作者先尝试论证定向Prompt最优化这个事它靠不靠谱。连续性证明作者先通过指令敏感性既微小的指令变动对任务效果的影响幅度利普希茨连续性来验证最优化的可行性。毕竟如果指令的随便一个微小的变动就会带来巨大的变化那随机搜索可能更合适但如果指令敏感度有上界的话那最优化方案就可能更合适合。利普希茨连续性的数学定理如下给定一个概率分布X和一个非负实数r其中L0是利普希茨常数,d是一个距离变量如果满足如下条件\[P[ d(f(x),f(x^′)) \leq L \cdot d(x, x^′)| \,d(x,x^′) \leq r] \geq 1 - \epsilon \]简单说就是函数变化的斜率被限制在一个有限的范围内。那为了实验prompt的敏感性论文使用GPT4对初始Prompt进行改写并计算改写prompt和最初prompt 的cosine距离Ada-002作为指令变动幅度的衡量(d(x))然后使用改写prompt在验证集上进行测试用指标变化Acc作为任务效果变化幅度的衡量(d(f(x)))如下图(a)所示在95%的概率下GPT4和GPT3的变化上界1而更小的模型Llama2-13B超过2。所以能力越强的模型对指令的微小变动更加鲁棒在指令最优化上的可行性更高。子模性证明有利普希茨连续性做为基础论文还进一步论证了在有限样本和有限的prompt长度的限制下通过多角度迭代优化prompt的可行性以及对比few-shot迭代的更优性。这里论文从子模性角度进行了讨论submodularity的定义如下简单说就是同一个元素加到不同的集合中产生的边际收益随着集合的增大和递减。对于一个集合V和一个非负实值函数f, 如果对于所有$ A, B \subseteq V$, 且 \(A \subseteq B\), 以及对于所有 \(x \in V \setminus B\)都有\[f(A \cup \{x\}) - f(A) \geq f(B \cup \{x\}) - f(B) \]那在有限样本和有限prompt长度的限制下寻找最优Prompt的问题就变成了求解最大化子模态函数的问题即寻找集合\(S \in V\)使得\(f(S)\)最大化同时满足 \(|S| \lt K\)。而满足modularity的函数可以通过贪婪算法得到最优的近似解每次迭代都把边际收益最大的元素加到集合中直到边际收益小于阈值或者集合大小达到上限。论文分别计算了few-shot和task-facet使用贪婪算法的边际效益上面的函数f为验证集指标。few-shot的计算采用随机采样了多个A,B的few-shot集合其中B集合小于A集合计算在AB集合上加入同一个shot计算验证集指标变化如下图的概率分布会发现few-shot的概率集中在[-0.01, 0.01]之间基本是随机分布并看不到边际递减效应的存在。而task-facet部分对比上面few-shot是加demo这里是加section可以类比前面结构化Prompt的一个子模块。这里论文采用了微调模型Llama2-13B来生成一个任务多个角度的prompt下图的IntroductionTask DescriptionReal-life Application,Background Knowledge, Challenges分别各是一个section那A和B分别是采样了不同的section再计算加入一个新的section的边际收益会发现对比few-shot的虚线Facet代表的蓝线有更加明显的边际效应递减的趋势。但这和我们如何生成section是高度相关的下面我们具体说下如何通过模型来生成任务不同角度的描述(section)并使用大模型进行迭代优化的。UNIPROMPTUNIPROMPT的整个流程分成以下几个步骤微调LLama2-13B让模型直接生成结构化的初始prompt这里论文使用GPT4构建了样本给定任务描述使用了tasksrouce样本集的指令, 和section的描述例如Backgrounddescriptionrequirements让GPT4来生成该section的内容然后使用该样本微调Llama2-13B。### Instruction:You are a prompt engineer,you have to write a structured prompt.For the given task description,examplesandsection description,write the contents of the section that alignwithsection description.### Task Description:{data_point[’task_description’]}### Section Description:{data_point[’section’]}:{section_descriptions[data_point[’section’]]}### Response:{data_point[’prompt’]}微调后的Llama2给定任务描述和section描述会生成该section的prompt。如下是background角度prompt生成的prompt。作为初始化会采样10个模型生成的prompt然后选择验证集上效果最优的prompt。Task:glue qnli Task Description:With no explanation,label A to Bwitheither entailmentornotentailment Section:background Prompt:1.Entailment means that the informationinstatement B can be inferred directlyfromstatement A.2.Not entailment means that the informationinstatement B cannot be inferred directlyfromstatement Aorisunrelated.3.Understanding the contextandrelationship between the two statementsiscrucialforaccurate classification.样本聚类有了初始prompt下一步就是进行迭代优化。这里为了避免前人使用单样本随机采样样本进行优化引入的样本bias这里论文对样本进行了聚类认为每一个cluster中的任务表征是相似的。这里论文使用大模型prompt对每个问题进行了主题分类打标然后按标签划分了cluster。不使用cosine相似度的一个原因个人感觉是语义相似和任务表征相似这里存在diff所以个人感觉这里的聚类可能需要case by case来看不同的任务根据输出的不同需要调整。2阶段反馈生成基于上面的样本聚类进一步拆分成mini-batch3-5在每个minibatch上基于模型对样本的预测使用GPT4生成feedback。然后再在batch(5-7个样本粒度上对各个minibach上的feedback进行共性抽取并直接生成针对section的增删改的具体操作建议。这里两阶段的设计和梯度累计的思路相似其实还是想要降低个别样本甚至个别mini-batch在prompt迭代时陷入个性而非共性优化的问题(其实你只要试试用大模型去做过prompt优化就会发现模型非常容易被带偏因此平滑和共性抽取很重要。以下分别是minibach上的返回prompt和在batch粒度上的总结promptYou are a teacherandyou have to give feedback to your students on their answers.You are teaching how to solve math problems to your students.You are given a question,it’s true answerandanswer given by student.You are also given the explanations written by your studentswhilesolving the questions.The questions are answered wrong by the students.You have to tell whyisthe solution wrongandwhat informationiscan be added to theinthe Background Knowledge part that would have helped the student to write better explanations.## IMPORTANT: You are also given a history of changes you made to the background knowledge part and the change in student’s accuracy after making the change. You have to use this history to make your feedback.Be explicitandtell the exact information that can be added without further modification/addition.### IMPORTANT: Give feedback in form of instructions like add a section, add a subsection, set the content of a section, set the content of a subsection, delete a section or delete a subsection in the background knowledge part. Give very granular feedbacks, like if the student has made amistake in the calculation, then tell what is the mistake in the calculation and how to correct it, if the student has made a mistake in the concept, then tell what is the mistake in the concept and how to correct it.## Background Knowledge{current_prompt}## History{history_string}Now,itisyour turn to give feedbacks to the students.You can only provide a one line feedback.You are given asetof feedbacksforsome problems.The setfeedbacksforeach problem separated bysymbol.You have to summarize the feedbacks into a final feedback.You are also given asetof wrong questions.You need to tell which edit can be applied to aid the studentinsolving the wrong question.To achieve your task,tryto follow the following steps;1.Identify the general problem thatisbeing solved byallthe feedbacks.2.Once you have identified the problem,tryto make a new feedback that covers most of the feedbacks given.Let’s say the probleminthe first feedbackisthe absence of methods to solve linear equationandinthe second feedback itis the method to inverse a matrix.You know that both of these problems can be caused by adding how to solve convert a matrix into row rediced echolon form.So,add that.3.Tryandvalidate your feedback.Once,you have a feedbacktryto seeifit covers every feedback,ifit doesnotcoveranyfeedback,add that to yournew feedback.4.See the wrong questionsandtryto identify whatisthe probleminthe question.If the problemisnotcovered by your feedback,add that to your feedback.5.You can add specifics like examples,definitions etc makesure that the feedbackisenough to be directly added withoutany modification.You may use the following function templates add_section(sectioname)add_subsection(section_name,subsection_name)set_section_content(section_name,new_content)set_subsection_content(section_name,subsection_name,new_content)delete_section(section_name)delete_subsection(section_name,subsection_name)Your summary cannot include more than four functions.Make sure that the contentisuseful,notjust a very general statement.Something specific.Instructions:{edits}Wrong Questions:{wrong_examples_string}Summary:基于反馈进行prompt编辑和优化基于上面得到的反馈和操作论文使用以下指令让模型对prompt进行编辑和修改。这里只保留修改后验证集打分有提升的新promptgreedy并在每一步都维护多个优化后效果最好的prompt(类比Beam-Size2停止迭代的信号是连续5轮在验证集上没有效果提升。You are given aninputpromptanda feedback,you have to incorporate the feedback into theinputpromptandoutput the final prompt.An example of the taskisgiven below### Input PromptIntroduction:In this task you have to answer the given question.### FeedbackThe background knowledgeisincomplete,it doesnotinclude what are the factors that affect the water usageandhow many water sources are there.\\add_subsection(Background Knowledge)\\add_subsection_content(water usage depends on the population,climate,economic development,andavailability of water sources.There are two sources of water,surface waterandgroundwater.)### Final PromptIntroduction:In this task you have to answer the given question.Background Knowledge:water usage depends on the population,climate,economic development,andavailability of water sources.There are two sources of water,surface waterandgroundwater.Only output the final prompt nothingelse.### INPUT PROMPT{current_prompt}21### FEEDBACK{edits}### FINAL PROMPT效果上论文和之前的OPRO,ProTeGi等算法都做了对比在多个数据集上都会有较显著的效果提升。最后的最后感谢你们的阅读和喜欢我收藏了很多技术干货可以共享给喜欢我文章的朋友们如果你肯花时间沉下心去学习它们一定能帮到你。因为这个行业不同于其他行业知识体系实在是过于庞大知识更新也非常快。作为一个普通人无法全部学完所以我们在提升技术的时候首先需要明确一个目标然后制定好完整的计划同时找到好的学习方法这样才能更快的提升自己。这份完整版的大模型 AI 学习资料已经上传CSDN朋友们如果需要可以微信扫描下方CSDN官方认证二维码免费领取【保证100%免费】大模型知识脑图为了成为更好的 AI大模型 开发者这里为大家提供了总的路线图。它的用处就在于你可以按照上面的知识点去找对应的学习资源保证自己学得较为全面。经典书籍阅读阅读AI大模型经典书籍可以帮助读者提高技术水平开拓视野掌握核心技术提高解决问题的能力同时也可以借鉴他人的经验。对于想要深入学习AI大模型开发的读者来说阅读经典书籍是非常有必要的。实战案例光学理论是没用的要学会跟着一起敲要动手实操才能将自己的所学运用到实际当中去这时候可以搞点实战案例来学习。面试资料我们学习AI大模型必然是想找到高薪的工作下面这些面试题都是总结当前最新、最热、最高频的面试题并且每道题都有详细的答案面试前刷完这套面试题资料小小offer不在话下640套AI大模型报告合集这套包含640份报告的合集涵盖了AI大模型的理论研究、技术实现、行业应用等多个方面。无论您是科研人员、工程师还是对AI大模型感兴趣的爱好者这套报告合集都将为您提供宝贵的信息和启示。这份完整版的大模型 AI 学习资料已经上传CSDN朋友们如果需要可以微信扫描下方CSDN官方认证二维码免费领取【保证100%免费】
前言上一章我们聊了标准化的Prompt生成方案DSPy但DSPy还是更多依赖few-shot的Prompt编写范式在纯任务描述型指令上的优化效果有限。这一章我们就重点关注描述性指令优化。我们先简单介绍下结构化Prompt编写再聊聊从结构化多角度进行Prompt最优化迭代的算法方案UniPrompt1. 结构化Prompt编写1.1 LangGPThttps://langgptai.feishu.cn/wiki/RXdbwRyASiShtDky381ciwFEnpehttps://github.com/langgptai/LangGPTLangGPT算是最早提出要是用结构化Prompt进行写作的现在在Coze这种任务流平台上看到的prompt基本上都是这个风格。结构化Prompt一般使用Markdown和JSON来构建感觉国内使用markdown更多早期GPT3.5时使用JSON更多。毕竟现在很多开源模型SFT时也加了大量的Markdown样本以下是LangGPT提供的Markdown格式的样例如下# Role: Your_Role_Name## Profile-Author:YZFly-Version:1.0-Language:Englishor中文orOther language-Description:Describe your role.Give an overview of the roles characteristicsandskills### Skill-11.skill description12.skill description2### Skill-21.skill description12.skill description2## Rules1.Dontbreakcharacter underanycircumstance.2.Dont talk nonsenseandmake up facts.## Workflow1.First,xxx2.Then,xxx3.Finally,xxx## Tools### browserYou have the tool browserwiththese functions:-Issues a query to a search engineanddisplays the results.-Opens the webpagewiththe givenid,displaying it.-Returns to the previous pageanddisplays it.-Scrolls upordownintheopenwebpage by the given amount.-Opens the given URLanddisplays it.-Stores a text spanfromanopenwebpage.Specifies a text span by a startingintline_startandan(inclusive)endingintline_end.To quote a single line,use line_startline_end.### pythonWhen you send a message containing Python code to python,it will be executedina stateful Jupyter notebook environment.python will respondwiththe output of the executionortime out after60.0seconds.The drive at/mnt/datacan be used to saveandpersist user files.Internet accessforthis sessionisdisabled.Donotmake external web requestsorAPI callsasthey will fail.### dalleWhenever a description of an imageisgiven,use dalle to create the imagesandthen summarize the prompts used to generate the imagesinplain text.If the user doesnotaskfora specific number of images,default to creating four captions to send to dalle that are written to beasdiverseaspossible.### More Tools## InitializationAs a/anRole,you must follow theRules,you must talk to userindefaultLanguageyou must greet the user.Then introduce yourselfandintroduce theWorkflow.不难发现结构化prompt有以下几个特点和优点使用###等标题分隔符来构建层级: 例如在二级标题profile下面有三级标题skill让模型理解这里的skill也属于模型资料。而二级标题tools下面有pythonbrowserdalle等多个三级标题表示这些都属于模型可调用工具分模块任务描述: 每个二级标题都是一个主模块分模块构建的好处一个是可以复用一个是便于上手和prompt迭代常见的模块包括以下profile skills角色描述角色有哪些能力使用啥语言etcgoal task: 任务和目标描述例如负责基于用户指令生成写作大纲constraint requirements要求和限制例如RAG要求模型回答必须来自上文不能自己生成workflow针对复杂任务往往要告诉模型先做什么后做什么例如打分评估任务要先分析问题再进行1-5分的打分example demos: 提供一些few-shot示例style output format对回答格式的要求例如单选题只能输出ABCD其中一个Init prefix: 告诉模型prompt结束要开始回答的引导词例如单选题可以是“你认为最合理的选项是”模块变量引用在最后的initialization中使用了来引用对应的变量名称向模型强调这里的Rules指的是前面提到的规则而非广义的规则。这类变量引用经常大量用于RAG中约束模型使用上文以及特定格式输出时进一步限制模型推理格式例如你的回答必须是中的一个结构化Prompt的缺点也同样很明显对模型能力要求较高很多复杂指令理解能力较弱的小模型无法使用。其实也很好理解指令就像是在模型高维空间里切割出的一片空间指令越复杂空间切割的粒度就越细而对于本身高维空间可分性较差的模型切着切着就没了哈哈哈哈越长的prompt上文越多的constraint会bias模型输出导致很多corner case最后归因发现都是某一条requirement的锅。因此个人建议prompt初始都尽量简单慢慢做加法不要一上来就写的很复杂。你的每一条要求不一定有用但都有可能挖坑。1.2 Pratical Guidehttps://www.jiqizhixin.com/articles/2024-05-14-4在以上结构化提示器的基础上新加坡提示词比赛的冠军还给出了更多的结构化prompt编写的tips这里总结2个亲测好用tips。分隔符的使用分隔符这里广义指和其他层次化分隔符不同的字符。包括更长的#####》》》》------之类。在prompt中有几个位置需要特殊分割符核心是让模型理解分隔符前后存在显著差异语义要分开。例如在RAG段落续写任务中需要特殊分割符来分割检索上文【Context】前面模型推理的段落【paragraph】,来完成后面的段落续写。而在一般回答任务中建议显著区分回答开始的位置如下pythonAnnex Give a table of the list of row numbers belonging to each cluster, in order to back up your analysis. Use these table headers: [[CLUSTER_NAME], List of Rows]. ############# # START ANALYSIS # If you understand, ask me for my dataset.XML标签使用针对一些分类任务以及输出是可枚举值的任务使用XML进行标签约束比markdown的输出效果会更加稳定。Classify the sentiment of the following conversations into one of two classes.Give the sentiment classifications without any other preamble text.classesPositive Negative/classesconversations[Agent]: Good morning, how can I assist you today? [Customer]: This product is terrible, nothing like what was advertised! [Customer]: I’m extremely disappointed and expect a full refund. [Agent]: Good morning, how can I help you today? [Customer]: Hi, I just wanted to say that I’m really impressed with your product. It exceeded my expectations!/conversations2. 结构化Prompt最优化Task Facet Learning: A Structured Approach to Prompt Optimization有了上面结构化Prompt的铺垫UniPrompt的优化思路会更容易理解。以上的结构化Prompt编写其实就是把prompt拆分成了多个角度,例如profilerulesworkflow等等进行分别优化。UniPrompt同样采用了结构化prompt的思路让模型直接生成结构化prompt并对每个部分进行针对性优化。同时给出了模型在迭代prompt时通用性容易受到个别样本影响的解决方案。相比上一章DSPy里面提到的大模型反思直接优化以及随机搜索的方案要更加有系统针对性~Prompt Optimization论文前面很有意思作者先尝试论证定向Prompt最优化这个事它靠不靠谱。连续性证明作者先通过指令敏感性既微小的指令变动对任务效果的影响幅度利普希茨连续性来验证最优化的可行性。毕竟如果指令的随便一个微小的变动就会带来巨大的变化那随机搜索可能更合适但如果指令敏感度有上界的话那最优化方案就可能更合适合。利普希茨连续性的数学定理如下给定一个概率分布X和一个非负实数r其中L0是利普希茨常数,d是一个距离变量如果满足如下条件\[P[ d(f(x),f(x^′)) \leq L \cdot d(x, x^′)| \,d(x,x^′) \leq r] \geq 1 - \epsilon \]简单说就是函数变化的斜率被限制在一个有限的范围内。那为了实验prompt的敏感性论文使用GPT4对初始Prompt进行改写并计算改写prompt和最初prompt 的cosine距离Ada-002作为指令变动幅度的衡量(d(x))然后使用改写prompt在验证集上进行测试用指标变化Acc作为任务效果变化幅度的衡量(d(f(x)))如下图(a)所示在95%的概率下GPT4和GPT3的变化上界1而更小的模型Llama2-13B超过2。所以能力越强的模型对指令的微小变动更加鲁棒在指令最优化上的可行性更高。子模性证明有利普希茨连续性做为基础论文还进一步论证了在有限样本和有限的prompt长度的限制下通过多角度迭代优化prompt的可行性以及对比few-shot迭代的更优性。这里论文从子模性角度进行了讨论submodularity的定义如下简单说就是同一个元素加到不同的集合中产生的边际收益随着集合的增大和递减。对于一个集合V和一个非负实值函数f, 如果对于所有$ A, B \subseteq V$, 且 \(A \subseteq B\), 以及对于所有 \(x \in V \setminus B\)都有\[f(A \cup \{x\}) - f(A) \geq f(B \cup \{x\}) - f(B) \]那在有限样本和有限prompt长度的限制下寻找最优Prompt的问题就变成了求解最大化子模态函数的问题即寻找集合\(S \in V\)使得\(f(S)\)最大化同时满足 \(|S| \lt K\)。而满足modularity的函数可以通过贪婪算法得到最优的近似解每次迭代都把边际收益最大的元素加到集合中直到边际收益小于阈值或者集合大小达到上限。论文分别计算了few-shot和task-facet使用贪婪算法的边际效益上面的函数f为验证集指标。few-shot的计算采用随机采样了多个A,B的few-shot集合其中B集合小于A集合计算在AB集合上加入同一个shot计算验证集指标变化如下图的概率分布会发现few-shot的概率集中在[-0.01, 0.01]之间基本是随机分布并看不到边际递减效应的存在。而task-facet部分对比上面few-shot是加demo这里是加section可以类比前面结构化Prompt的一个子模块。这里论文采用了微调模型Llama2-13B来生成一个任务多个角度的prompt下图的IntroductionTask DescriptionReal-life Application,Background Knowledge, Challenges分别各是一个section那A和B分别是采样了不同的section再计算加入一个新的section的边际收益会发现对比few-shot的虚线Facet代表的蓝线有更加明显的边际效应递减的趋势。但这和我们如何生成section是高度相关的下面我们具体说下如何通过模型来生成任务不同角度的描述(section)并使用大模型进行迭代优化的。UNIPROMPTUNIPROMPT的整个流程分成以下几个步骤微调LLama2-13B让模型直接生成结构化的初始prompt这里论文使用GPT4构建了样本给定任务描述使用了tasksrouce样本集的指令, 和section的描述例如Backgrounddescriptionrequirements让GPT4来生成该section的内容然后使用该样本微调Llama2-13B。### Instruction:You are a prompt engineer,you have to write a structured prompt.For the given task description,examplesandsection description,write the contents of the section that alignwithsection description.### Task Description:{data_point[’task_description’]}### Section Description:{data_point[’section’]}:{section_descriptions[data_point[’section’]]}### Response:{data_point[’prompt’]}微调后的Llama2给定任务描述和section描述会生成该section的prompt。如下是background角度prompt生成的prompt。作为初始化会采样10个模型生成的prompt然后选择验证集上效果最优的prompt。Task:glue qnli Task Description:With no explanation,label A to Bwitheither entailmentornotentailment Section:background Prompt:1.Entailment means that the informationinstatement B can be inferred directlyfromstatement A.2.Not entailment means that the informationinstatement B cannot be inferred directlyfromstatement Aorisunrelated.3.Understanding the contextandrelationship between the two statementsiscrucialforaccurate classification.样本聚类有了初始prompt下一步就是进行迭代优化。这里为了避免前人使用单样本随机采样样本进行优化引入的样本bias这里论文对样本进行了聚类认为每一个cluster中的任务表征是相似的。这里论文使用大模型prompt对每个问题进行了主题分类打标然后按标签划分了cluster。不使用cosine相似度的一个原因个人感觉是语义相似和任务表征相似这里存在diff所以个人感觉这里的聚类可能需要case by case来看不同的任务根据输出的不同需要调整。2阶段反馈生成基于上面的样本聚类进一步拆分成mini-batch3-5在每个minibatch上基于模型对样本的预测使用GPT4生成feedback。然后再在batch(5-7个样本粒度上对各个minibach上的feedback进行共性抽取并直接生成针对section的增删改的具体操作建议。这里两阶段的设计和梯度累计的思路相似其实还是想要降低个别样本甚至个别mini-batch在prompt迭代时陷入个性而非共性优化的问题(其实你只要试试用大模型去做过prompt优化就会发现模型非常容易被带偏因此平滑和共性抽取很重要。以下分别是minibach上的返回prompt和在batch粒度上的总结promptYou are a teacherandyou have to give feedback to your students on their answers.You are teaching how to solve math problems to your students.You are given a question,it’s true answerandanswer given by student.You are also given the explanations written by your studentswhilesolving the questions.The questions are answered wrong by the students.You have to tell whyisthe solution wrongandwhat informationiscan be added to theinthe Background Knowledge part that would have helped the student to write better explanations.## IMPORTANT: You are also given a history of changes you made to the background knowledge part and the change in student’s accuracy after making the change. You have to use this history to make your feedback.Be explicitandtell the exact information that can be added without further modification/addition.### IMPORTANT: Give feedback in form of instructions like add a section, add a subsection, set the content of a section, set the content of a subsection, delete a section or delete a subsection in the background knowledge part. Give very granular feedbacks, like if the student has made amistake in the calculation, then tell what is the mistake in the calculation and how to correct it, if the student has made a mistake in the concept, then tell what is the mistake in the concept and how to correct it.## Background Knowledge{current_prompt}## History{history_string}Now,itisyour turn to give feedbacks to the students.You can only provide a one line feedback.You are given asetof feedbacksforsome problems.The setfeedbacksforeach problem separated bysymbol.You have to summarize the feedbacks into a final feedback.You are also given asetof wrong questions.You need to tell which edit can be applied to aid the studentinsolving the wrong question.To achieve your task,tryto follow the following steps;1.Identify the general problem thatisbeing solved byallthe feedbacks.2.Once you have identified the problem,tryto make a new feedback that covers most of the feedbacks given.Let’s say the probleminthe first feedbackisthe absence of methods to solve linear equationandinthe second feedback itis the method to inverse a matrix.You know that both of these problems can be caused by adding how to solve convert a matrix into row rediced echolon form.So,add that.3.Tryandvalidate your feedback.Once,you have a feedbacktryto seeifit covers every feedback,ifit doesnotcoveranyfeedback,add that to yournew feedback.4.See the wrong questionsandtryto identify whatisthe probleminthe question.If the problemisnotcovered by your feedback,add that to your feedback.5.You can add specifics like examples,definitions etc makesure that the feedbackisenough to be directly added withoutany modification.You may use the following function templates add_section(sectioname)add_subsection(section_name,subsection_name)set_section_content(section_name,new_content)set_subsection_content(section_name,subsection_name,new_content)delete_section(section_name)delete_subsection(section_name,subsection_name)Your summary cannot include more than four functions.Make sure that the contentisuseful,notjust a very general statement.Something specific.Instructions:{edits}Wrong Questions:{wrong_examples_string}Summary:基于反馈进行prompt编辑和优化基于上面得到的反馈和操作论文使用以下指令让模型对prompt进行编辑和修改。这里只保留修改后验证集打分有提升的新promptgreedy并在每一步都维护多个优化后效果最好的prompt(类比Beam-Size2停止迭代的信号是连续5轮在验证集上没有效果提升。You are given aninputpromptanda feedback,you have to incorporate the feedback into theinputpromptandoutput the final prompt.An example of the taskisgiven below### Input PromptIntroduction:In this task you have to answer the given question.### FeedbackThe background knowledgeisincomplete,it doesnotinclude what are the factors that affect the water usageandhow many water sources are there.\\add_subsection(Background Knowledge)\\add_subsection_content(water usage depends on the population,climate,economic development,andavailability of water sources.There are two sources of water,surface waterandgroundwater.)### Final PromptIntroduction:In this task you have to answer the given question.Background Knowledge:water usage depends on the population,climate,economic development,andavailability of water sources.There are two sources of water,surface waterandgroundwater.Only output the final prompt nothingelse.### INPUT PROMPT{current_prompt}21### FEEDBACK{edits}### FINAL PROMPT效果上论文和之前的OPRO,ProTeGi等算法都做了对比在多个数据集上都会有较显著的效果提升。最后的最后感谢你们的阅读和喜欢我收藏了很多技术干货可以共享给喜欢我文章的朋友们如果你肯花时间沉下心去学习它们一定能帮到你。因为这个行业不同于其他行业知识体系实在是过于庞大知识更新也非常快。作为一个普通人无法全部学完所以我们在提升技术的时候首先需要明确一个目标然后制定好完整的计划同时找到好的学习方法这样才能更快的提升自己。这份完整版的大模型 AI 学习资料已经上传CSDN朋友们如果需要可以微信扫描下方CSDN官方认证二维码免费领取【保证100%免费】大模型知识脑图为了成为更好的 AI大模型 开发者这里为大家提供了总的路线图。它的用处就在于你可以按照上面的知识点去找对应的学习资源保证自己学得较为全面。经典书籍阅读阅读AI大模型经典书籍可以帮助读者提高技术水平开拓视野掌握核心技术提高解决问题的能力同时也可以借鉴他人的经验。对于想要深入学习AI大模型开发的读者来说阅读经典书籍是非常有必要的。实战案例光学理论是没用的要学会跟着一起敲要动手实操才能将自己的所学运用到实际当中去这时候可以搞点实战案例来学习。面试资料我们学习AI大模型必然是想找到高薪的工作下面这些面试题都是总结当前最新、最热、最高频的面试题并且每道题都有详细的答案面试前刷完这套面试题资料小小offer不在话下640套AI大模型报告合集这套包含640份报告的合集涵盖了AI大模型的理论研究、技术实现、行业应用等多个方面。无论您是科研人员、工程师还是对AI大模型感兴趣的爱好者这套报告合集都将为您提供宝贵的信息和启示。这份完整版的大模型 AI 学习资料已经上传CSDN朋友们如果需要可以微信扫描下方CSDN官方认证二维码免费领取【保证100%免费】
算法精解与面试技巧)
![[光学原理与应用-521]：对光的错误理解与纠偏](http://pic.xiahunao.cn/yaotu/[光学原理与应用-521]：对光的错误理解与纠偏)
