用 Instructor 蒸馏 Chain of Density将 GPT-4 的迭代摘要能力压缩进单一微调模型【免费下载链接】instructorstructured outputs for llms项目地址: https://gitcode.com/GitHub_Trending/in/instructor本指南基于 docs/blog/posts/chain-of-density.md 与仓库内examples/chain-of-density示例完整讲解如何用 Instructor 实现 Chain of Density 迭代摘要再通过Instructions.distil蒸馏管道把多轮迭代能力固化进一个 GPT-3.5 微调模型在保持实体密度的同时把延迟降低约 20 倍、推理成本降低数十倍。Chain of Density密度链下文简称 CoD是一种迭代式摘要技术先让模型生成一段冗长、非特定的初始摘要再通过多轮识别缺失实体 → 重写为等长但更密集的摘要来不断压入文章中的实体。本文将以 Instructor 为骨架从零实现这一流程随后利用 Instructor 的蒸馏工具生成微调数据集、提交 OpenAI 微调任务最后对比微调模型与 GPT-4 在实体密度、延迟与成本上的表现。读完本文你将掌握用 Pydantic 建模迭代摘要状态、用字段校验器强制摘要质量、用instructions.distil自动录制训练样本以及用instructor jobs create-from-file一键发起微调。Part 1用 Instructor 实现 Chain of Density用 AI 摘要长文本长期面临技术不一致、结果不稳定的痛点。CoD 方法源自论文From Sparse to Dense: GPT-4 Summarization with Chain of Density PromptingAdams et al., 2023给出了一条可复现的路径模型先产出初始摘要然后经过多轮重写每轮都从原文中找出上一轮摘要遗漏的实体并补进去同时保持摘要长度基本不变。最终得到一份实体密集、信息量大且自包含的摘要作者团队发现该方法的产出稳定优于人工标注的摘要。从原始 Prompt 拆分出的迭代流程原始方法把找出缺失实体 重写这两步重复 5 次。原文的 Prompt 核心如下Article: {{ARTICLE}} You will generate increasingly concise, entity-dense summaries of the above Article. Repeat the following 2 steps 5 times. Step 1. Identify 1-3 informative Entities (; delimited) from the Article which are missing from the previously generated summary. Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. A Missing Entity is: - Relevant: to the main story. - Specific: descriptive yet concise (5 words or fewer). - Novel; not in the previous summary. - Faithful: present in the Article. - Anywhere: located anywhere in the Article. Guidelines: - The first summary should be long (4-5 sentences, -80 words) yet highly non-specific... - Make every word count: re-write the previous summary to improve flow and make space for additional entities. - Never drop entities from the previous summary. If space cannot be made, add fewer new entities. Remember, use the exact same number of words for each summary. Answer in JSON. The JSON should be a list (length 5) of dictionaries whose keys are Missing_Entities and Denser_SummaryInstructor 的价值在于把这段提示词驱动的流程改造成结构化函数调用驱动的流程每一步都是一次独立的client.create调用可以单独指定response_model从而在每一轮都做类型强制与校验。本文实现的具体做法与论文有两处差异在原文档中已明确说明使用校验器而非提示词来保证重写摘要的最短长度只做 3 轮而非 5 轮重写因此最终实体密度会略低于论文。数据建模两个核心 response_model首先安装依赖对应 examples/chain-of-density/requirements.txt 的内容pip install instructor aiohttp rich以及 NLTK 分词所需的 punkt 资源nltk.download(punkt)spaCy 语言模型en_core_web_smpython -m spacy download en_core_web_sm。初始摘要 InitialSummary第一段摘要刻意要求冗长、高度非特定、充满 filler约 80 词。这个需求被直接编码进 Pydantic 模型的 docstring——docstring 不是装饰它们会被直接用作给 LLM 的指令class InitialSummary(BaseModel): This is an initial summary which should be long ( 4-5 sentences, ~80 words) yet highly non-specific, containing little information beyond the entities marked as missing. Use overly verbose languages and fillers (Eg. This article discusses) to reach ~80 words. summary: str Field( ..., descriptionThis is a summary of the article provided which is overly verbose and uses fillers. It should be roughly 80 words in length, )重写摘要 RewrittenSummary每一轮重写需要同时建模三个信息新的摘要文本、本轮遗漏的实体missing、被错误丢弃的上一轮实体absent。missing与absent会在下一轮作为反馈信号注入消息形成闭环class RewrittenSummary(BaseModel): This is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. Guidelines - Make every word count : Rewrite the previous summary to improve flow and make space for additional entities - Never drop entities from the previous summary. If space cannot be made, add fewer new entities. - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article. - Make space with fusion, compression, and removal of uninformative phrases like the article discusses - Missing entities can appear anywhere in the new summary An Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title. summary: str Field( ..., descriptionThis is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. It should have the same length ( ~ 80 words ) as the previous summary and should be easily understood without the Article, ) absent: List[str] Field( ..., default_factorylist, descriptionthis is a list of Entities found absent from the new summary that were present in the previous summary, ) missing: List[str] Field( default_factorylist, descriptionThis is a list of 1-3 informative Entities from the Article that are missing from the new summary which should be included in the next generated summary., )docstring 为什么会生效response_model 到 function call 的转换Instructor 会把传入的response_model解析成一次 OpenAI 函数调用因此最终输出与 Pydantic 模型强绑定。以用于微调的GeneratedSummary为例class GeneratedSummary(BaseModel): This represents a highly concise summary that includes as many entities as possible from the original source article. An Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title. Guidelines - Make every word count - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article. - Make space with fusion, compression, and removal of uninformative phrases like the article discusses summary: str Field( ..., descriptionThis represents the final summary generated that captures the meaning of the original article which is as concise as possible. , )它会被展开成下面的函数调用结构{ functions: [ { name: GeneratedSummary, description: This represents a highly concise summary that includes as many entities as possible from the original source article.\n\nAn Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title.\n\nGuidelines\n- Make every word count\n- The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.\n- Make space with fusion, compression, and removal of uninformative phrases like \the article discusses\, parameters: { type: object, properties: { summary: { description: This represents the final summary generated that captures the meaning of the original article which is as concise as possible. , title: Summary, type: string } }, required: [summary] } } ] }这一转换在源码层面由 instructor/processing/function_calls.py 中的response_schema完成见 instructor/distil.py 中schema_model.openai_schema的调用。所以 docstring 与Field(description...)写得越精细LLM 越能产出符合预期的结构而因为底层就是 Pydantic你还可以对返回结果做任意校验与解析——Its all python all the way down。用 Pydantic 校验器强制摘要质量理想情况下我们希望missing长度在 1~3 之间、absent为空列表、重写摘要保持最低实体密度。这些都可以用原生 Pydanticfield_validator声明在类内部Instructor 会自动在每次生成后执行校验不通过就触发重试import nltk import spacy nlp spacy.load(en_core_web_sm) field_validator(summary) classmethod def min_length(_cls, v: str): tokens nltk.word_tokenize(v) # 与论文一致用 NLTK 分词器统计 token 数 num_tokens len(tokens) if num_tokens 60: raise ValueError( The current summary is too short. Please make sure that you generate a new summary that is around 80 words long. ) return v field_validator(missing) classmethod def has_missing_entities(_cls, missing_entities: List[str]): if len(missing_entities) 0: raise ValueError( You must identify 1-3 informative Entities from the Article which are missing from the previously generated summary to be used in a new summary ) return missing_entities field_validator(absent) classmethod def has_no_absent_entities(_cls, absent_entities: List[str]): absent_entity_string ,.join(absent_entities) if len(absent_entities) 0: print(fDetected absent entities of {absent_entity_string}) raise ValueError( fDo not omit the following Entities {absent_entity_string} from the new summary ) return absent_entities field_validator(summary) classmethod def min_entity_density(_cls, v: str): tokens nltk.word_tokenize(v) num_tokens len(tokens) # 用 spaCy 提取实体计算实体密度 doc nlp(v) num_entities len(doc.ents) density num_entities / num_tokens if density 0.08: # 0.08 是任意选择的经验阈值 raise ValueError( fThe summary of {v} has too few entities. Please regenerate a new summary with more new entities added to it. Remember that new entities can be added at any point of the summary. ) return v四个校验器的作用分别是min_length与论文一致用 NLTK 分词器统计 token 数目标至少 60 个 token避免重写后信息丢失has_missing_entities每轮必须识别出至少 1 个缺失实体否则无法推进迭代has_no_absent_entities禁止从上一轮摘要中丢弃任何实体检测到即提示并报错触发重试min_entity_density用 spaCy 计算实体密度实体数 / token 数低于 0.08 时强制重新生成——这样密度只升不降。关于校验器与 Instructor 的配合可参考仓库中的专题文章 Good LLM Validation is just Good Validation。把流程串起来summarize_article下面实现完整的 CoD 摘要函数对应示例 examples/chain-of-density/chain_of_density.pyimport instructor client instructor.from_provider(openai/gpt-5-nano) # 示例代码中使用 instructor.from_openai(OpenAI()) def summarize_article(article: str, summary_steps: int 3): summary_chain [] # 第一步生成初始摘要冗长、非特定、约 80 词 summary: InitialSummary client.create( modelgpt-5.4-mini, response_modelInitialSummary, messages[ { role: system, content: Write a summary about the article that is long (4-5 sentences) yet highly non-specific. Use overly, verbose language and fillers(eg.,this article discusses) to reach ~80 words, }, {role: user, content: fHere is the Article: {article}}, { role: user, content: The generated summary should be about 80 words., }, ], max_retries2, ) prev_summary None summary_chain.append(summary.summary) # 后续每一轮识别缺失实体 - 重写更密集的等长摘要 for _ in range(summary_steps): missing_entity_message ( [] if prev_summary is None else [ { role: user, content: fPlease include these Missing Entities: {,.join(prev_summary.missing)}, }, ] ) new_summary: RewrittenSummary client.create( modelgpt-5.4-mini, messages[ { role: system, content: You are going to generate an increasingly concise,entity-dense summary of the following article. Perform the following two tasks - Identify 1-3 informative entities from the following article which is missing from the previous summary - Write a new denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities Guidelines - Make every word count: re-write the previous summary to improve flow and make space for additional entities - Make space with fusion, compression, and removal of uninformative phrases like the article discusses. - The summaries should become highly dense and concise yet self-contained, e.g., easily understood without the Article. - Missing entities can appear anywhere in the new summary - Never drop entities from the previous summary. If space cannot be made, add fewer new entities. , }, {role: user, content: fHere is the Article: {article}}, { role: user, content: fHere is the previous summary: {summary_chain[-1]}, }, *missing_entity_message, ], max_retries3, # 若你把密度阈值调高到 0.08 以上可相应增大该值 max_tokens1000, response_modelRewrittenSummary, ) summary_chain.append(new_summary.summary) prev_summary new_summary return summary_chain几个关键点对 OpenAI 客户端应用from_provider旧版本为from_openai或patch后即可获得 Instructor 的全部能力输出自动类型强转 非法输出自动重试初始摘要的系统提示明确要求冗长 充满 filler 约 80 词为后续压入实体预留空间重写轮次对原论文 prompt 做了小幅改编且会触发前面定义的所有field_validatormax_retries3与密度阈值 0.08 是配套的如果你把阈值调大应同步调大重试次数否则可能频繁重试耗尽配额。实际运行中以示例仓库使用gpt-4-0613为例同样长度的文本首轮与末轮的差异非常直观——实体数量成倍增长措辞也从灌水变得自然、信息密集第一轮初始摘要This article discusses the highly-anticipated boxing match between Manny Pacquiao and Floyd Mayweather. The article revolves around Manny Pacquiaos statements about his upcoming fight and his preparations for the same. A portion of the article provides details about the financial stipulations of the match and its significance in the sporting arena. Quotes from Pacquiao illustrating his determination and his battle strategy are highlighted. The tone of the article is largely centered around creating a build-up to the upcoming mega event.最后一轮实体密集摘要Manny Pacquiao, the Filipino boxer, anticipates the forthcoming May 2 showdown at the MGM Grand as the fight of his life, against the undefeated American Floyd Mayweather, in a $300m bout. Despite being seen as the underdog in this high-stakes Las Vegas match, Pacquiao is confident, promising a warriors spirit and assuring the fans who have been awaiting this encounter for a decade, that it will indeed be the biggest sporting spectacle in history worthy of their anticipationPart 2把迭代方法蒸馏进单一模型CoD 每篇摘要要发起多次串行 API 调用延迟与成本都很高。更聪明的做法是让 GPT-4 跑完整 CoD 流程生成金标摘要再用这些数据微调一个小模型让它在单次调用里直接产出同等质量的摘要。生成训练集instructions.distil为了防止数据污染作者从griffin/chain-of-density数据集中随机抽取了 120 篇文章拆成train.csv与test.csv作者将生成数据上传至 Hugging Face 供复现。接下来用 Instructor 的Instructions模块把每次调用自动录制成.jsonl训练文件完整脚本见 examples/chain-of-density/finetune.pyfrom typing import List from chain_of_density import summarize_article # 复用上面定义的函数 import csv import logging import instructor from pydantic import BaseModel client instructor.from_provider(openai/gpt-5-nano) # 示例代码中使用 instructor.from_openai(OpenAI()) logging.basicConfig(levellogging.INFO) # 必须配置 INFO 级别日志否则不会输出训练数据 instructions instructor.Instructions( nameChain Of Density, finetune_formatmessages, # log handler 用于把数据保存到文件也可以换成数据库等任意存储 log_handlers[logging.FileHandler(generated.jsonl)], openai_clientclient, ) class GeneratedSummary(BaseModel): This represents a highly concise summary that includes as many entities as possible from the original source article. An Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title. Guidelines - Make every word count - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article. - Make space with fusion, compression, and removal of uninformative phrases like the article discusses summary: str Field( ..., descriptionThis represents the final summary generated that captures the meaning of the original article which is as concise as possible. , ) instructions.distil # 自动捕获函数的输入与输出 def distil_summarization(text: str) - GeneratedSummary: summary_chain: List[str] summarize_article(text) return GeneratedSummary(summarysummary_chain[-1]) # 取链条最后一轮的摘要作为金标 with open(train.csv) as file: reader csv.reader(file) next(reader) # Skip the header for article, _summary in reader: # Run Distillisation to generate the values distil_summarization(article)脚本要点logging.basicConfig(levellogging.INFO)必须配置蒸馏数据是通过日志处理器落盘的不开启 INFO 日志就不会生成generated.jsonlInstructions的log_handlers参数决定数据写到哪这里用logging.FileHandler(generated.jsonl)instructions.distil装饰器要求函数返回类型注解必须是 Pydantic BaseModel 且实际返回 Pydantic 对象——从源码 instructor/distil.py 可以看到is_return_type_base_model_or_instance会强制断言这一点数据录制走的是messages格式FinetuneFormat.MESSAGES源码 instructor/distil.py 会把函数的系统/用户消息、函数签名、以及response_model的 JSON 输出拼装成一次带function_call的完整对话记录这正是 OpenAI 微调所要求的格式。建议先在数据集的小子集上跑一遍确认配置正确。正式运行前记得设置OPENAI_API_KEY环境变量并按需用 tenacity 增加限流重试。创建微调任务脚本跑完后本地会生成generated.jsonl。接下来只需要一条命令即可发起微调instructor jobs create-from-file generated.jsonlCLI 提供四个子命令create-from-file/create-from-id/list/cancel完整说明见 docs/cli/finetune.md。create-from-file会把上传文件 发起训练一步完成常用参数包括参数说明默认值--model用于微调的基础模型gpt-5.4-mini--n-epochs训练轮数由调度器决定--batch-size批大小未指定--learning-rate-multiplier学习率倍率未指定--validation-file验证集文件路径None--model-suffix模型标识后缀None--poll轮询间隔秒2例如带验证集与超参数的一次训练instructor jobs create-from-file generated.jsonl \ --validation_file validation.jsonl \ --n_epochs 3 \ --batch_size 16 \ --learning_rate_multiplier 0.5训练期间可用instructor jobs list实时监控任务状态每 5 秒自动刷新用instructor files list查看已上传文件。任务完成后只需把instructions.distil改为显式指定微调模型并以dispatch模式运行就能用新模型直接出结果instructions.distil(modelgpt-5.4-mini:finetuned-123, modedispatch) # 替换成你的模型 id def distil_summarization(text: str) - GeneratedSummary: summary_chain: List[str] summarize_article(text) return GeneratedSummary(summarysummary_chain[-1])OpenAI 的微调模型 id 形如ft:gpt-5.4-mini:personal::id可在其仪表盘 Fine-tuning 页签下找到。源码 instructor/distil.py 显示distil支持两种模式modedistil默认执行函数并录制数据与modedispatch不再执行原函数而是直接把输入打包成消息发给微调模型走response_model结构化返回——这正是从蒸馏数据切换到生产推理的开关。结果与基准对比作者用 20 篇未参与微调的文章从三个维度对比了几种方案实体密度实体数/token越高越好、延迟生成最后一个 token 的秒数、成本拆分为训练成本与推理成本。3.5 Finetuned (n)在 n 个样本上微调的 GPT-3.5 模型每个模型训练 4~5 个 epochepoch 数由 OpenAI 调度器自动决定GPT-4 (COD)用上述方法对 GPT-4 应用 3 轮 CoD 重写GPT-3.5 (Vanilla)单次调用生成 80~90 token 的实体密集摘要作为基线。ModelMean Latency (s)Mean Entity Density3.5 Finetuned (20)2.10.153.5 Finetuned (50)2.10.143.5 Finetuned (76)2.10.14GPT-3.5 (Vanilla)16.80.12GPT-4 (COD)49.50.15成本方面基于 OpenAI Usage Dashboard 对 20 篇摘要的统计ModelTraining Cost ($)Inference Cost ($)Tokens UsedTotal Cost ($)GPT-3.5 (Vanilla)-0.2051,1620.23.5 Finetuned (20)0.70.2056,5730.83.5 Finetuned (50)1.40.1749,0571.33.5 Finetuned (76)1.80.1751,5832.5GPT-4 (COD)-12.9409,06212.9根据作者测算GPT-4 每篇摘要的推理成本约 0.65 美元而微调模型仅约 0.0091 美元便宜约 72 倍延迟从 GPT-4 (COD) 的 49.5 秒降到 2.1 秒约 20x 加速综合训练与推理总成本节省约 50 倍这正是原文档 description 中 20x latency reduction、50x cost savings 的来源。一个值得注意的现象样本最少的微调模型20 例实体密度反而最高。作者给出的推测是要么默认 5 个 epoch 训练不足要么样本更多后模型开始模仿其他行为如更抽象的写作风格反而拉低了实体密度。这里补充一个原文档提及但更严格的进阶策略构建微调数据集时只保留密度 ≥ 0.15 的摘要、取整条链中密度最高的一轮作为金标、强制每次重写密度 ≥ 0.12、不达标最多重试 3 次——这种策略成本约为本教程的 2.5 倍以上作者实测生成 75 例共花费 63.46 美元约 0.85 美元/例但对性能提升明显适合对质量要求更高的场景。结论将 CoD 这种迭代方法蒸馏进单一微调模型可以获得约 20~40 倍的加速同时整体质量不降反升——这正是用蒸馏把昂贵能力固化进专用小模型带来的效率红利。从数据建模、字段校验、蒸馏录制到微调任务下发Instructor 在整个链条上都提供了结构化、可复现的工具支持用 Pydantic 模型 docstring 直接塑造 LLM 输出结构用field_validator在每一轮强制摘要长度与实体密度用Instructions.distil一键把函数调用录制为messages格式的微调数据用instructor jobsCLI 一行发起训练、监控进度。完整的可运行示例数据建模、CoD 流程、蒸馏脚本、依赖清单都在仓库 examples/chain-of-density 目录下相关 CLI 细节可查阅 docs/cli/finetune.mdInstructions与distil的底层实现见 instructor/distil.py。【免费下载链接】instructorstructured outputs for llms项目地址: https://gitcode.com/GitHub_Trending/in/instructor创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 用 Instructor 蒸馏 Chain of Density：将 GPT-4 的迭代摘要能力压缩进单一微调模型
用 Instructor 蒸馏 Chain of Density将 GPT-4 的迭代摘要能力压缩进单一微调模型【免费下载链接】instructorstructured outputs for llms项目地址: https://gitcode.com/GitHub_Trending/in/instructor本指南基于 docs/blog/posts/chain-of-density.md 与仓库内examples/chain-of-density示例完整讲解如何用 Instructor 实现 Chain of Density 迭代摘要再通过Instructions.distil蒸馏管道把多轮迭代能力固化进一个 GPT-3.5 微调模型在保持实体密度的同时把延迟降低约 20 倍、推理成本降低数十倍。Chain of Density密度链下文简称 CoD是一种迭代式摘要技术先让模型生成一段冗长、非特定的初始摘要再通过多轮识别缺失实体 → 重写为等长但更密集的摘要来不断压入文章中的实体。本文将以 Instructor 为骨架从零实现这一流程随后利用 Instructor 的蒸馏工具生成微调数据集、提交 OpenAI 微调任务最后对比微调模型与 GPT-4 在实体密度、延迟与成本上的表现。读完本文你将掌握用 Pydantic 建模迭代摘要状态、用字段校验器强制摘要质量、用instructions.distil自动录制训练样本以及用instructor jobs create-from-file一键发起微调。Part 1用 Instructor 实现 Chain of Density用 AI 摘要长文本长期面临技术不一致、结果不稳定的痛点。CoD 方法源自论文From Sparse to Dense: GPT-4 Summarization with Chain of Density PromptingAdams et al., 2023给出了一条可复现的路径模型先产出初始摘要然后经过多轮重写每轮都从原文中找出上一轮摘要遗漏的实体并补进去同时保持摘要长度基本不变。最终得到一份实体密集、信息量大且自包含的摘要作者团队发现该方法的产出稳定优于人工标注的摘要。从原始 Prompt 拆分出的迭代流程原始方法把找出缺失实体 重写这两步重复 5 次。原文的 Prompt 核心如下Article: {{ARTICLE}} You will generate increasingly concise, entity-dense summaries of the above Article. Repeat the following 2 steps 5 times. Step 1. Identify 1-3 informative Entities (; delimited) from the Article which are missing from the previously generated summary. Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. A Missing Entity is: - Relevant: to the main story. - Specific: descriptive yet concise (5 words or fewer). - Novel; not in the previous summary. - Faithful: present in the Article. - Anywhere: located anywhere in the Article. Guidelines: - The first summary should be long (4-5 sentences, -80 words) yet highly non-specific... - Make every word count: re-write the previous summary to improve flow and make space for additional entities. - Never drop entities from the previous summary. If space cannot be made, add fewer new entities. Remember, use the exact same number of words for each summary. Answer in JSON. The JSON should be a list (length 5) of dictionaries whose keys are Missing_Entities and Denser_SummaryInstructor 的价值在于把这段提示词驱动的流程改造成结构化函数调用驱动的流程每一步都是一次独立的client.create调用可以单独指定response_model从而在每一轮都做类型强制与校验。本文实现的具体做法与论文有两处差异在原文档中已明确说明使用校验器而非提示词来保证重写摘要的最短长度只做 3 轮而非 5 轮重写因此最终实体密度会略低于论文。数据建模两个核心 response_model首先安装依赖对应 examples/chain-of-density/requirements.txt 的内容pip install instructor aiohttp rich以及 NLTK 分词所需的 punkt 资源nltk.download(punkt)spaCy 语言模型en_core_web_smpython -m spacy download en_core_web_sm。初始摘要 InitialSummary第一段摘要刻意要求冗长、高度非特定、充满 filler约 80 词。这个需求被直接编码进 Pydantic 模型的 docstring——docstring 不是装饰它们会被直接用作给 LLM 的指令class InitialSummary(BaseModel): This is an initial summary which should be long ( 4-5 sentences, ~80 words) yet highly non-specific, containing little information beyond the entities marked as missing. Use overly verbose languages and fillers (Eg. This article discusses) to reach ~80 words. summary: str Field( ..., descriptionThis is a summary of the article provided which is overly verbose and uses fillers. It should be roughly 80 words in length, )重写摘要 RewrittenSummary每一轮重写需要同时建模三个信息新的摘要文本、本轮遗漏的实体missing、被错误丢弃的上一轮实体absent。missing与absent会在下一轮作为反馈信号注入消息形成闭环class RewrittenSummary(BaseModel): This is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. Guidelines - Make every word count : Rewrite the previous summary to improve flow and make space for additional entities - Never drop entities from the previous summary. If space cannot be made, add fewer new entities. - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article. - Make space with fusion, compression, and removal of uninformative phrases like the article discusses - Missing entities can appear anywhere in the new summary An Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title. summary: str Field( ..., descriptionThis is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. It should have the same length ( ~ 80 words ) as the previous summary and should be easily understood without the Article, ) absent: List[str] Field( ..., default_factorylist, descriptionthis is a list of Entities found absent from the new summary that were present in the previous summary, ) missing: List[str] Field( default_factorylist, descriptionThis is a list of 1-3 informative Entities from the Article that are missing from the new summary which should be included in the next generated summary., )docstring 为什么会生效response_model 到 function call 的转换Instructor 会把传入的response_model解析成一次 OpenAI 函数调用因此最终输出与 Pydantic 模型强绑定。以用于微调的GeneratedSummary为例class GeneratedSummary(BaseModel): This represents a highly concise summary that includes as many entities as possible from the original source article. An Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title. Guidelines - Make every word count - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article. - Make space with fusion, compression, and removal of uninformative phrases like the article discusses summary: str Field( ..., descriptionThis represents the final summary generated that captures the meaning of the original article which is as concise as possible. , )它会被展开成下面的函数调用结构{ functions: [ { name: GeneratedSummary, description: This represents a highly concise summary that includes as many entities as possible from the original source article.\n\nAn Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title.\n\nGuidelines\n- Make every word count\n- The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.\n- Make space with fusion, compression, and removal of uninformative phrases like \the article discusses\, parameters: { type: object, properties: { summary: { description: This represents the final summary generated that captures the meaning of the original article which is as concise as possible. , title: Summary, type: string } }, required: [summary] } } ] }这一转换在源码层面由 instructor/processing/function_calls.py 中的response_schema完成见 instructor/distil.py 中schema_model.openai_schema的调用。所以 docstring 与Field(description...)写得越精细LLM 越能产出符合预期的结构而因为底层就是 Pydantic你还可以对返回结果做任意校验与解析——Its all python all the way down。用 Pydantic 校验器强制摘要质量理想情况下我们希望missing长度在 1~3 之间、absent为空列表、重写摘要保持最低实体密度。这些都可以用原生 Pydanticfield_validator声明在类内部Instructor 会自动在每次生成后执行校验不通过就触发重试import nltk import spacy nlp spacy.load(en_core_web_sm) field_validator(summary) classmethod def min_length(_cls, v: str): tokens nltk.word_tokenize(v) # 与论文一致用 NLTK 分词器统计 token 数 num_tokens len(tokens) if num_tokens 60: raise ValueError( The current summary is too short. Please make sure that you generate a new summary that is around 80 words long. ) return v field_validator(missing) classmethod def has_missing_entities(_cls, missing_entities: List[str]): if len(missing_entities) 0: raise ValueError( You must identify 1-3 informative Entities from the Article which are missing from the previously generated summary to be used in a new summary ) return missing_entities field_validator(absent) classmethod def has_no_absent_entities(_cls, absent_entities: List[str]): absent_entity_string ,.join(absent_entities) if len(absent_entities) 0: print(fDetected absent entities of {absent_entity_string}) raise ValueError( fDo not omit the following Entities {absent_entity_string} from the new summary ) return absent_entities field_validator(summary) classmethod def min_entity_density(_cls, v: str): tokens nltk.word_tokenize(v) num_tokens len(tokens) # 用 spaCy 提取实体计算实体密度 doc nlp(v) num_entities len(doc.ents) density num_entities / num_tokens if density 0.08: # 0.08 是任意选择的经验阈值 raise ValueError( fThe summary of {v} has too few entities. Please regenerate a new summary with more new entities added to it. Remember that new entities can be added at any point of the summary. ) return v四个校验器的作用分别是min_length与论文一致用 NLTK 分词器统计 token 数目标至少 60 个 token避免重写后信息丢失has_missing_entities每轮必须识别出至少 1 个缺失实体否则无法推进迭代has_no_absent_entities禁止从上一轮摘要中丢弃任何实体检测到即提示并报错触发重试min_entity_density用 spaCy 计算实体密度实体数 / token 数低于 0.08 时强制重新生成——这样密度只升不降。关于校验器与 Instructor 的配合可参考仓库中的专题文章 Good LLM Validation is just Good Validation。把流程串起来summarize_article下面实现完整的 CoD 摘要函数对应示例 examples/chain-of-density/chain_of_density.pyimport instructor client instructor.from_provider(openai/gpt-5-nano) # 示例代码中使用 instructor.from_openai(OpenAI()) def summarize_article(article: str, summary_steps: int 3): summary_chain [] # 第一步生成初始摘要冗长、非特定、约 80 词 summary: InitialSummary client.create( modelgpt-5.4-mini, response_modelInitialSummary, messages[ { role: system, content: Write a summary about the article that is long (4-5 sentences) yet highly non-specific. Use overly, verbose language and fillers(eg.,this article discusses) to reach ~80 words, }, {role: user, content: fHere is the Article: {article}}, { role: user, content: The generated summary should be about 80 words., }, ], max_retries2, ) prev_summary None summary_chain.append(summary.summary) # 后续每一轮识别缺失实体 - 重写更密集的等长摘要 for _ in range(summary_steps): missing_entity_message ( [] if prev_summary is None else [ { role: user, content: fPlease include these Missing Entities: {,.join(prev_summary.missing)}, }, ] ) new_summary: RewrittenSummary client.create( modelgpt-5.4-mini, messages[ { role: system, content: You are going to generate an increasingly concise,entity-dense summary of the following article. Perform the following two tasks - Identify 1-3 informative entities from the following article which is missing from the previous summary - Write a new denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities Guidelines - Make every word count: re-write the previous summary to improve flow and make space for additional entities - Make space with fusion, compression, and removal of uninformative phrases like the article discusses. - The summaries should become highly dense and concise yet self-contained, e.g., easily understood without the Article. - Missing entities can appear anywhere in the new summary - Never drop entities from the previous summary. If space cannot be made, add fewer new entities. , }, {role: user, content: fHere is the Article: {article}}, { role: user, content: fHere is the previous summary: {summary_chain[-1]}, }, *missing_entity_message, ], max_retries3, # 若你把密度阈值调高到 0.08 以上可相应增大该值 max_tokens1000, response_modelRewrittenSummary, ) summary_chain.append(new_summary.summary) prev_summary new_summary return summary_chain几个关键点对 OpenAI 客户端应用from_provider旧版本为from_openai或patch后即可获得 Instructor 的全部能力输出自动类型强转 非法输出自动重试初始摘要的系统提示明确要求冗长 充满 filler 约 80 词为后续压入实体预留空间重写轮次对原论文 prompt 做了小幅改编且会触发前面定义的所有field_validatormax_retries3与密度阈值 0.08 是配套的如果你把阈值调大应同步调大重试次数否则可能频繁重试耗尽配额。实际运行中以示例仓库使用gpt-4-0613为例同样长度的文本首轮与末轮的差异非常直观——实体数量成倍增长措辞也从灌水变得自然、信息密集第一轮初始摘要This article discusses the highly-anticipated boxing match between Manny Pacquiao and Floyd Mayweather. The article revolves around Manny Pacquiaos statements about his upcoming fight and his preparations for the same. A portion of the article provides details about the financial stipulations of the match and its significance in the sporting arena. Quotes from Pacquiao illustrating his determination and his battle strategy are highlighted. The tone of the article is largely centered around creating a build-up to the upcoming mega event.最后一轮实体密集摘要Manny Pacquiao, the Filipino boxer, anticipates the forthcoming May 2 showdown at the MGM Grand as the fight of his life, against the undefeated American Floyd Mayweather, in a $300m bout. Despite being seen as the underdog in this high-stakes Las Vegas match, Pacquiao is confident, promising a warriors spirit and assuring the fans who have been awaiting this encounter for a decade, that it will indeed be the biggest sporting spectacle in history worthy of their anticipationPart 2把迭代方法蒸馏进单一模型CoD 每篇摘要要发起多次串行 API 调用延迟与成本都很高。更聪明的做法是让 GPT-4 跑完整 CoD 流程生成金标摘要再用这些数据微调一个小模型让它在单次调用里直接产出同等质量的摘要。生成训练集instructions.distil为了防止数据污染作者从griffin/chain-of-density数据集中随机抽取了 120 篇文章拆成train.csv与test.csv作者将生成数据上传至 Hugging Face 供复现。接下来用 Instructor 的Instructions模块把每次调用自动录制成.jsonl训练文件完整脚本见 examples/chain-of-density/finetune.pyfrom typing import List from chain_of_density import summarize_article # 复用上面定义的函数 import csv import logging import instructor from pydantic import BaseModel client instructor.from_provider(openai/gpt-5-nano) # 示例代码中使用 instructor.from_openai(OpenAI()) logging.basicConfig(levellogging.INFO) # 必须配置 INFO 级别日志否则不会输出训练数据 instructions instructor.Instructions( nameChain Of Density, finetune_formatmessages, # log handler 用于把数据保存到文件也可以换成数据库等任意存储 log_handlers[logging.FileHandler(generated.jsonl)], openai_clientclient, ) class GeneratedSummary(BaseModel): This represents a highly concise summary that includes as many entities as possible from the original source article. An Entity is a real-world object thats assigned a name - for example, a person, country a product or a book title. Guidelines - Make every word count - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article. - Make space with fusion, compression, and removal of uninformative phrases like the article discusses summary: str Field( ..., descriptionThis represents the final summary generated that captures the meaning of the original article which is as concise as possible. , ) instructions.distil # 自动捕获函数的输入与输出 def distil_summarization(text: str) - GeneratedSummary: summary_chain: List[str] summarize_article(text) return GeneratedSummary(summarysummary_chain[-1]) # 取链条最后一轮的摘要作为金标 with open(train.csv) as file: reader csv.reader(file) next(reader) # Skip the header for article, _summary in reader: # Run Distillisation to generate the values distil_summarization(article)脚本要点logging.basicConfig(levellogging.INFO)必须配置蒸馏数据是通过日志处理器落盘的不开启 INFO 日志就不会生成generated.jsonlInstructions的log_handlers参数决定数据写到哪这里用logging.FileHandler(generated.jsonl)instructions.distil装饰器要求函数返回类型注解必须是 Pydantic BaseModel 且实际返回 Pydantic 对象——从源码 instructor/distil.py 可以看到is_return_type_base_model_or_instance会强制断言这一点数据录制走的是messages格式FinetuneFormat.MESSAGES源码 instructor/distil.py 会把函数的系统/用户消息、函数签名、以及response_model的 JSON 输出拼装成一次带function_call的完整对话记录这正是 OpenAI 微调所要求的格式。建议先在数据集的小子集上跑一遍确认配置正确。正式运行前记得设置OPENAI_API_KEY环境变量并按需用 tenacity 增加限流重试。创建微调任务脚本跑完后本地会生成generated.jsonl。接下来只需要一条命令即可发起微调instructor jobs create-from-file generated.jsonlCLI 提供四个子命令create-from-file/create-from-id/list/cancel完整说明见 docs/cli/finetune.md。create-from-file会把上传文件 发起训练一步完成常用参数包括参数说明默认值--model用于微调的基础模型gpt-5.4-mini--n-epochs训练轮数由调度器决定--batch-size批大小未指定--learning-rate-multiplier学习率倍率未指定--validation-file验证集文件路径None--model-suffix模型标识后缀None--poll轮询间隔秒2例如带验证集与超参数的一次训练instructor jobs create-from-file generated.jsonl \ --validation_file validation.jsonl \ --n_epochs 3 \ --batch_size 16 \ --learning_rate_multiplier 0.5训练期间可用instructor jobs list实时监控任务状态每 5 秒自动刷新用instructor files list查看已上传文件。任务完成后只需把instructions.distil改为显式指定微调模型并以dispatch模式运行就能用新模型直接出结果instructions.distil(modelgpt-5.4-mini:finetuned-123, modedispatch) # 替换成你的模型 id def distil_summarization(text: str) - GeneratedSummary: summary_chain: List[str] summarize_article(text) return GeneratedSummary(summarysummary_chain[-1])OpenAI 的微调模型 id 形如ft:gpt-5.4-mini:personal::id可在其仪表盘 Fine-tuning 页签下找到。源码 instructor/distil.py 显示distil支持两种模式modedistil默认执行函数并录制数据与modedispatch不再执行原函数而是直接把输入打包成消息发给微调模型走response_model结构化返回——这正是从蒸馏数据切换到生产推理的开关。结果与基准对比作者用 20 篇未参与微调的文章从三个维度对比了几种方案实体密度实体数/token越高越好、延迟生成最后一个 token 的秒数、成本拆分为训练成本与推理成本。3.5 Finetuned (n)在 n 个样本上微调的 GPT-3.5 模型每个模型训练 4~5 个 epochepoch 数由 OpenAI 调度器自动决定GPT-4 (COD)用上述方法对 GPT-4 应用 3 轮 CoD 重写GPT-3.5 (Vanilla)单次调用生成 80~90 token 的实体密集摘要作为基线。ModelMean Latency (s)Mean Entity Density3.5 Finetuned (20)2.10.153.5 Finetuned (50)2.10.143.5 Finetuned (76)2.10.14GPT-3.5 (Vanilla)16.80.12GPT-4 (COD)49.50.15成本方面基于 OpenAI Usage Dashboard 对 20 篇摘要的统计ModelTraining Cost ($)Inference Cost ($)Tokens UsedTotal Cost ($)GPT-3.5 (Vanilla)-0.2051,1620.23.5 Finetuned (20)0.70.2056,5730.83.5 Finetuned (50)1.40.1749,0571.33.5 Finetuned (76)1.80.1751,5832.5GPT-4 (COD)-12.9409,06212.9根据作者测算GPT-4 每篇摘要的推理成本约 0.65 美元而微调模型仅约 0.0091 美元便宜约 72 倍延迟从 GPT-4 (COD) 的 49.5 秒降到 2.1 秒约 20x 加速综合训练与推理总成本节省约 50 倍这正是原文档 description 中 20x latency reduction、50x cost savings 的来源。一个值得注意的现象样本最少的微调模型20 例实体密度反而最高。作者给出的推测是要么默认 5 个 epoch 训练不足要么样本更多后模型开始模仿其他行为如更抽象的写作风格反而拉低了实体密度。这里补充一个原文档提及但更严格的进阶策略构建微调数据集时只保留密度 ≥ 0.15 的摘要、取整条链中密度最高的一轮作为金标、强制每次重写密度 ≥ 0.12、不达标最多重试 3 次——这种策略成本约为本教程的 2.5 倍以上作者实测生成 75 例共花费 63.46 美元约 0.85 美元/例但对性能提升明显适合对质量要求更高的场景。结论将 CoD 这种迭代方法蒸馏进单一微调模型可以获得约 20~40 倍的加速同时整体质量不降反升——这正是用蒸馏把昂贵能力固化进专用小模型带来的效率红利。从数据建模、字段校验、蒸馏录制到微调任务下发Instructor 在整个链条上都提供了结构化、可复现的工具支持用 Pydantic 模型 docstring 直接塑造 LLM 输出结构用field_validator在每一轮强制摘要长度与实体密度用Instructions.distil一键把函数调用录制为messages格式的微调数据用instructor jobsCLI 一行发起训练、监控进度。完整的可运行示例数据建模、CoD 流程、蒸馏脚本、依赖清单都在仓库 examples/chain-of-density 目录下相关 CLI 细节可查阅 docs/cli/finetune.mdInstructions与distil的底层实现见 instructor/distil.py。【免费下载链接】instructorstructured outputs for llms项目地址: https://gitcode.com/GitHub_Trending/in/instructor创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考