History Registry Protocol【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge1. Positioning and Responsibility BoundariesThe history registry is along-term maintained protocol and data artifactused toarchive operator IR prototype information across versionsand provide query capabilities externally in a stable manner.This protocol focuses on:What the data is (field semantics and constraints)Where the data is stored (directories and packaging)How data is generated and published (currently generated bygen_esband published with Ops run package)How consumers read it (file system interface)How the protocol evolves (backward compatibility rules)This protocol does not focus on:Compatibility judgment/business decision logic2. TerminologyIR Prototype: Operators input/output/attribute/subgraph definition information at the IR layer.Historical Prototype Data (Structured Data): The collection of data files after IR prototype extraction and serialization.Producer: Tool that generates structured data. Currently fixed asgen_esb --extract-historyin the short term.Consumer: Any tool/process that reads structured data. Currently the main consumer isgen_esb(for ES API generation), can be extended to other toolchains in the future.Ops Run Package: Ops runtime package form after installation, historical prototype library data is published with it.Compatibility Window: The version range that consumers select based on version metadata (e.g., release dates) for comparison/generation (e.g., one-year window).3. Directory Structure and PackagingThe historical prototype library is published with Ops run package and organized by operator packages (such asmath/nn/cv/...).Directory structure (example, after installation):/${CANN_INSTALL_PATH}/cann/opp/history_registry/pkg/ ├── index.json └── registry/ ├── release_version_1/ │ ├── operators.json │ └── metadata.json ├── release_version_2/ │ ├── operators.json │ └── metadata.json └── ...Notes:pkg: Package name, such asmath,nn,cv.release_version_x: Version directory name, recommended to be consistent withmetadata.json.release_version(e.g.,8.0.RC1).4. Files and Data FormatsThis protocol contains three file types:index.json: Version index (which versions are available in this package)registry/ver/metadata.json: Version metadata (for selecting version range, traceability)registry/ver/operators.json: Operator prototype data for that version4.1 index.json (Version Index)Minimum fields:version: Index file schema version (string)releases[]: Version arrayrelease_version: Version number (string)release_date: Release date (string, recommendedYYYY-MM-DD)Example see Appendix A.4.2 metadata.json (Version Metadata)Recommended fields (minimum usable extensible):release_version: Version numberbranch_name: Branch name (optional, but recommended to provide)Example see Appendix A.4.3 operators.json (Operator Prototype Data)Minimum fields:operators[]op_typeinputs[]: Each item must contain at leastname/type/dtypeoutputs[]: Each item must contain at leastname/type/dtypeattrs[]: Each item must contain at leastname/type/required, can includedefault_valuesubgraphs[]: Each item must contain at leastname/typeConstraint recommendations:default_valueuses string to carry JSON literal text, needs to be parsed in combination withtype; used for cross-language consistency and to avoid numerical precision/large integer representation issues.Example see Appendix A.5. Generation and Publication (Currently Fixed by gen_esb)In the short term, to reduce system complexity:Producer: Still reusesgen_esb, extracts IR prototypes from current version prototype .so through--extract-historyand outputs structured data.5.1 Data Flow for Building Ops Package (Generation and Packaging Publication)Prerequisites:Build environment needs to install Ops package (historical prototype data comes from installed Ops run package); if no Ops package, cannot obtain historical prototype data.Ops package build data flow diagram:Notes:Historical prototype information of each version exists in Ops run package, stored in different packages (such asmath/nn/cv).Simplified data flow diagram:Generation mode (example):gen_esb --extract-history --output-dir out --release-version verPublication principles:Generate and archive this versions data before official version release.Structured data is packaged and published with Ops run package.6. Consumption Method (File System Interface)Consumers read in the following manner:Readpkg/index.jsonto get available version list and release datesSelect target version set (e.g., one-year window)Readregistry/ver/operators.jsonandmetadata.jsonto get prototypes and metadataNote:Protocol layer does not bind to any specific consumption strategy; consumption strategy (e.g., which overloads to generate, how to disambiguate) is implemented by consumers.7. Protocol Evolution and Backward CompatibilityPrinciples:Preferadding new fields, avoid deleting/renaming published fields.New fields need to provide default semantics to ensure old consumers can ignore them.When destructive changes are needed, should be distinguished byindex.json.version(or introducing explicitschema_version) and provide migration strategy.Appendix A: JSON Minimum ExampleA.1 index.json{ version: 1.0.0, releases: [ { release_version: 8.0.RC1, release_date: 2024-09-30 }, { release_version: 8.0.0, release_date: 2024-12-30 } ] }A.2 registry/ /metadata.json{ release_version: 8.0.RC1, branch_name: master }A.3 registry/ /operators.json{ operators: [ { op_type: Foo, inputs: [ { name: x, type: INPUT, dtype: TensorType({DT_FLOAT}) }, { name: xo1, type: OPTIONAL_INPUT, dtype: TensorType({DT_FLOAT}) } ], outputs: [ { name: y, type: OUTPUT, dtype: TensorType({DT_FLOAT}) } ], attrs: [ { name: a, type: Int, required: false, default_value: 0 }, { name: flag, type: Bool, required: true } ], subgraphs: [] } ] }【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考