WIP: Proposed CycloneDX v2.0 additions to the AI/ML property Taxonomy - #175
WIP: Proposed CycloneDX v2.0 additions to the AI/ML property Taxonomy#175mrutkows wants to merge 20 commits into
Conversation
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
JunHwan-Kwon
left a comment
There was a problem hiding this comment.
@mrutkows Thank you for the overview in today's working group meeting and for adding the affine quantization vocabulary and worked examples so quickly. I reviewed the affine additions in this PR.
The direction aligns well with the use case I raised. To clarify its scope, I was proposing the optional recording of affine mapping parameters for each schema-addressable external input or output of a hash-identified model artifact. I was not proposing the enumeration of every internal layer or weight tensor.
I have one structural concern with the current worked examples, together with several localized wording and example points. Concretely, the main change I am suggesting is to move the external-interface quantization properties from the aggregate modelParameters.properties array to the sibling properties array of the relevant named input or output. The remaining items are localized corrections to the new table wording and examples.
Parameter binding
My main concern is the placement of the worked examples. Both currently place their quantization properties under modelCard.modelParameters.properties. At that level, the values are not bound to a particular named input or output.
Different external tensors may use the same integer dataType while having different scale and zeroPoint values. An aggregate model-level property set therefore loses the distinction needed to compare the affine mapping used by a preprocessing or runtime harness with the mapping declared for a named model input or output.
The evidence I shared in CycloneDX/specification#948 is not limited to the MobileNetV2 example. In a predeclared, SHA-256-pinned corpus of 50 public TFLite artifacts spanning four quantization-focused subcohorts, 62 of 114 external parameter instances across 32 artifacts carried complete affine mappings, and 30 artifacts contained more than one distinct complete external affine contract.
All 114 interface records, including the affine fields of the 62 quantized parameters, were independently cross-checked against the ai-edge-litert 2.1.4 Interpreter metadata API with zero mismatches. The corpus, artifact identities, and verification scripts are available in the public evidence repository.
These figures are exact measurements of the defined corpus and are not intended as an estimate of ecosystem prevalence. The broader point is structural: LiteRT and ONNX both associate affine quantization parameters with a particular tensor. An aggregate model-level property set therefore cannot preserve the mapping when multiple named parameters carry distinct affine values.
MobileNetV2 was simply a compact worked example in which the uint8 input and output used different scale and zero-point values.
Given a declared preprocessing or runtime harness contract, a consumer can statically compare it with the affine mapping recorded for the named model parameter and flag an affine-contract mismatch at the deployment boundary without running inference.
From a medical-device AI perspective, this is a narrow but concrete example of how parameter-bound metadata could support deployment-assurance and threat-modeling workflows for a hash-identified model artifact used within a separately documented validated deployment configuration. This check detects inconsistency between declared affine mappings; it does not by itself establish that the deployed harness implements the declared contract or validate the full preprocessing pipeline.
Could the properties in the external-interface example therefore be placed in the properties array of the relevant named modelParameters.inputs[] or modelParameters.outputs[] entry? That would preserve the parameter binding while keeping format.dataType as the declared integer representation.
The required parameter-level extension point is already present in the current JSON draft in CycloneDX/specification#990. For the JSON representation, this placement therefore appears achievable through taxonomy wording and worked-example updates without introducing a new schema field.
For example, the relevant named-parameter fragment for the per-tensor case could be expressed along these lines:
{
"modelParameters": {
"inputs": [
{
"name": "image",
"modality": "image",
"format": {
"dataType": "uint8",
"encoding": "raw"
},
"properties": [
{
"name": "cdx:ai-ml:model:parameter:quantization:scheme",
"value": "affine_asymmetric"
},
{
"name": "cdx:ai-ml:model:parameter:quantization:granularity",
"value": "per-tensor"
},
{
"name": "cdx:ai-ml:model:parameter:quantization:scale",
"value": "0.0078125"
},
{
"name": "cdx:ai-ml:model:parameter:quantization:zeroPoint",
"value": "128"
}
],
"shape": [1, 224, 224, 3]
}
]
}
}Localized wording and example points
-
The per-tensor example is described as TFLite-style INT8 but uses a
zeroPointof128. That value is outside the signed int8 range. The example should either declaredataType: "uint8"and describe the representation as UINT8, as above, or use a zero point representable by signed int8. -
The
schemedescription should say that it "MUST be present whenever any other quantization sub-property is used," rather than that it "MUST be set before." Consumers should not depend on the order of entries in the properties array. The description should also refer to the scheme applied to the named parameter or tensor rather than to the model as a whole. -
For per-axis quantization, "ordered by axis index" is misleading. The
axisvalue selects one tensor dimension, and elementiof the scale or zero-point vector applies to sliceialong that dimension. When that dimension is static, the vector cardinality should equalshape[axis]. This is consistent with the per-axis semantics specified by LiteRT and ONNX QuantizeLinear. The per-axis worked example already describes this correctly as "one element per slice along the quantized axis," so the property-table wording could be aligned with that explanation. -
The per-axis example refers to an internal weight tensor but does not identify that tensor. Since internal tensor enumeration was not part of the original use case, deferring this example appears to be the safer option unless a real schema-addressable external per-axis parameter is available for the worked example.
-
The
affine_symmetricdescription could be narrowed. For the current signed INT8 weight example, a zero point of0is appropriate. The LiteRT int8 specification represents weights over[-127, 127]with zero point0, which removes one integer code rather than half of the range. For a framework-neutral taxonomy, the general scheme description should either qualify the zero-point rule bydataTypeor state explicitly that it is limited to zero-centered signed storage. PyTorch MinMaxObserver, for example, defines the symmetric zero point as0forqint8and128forquint8. -
If
affine_asymmetricis intended to describe affine quantization without a symmetry constraint, it should not require a non-zerozeroPoint. The LiteRT int8 specification permits activation zero points anywhere in[-128, 127], which includes0. If the scheme is instead intended to classify representations according to the resulting zero-point value, that classification rule could be stated explicitly.
With the parameter binding clarified and these localized points addressed, the affine additions would align well with the use case I raised.
Given the August 20 timeline, the highest-value low-impact changes for 2.0 seem to be the parameter-bound per-tensor example and the localized table corrections that require no new JSON schema field. If helpful, I would be happy to prepare the exact Markdown patch or a focused PR for those items in whichever contribution form is most convenient. If the internal per-axis tensor example is better handled after the cutoff, deferring that part would be completely reasonable.
Thank you again for moving this forward so quickly.
|
A short addendum on placement: the affine sketch in CycloneDX/specification#948 already preserved named-parameter binding under My main point here is unchanged and does not depend on that choice: the affine properties should be bound to the relevant named parameter rather than placed at the aggregate |
JunHwan-Kwon
left a comment
There was a problem hiding this comment.
@mrutkows I added seven apply-ready, line-level corrections from my earlier affine review. They can be reviewed separately and batch-applied as appropriate.
Three of these suggestions concern vector-valued quantization metadata: because property.value is a string in the schema, the current wording could lead implementers to emit a native JSON array.
I left the named-parameter relocation separate pending confirmation of the intended #990 extension point.
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Co-authored-by: JunHwanKwon <kjh0442@yuhs.ac> Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
|
@mrutkows Thank you for applying all seven affine corrections. @mrutkows @stevespringett One placement decision remains before I prepare the named-parameter worked-example update. In #948, commit Which extension point should the #175 examples use?
Either option preserves named-parameter binding. I can align the examples with whichever location is intended for #990 and provide the update as an apply-ready suggestion. |
|
@stevespringett @mrutkows Following commit 49a9456, I see that the current #990 draft now addresses named-parameter placement directly through the typed Before I prepare the #175 worked-example update, one scope question remains for the external-interface affine-contract use case: whether the scale and zeroPoint values needed to distinguish named interface contracts should be represented as BOM metadata, while the underlying scale and zero-point tensors remain part of the model artifact. For the #175 worked example, would you prefer that I:
A short indication of either option, or another intended direction, is enough for me to prepare or defer the apply-ready follow-up. |
|
@mrutkows @stevespringett A short reproducibility update to the scope question above, in case it is useful. I tested the pinned draft combination ( Both the typed-only form and the combination of typed Under the current #175 wording, however, the numeric-property form still lacks the required taxonomy Reproducible evidence: Whichever direction is intended, I would be glad to align the localized wording and worked-example update accordingly. |
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
JunHwan-Kwon
left a comment
There was a problem hiding this comment.
@mrutkows @stevespringett Thank you both for your work on the current schema and taxonomy drafts. Based on the current drafts and the validation results already posted, I added four localized, apply-ready suggestions that may help align the affine taxonomy in #175 with the named input/output structure currently proposed in CycloneDX/specification#990.
Suggestions 1-3 address one bounded issue: how affine quantization metadata for named inputs and outputs should be divided between typed fields and taxonomy properties. My proposal is that, when present, the typed scheme, granularity, and axis fields are authoritative for that parameter, while sibling scale and zeroPoint properties may carry parameter-bound external affine mapping metadata. The worked example reflects this division by using a named uint8 input.
Suggestion 4 replaces the current unbound internal per-axis example with a concise statement of the information needed to interpret a per-axis mapping. No new schema field or broader model-level or runtime scope is proposed in this review.
These suggestions intentionally leave the predefined-vocabulary differences between the drafts unresolved. CycloneDX/specification#990 currently defines affine and symmetric, together with per-tensor, per-channel, and per-group, while #175 defines affine_asymmetric and affine_symmetric, together with per-tensor and per-axis. The worked example is deliberately limited to the shared per-tensor granularity and does not assume equivalence between the differing predefined terms. Once the canonical terminology is settled, I can align the #175 wording and examples accordingly, or document the intended relationship if both vocabularies are retained.
I would appreciate your views on whether this division is consistent with the intended direction of the two drafts. If Suggestions 1-3, or equivalent wording, are acceptable, that would resolve the bounded external-interface affine ownership and placement question from my side. Suggestion 4 can be considered independently.
Validation basis previously posted:
JunHwan-Kwon/tensor-quantization-metadata-study@0338f7f
| | `cdx:ai-ml:model:parameter:tune_method` | Describes how the model was fine-tuned on or adapted to new data. Value SHOULD be of industry-standard keywords such as those [listed in the section below](#names-of-industry-standard-fine-tuning-methods). Value MUST be a single keyword (e.g., `lora`) or a comma separated list of keywords (e.g., `sft,rlhf`). </br> This property MAY appear multiple times. | | ||
| | `cdx:ai-ml:model:parameter:_undefined:<NAME>` | `<NAME>` placeholder, used to provide an arbitrary model parameter name. Arbitrarty value and meaning. | | ||
| | `cdx:ai-ml:model:parameter:tune_method` | Describes how the model was fine-tuned on or adapted to new data. Values SHOULD be industry-standard keywords such as those [listed in the section below](#names-of-industry-standard-fine-tuning-methods). Value MUST be a single keyword (e.g., `lora`) or a comma separated list of keywords (e.g., `sft,rlhf`). </br> This property MAY appear multiple times. | | ||
| | `cdx:ai-ml:model:parameter:quantization:scheme` | Identifies the quantization scheme described by the accompanying quantization sub-properties. Value SHOULD be one of the [quantization scheme values listed below](#quantization-scheme-values). This property MUST be present whenever any other `quantization` sub-property is used. MAY appear once. | |
There was a problem hiding this comment.
| | `cdx:ai-ml:model:parameter:quantization:scheme` | Identifies the quantization scheme described by the accompanying quantization sub-properties. Value SHOULD be one of the [quantization scheme values listed below](#quantization-scheme-values). This property MUST be present whenever any other `quantization` sub-property is used. MAY appear once. | | |
| | `cdx:ai-ml:model:parameter:quantization:scheme` | Identifies the quantization scheme described by the accompanying quantization sub-properties. Value SHOULD be one of the [quantization scheme values listed below](#quantization-scheme-values). This property MUST be present whenever another `quantization` sub-property is used, except when those sub-properties annotate a named CycloneDX 2.0 model input or output and a typed `quantization.scheme` is present. In that context, the typed field is authoritative and this property SHOULD be omitted. This property MAY appear once. | |
|
|
||
| #### Quantization scheme values | ||
|
|
||
| The `cdx:ai-ml:model:parameter:quantization:scheme` property identifies the mathematical contract between floating-point values and their quantized integer representation. A scheme determines which companion sub-properties (`scale`, `zeroPoint`, `granularity`, `axis`) are meaningful. Future schemes MAY introduce additional sub-properties under the same `cdx:ai-ml:model:parameter:quantization:` namespace without conflicting with existing entries. |
There was a problem hiding this comment.
| The `cdx:ai-ml:model:parameter:quantization:scheme` property identifies the mathematical contract between floating-point values and their quantized integer representation. A scheme determines which companion sub-properties (`scale`, `zeroPoint`, `granularity`, `axis`) are meaningful. Future schemes MAY introduce additional sub-properties under the same `cdx:ai-ml:model:parameter:quantization:` namespace without conflicting with existing entries. | |
| The `cdx:ai-ml:model:parameter:quantization:scheme` property identifies the mathematical contract between floating-point values and their quantized integer representation. A scheme determines which companion sub-properties (`scale`, `zeroPoint`, `granularity`, `axis`) are meaningful. Future schemes MAY introduce additional sub-properties under the same `cdx:ai-ml:model:parameter:quantization:` namespace without conflicting with existing entries. | |
| When these affine quantization sub-properties annotate a named CycloneDX 2.0 model input or output, the typed `scheme`, `granularity`, and `axis` fields, when present, are authoritative for that parameter. Producers SHOULD NOT additionally provide `cdx:ai-ml:model:parameter:quantization:scheme` when typed `scheme` is present, `cdx:ai-ml:model:parameter:quantization:granularity` when typed `granularity` is present, or `cdx:ai-ml:model:parameter:quantization:axis` when typed `axis` is present. The sibling `scale` and `zeroPoint` properties MAY supplement the typed object with affine mapping values for that named input or output. They record parameter-bound external affine mapping metadata and do not supersede any corresponding serialized mapping data associated with the model artifact. |
| #### Example: Using quantization parameters (per-tensor, affine asymmetric) | ||
|
|
||
| The following pseudocode shows how to describe per-tensor affine asymmetric quantization parameters in a model card: | ||
|
|
||
| ```jsonc | ||
| { | ||
| // ... | ||
| "components": [{ | ||
| "type": "machine-learning-model", | ||
| "name": "my quantized model", | ||
| "modelCard": { | ||
| // ... | ||
| "modelParameters": { | ||
| "properties": [ | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:scheme", | ||
| "value": "affine_asymmetric" | ||
| }, | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:granularity", | ||
| "value": "per-tensor" | ||
| }, | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:scale", | ||
| "value": "0.0078125" | ||
| }, | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:zeroPoint", | ||
| "value": "128" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| }] | ||
| } | ||
| ``` |
There was a problem hiding this comment.
| #### Example: Using quantization parameters (per-tensor, affine asymmetric) | |
| The following pseudocode shows how to describe per-tensor affine asymmetric quantization parameters in a model card: | |
| ```jsonc | |
| { | |
| // ... | |
| "components": [{ | |
| "type": "machine-learning-model", | |
| "name": "my quantized model", | |
| "modelCard": { | |
| // ... | |
| "modelParameters": { | |
| "properties": [ | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:scheme", | |
| "value": "affine_asymmetric" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:granularity", | |
| "value": "per-tensor" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:scale", | |
| "value": "0.0078125" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:zeroPoint", | |
| "value": "128" | |
| } | |
| ] | |
| } | |
| } | |
| }] | |
| } | |
| ``` | |
| #### Example: Using quantization parameters for a named input (per-tensor affine) | |
| The following pseudocode records parameter-bound external affine mapping metadata for a named `uint8` input. The typed `quantization` object provides `bits`, `scheme`, and `granularity`, while sibling properties record the scalar `scale` and `zeroPoint` values for that input. | |
| ```jsonc | |
| { | |
| // ... | |
| "components": [{ | |
| "type": "machine-learning-model", | |
| "name": "my quantized model", | |
| "modelProperties": { | |
| "inputs": [{ | |
| "name": "image", | |
| "modality": "image", | |
| "dataType": "uint8", | |
| "shape": [1, 224, 224, 3], | |
| "quantization": { | |
| "bits": 8, | |
| "scheme": "affine", | |
| "granularity": "per-tensor" | |
| }, | |
| "properties": [ | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:scale", | |
| "value": "0.0078125" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:zeroPoint", | |
| "value": "128" | |
| } | |
| ] | |
| }] | |
| } | |
| }] | |
| } | |
| ``` |
There was a problem hiding this comment.
For clarity, this example follows the current CycloneDX/specification#990 head (49a9456) for the enclosing object and path. It is not intended to take a position on the separate modelCard / modelProperties grouping discussion. The named-parameter binding and the division between typed fields and sibling properties are independent of that decision, and I can align the example with the final schema structure once it is settled.
| #### Example: Using quantization parameters (per-axis, affine symmetric) | ||
|
|
||
| The following pseudocode shows how to describe a PyTorch-style per-axis INT8 affine symmetric quantization of a weight tensor along axis `0`. In this signed INT8 example, the symmetric convention uses `zeroPoint = 0`, which the example omits. The `scale` property value is a string containing a JSON-serialized numeric array with one element per slice along the quantized axis. | ||
|
|
||
| ```jsonc | ||
| { | ||
| // ... | ||
| "components": [{ | ||
| "type": "machine-learning-model", | ||
| "name": "my symmetric per-axis quantized model", | ||
| "modelCard": { | ||
| // ... | ||
| "modelParameters": { | ||
| "properties": [ | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:scheme", | ||
| "value": "affine_symmetric" | ||
| }, | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:granularity", | ||
| "value": "per-axis" | ||
| }, | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:axis", | ||
| "value": "0" | ||
| }, | ||
| { | ||
| "name": "cdx:ai-ml:model:parameter:quantization:scale", | ||
| "value": "[0.00787402, 0.01574803, 0.00393701]" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| }] | ||
| } | ||
| ``` | ||
|
|
There was a problem hiding this comment.
| #### Example: Using quantization parameters (per-axis, affine symmetric) | |
| The following pseudocode shows how to describe a PyTorch-style per-axis INT8 affine symmetric quantization of a weight tensor along axis `0`. In this signed INT8 example, the symmetric convention uses `zeroPoint = 0`, which the example omits. The `scale` property value is a string containing a JSON-serialized numeric array with one element per slice along the quantized axis. | |
| ```jsonc | |
| { | |
| // ... | |
| "components": [{ | |
| "type": "machine-learning-model", | |
| "name": "my symmetric per-axis quantized model", | |
| "modelCard": { | |
| // ... | |
| "modelParameters": { | |
| "properties": [ | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:scheme", | |
| "value": "affine_symmetric" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:granularity", | |
| "value": "per-axis" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:axis", | |
| "value": "0" | |
| }, | |
| { | |
| "name": "cdx:ai-ml:model:parameter:quantization:scale", | |
| "value": "[0.00787402, 0.01574803, 0.00393701]" | |
| } | |
| ] | |
| } | |
| } | |
| }] | |
| } | |
| ``` | |
| #### Per-axis affine quantization | |
| Interpretation of a per-axis mapping requires an unambiguously identified tensor and sufficient shape information to interpret the selected axis and verify the cardinality of the scale vector and, when present, the zero-point vector. |
See proposed CycloneDX v2.0 AI/ML JSON schema (proposed): CycloneDX/specification#948