Restructure and simplify the AWQModifier to be similar to SmoothQuant

[dsikka]

Summary

Similar to SmoothQuant, AWQ determines and applies smoothing scales. This process is facilitated by a set of model specific mappings which indicate the activation layers to smooth. Once smoothing is complete, the modifier applies redundant code that is already present within the QuantizationModifier to determine optimal quantization scales and zero_points

The scope of this task is to simplify this modifier by:

1. Removing all the code responsible for generating the quantization scales and zero-points. AWQ, like SmoothQuant, should have the ability to be used in a stackable manner with other quantization modifiers such as the GPTQModifier and QuantizationModifier.

Examples 1:

recipe = [
    AWQModifier(...),
    QuantizationModifier(...)
]

Examples 2:

recipe = [
    AWQModifier(...),
    GPTQModifier(...)
]

Example 3:

recipe = [
    AWQModifier(...),
    QuIPModifier(
        rotations=["v", "u"], transform_block_size=128, transform_type="hadamard"
    ),
    QuantizationModifier(targets="Linear", scheme="W4A16", ignore=["lm_head"]),
]

2. Simplify the modifier. This can be done by targeting key functionality that is quite complex, such as _compute_best_scale

Implementation Steps

1. Remove quantization scale and zero-point generation code and ensure AWQ can be used in a stackable manner with other quantization modifiers. Validate this through additional test scripts.
2. Make other simplifications within the modiifer to improve the complexity of the modifier
3. Update all examples and tests using the AWQModifier to now use AWQModifier stacked with the QuantizationModifier. Validate performance and accuracy remains the same
4. As the AWQModifier will still require quantization arguments to generate the means, potentially add a recipe validation step to validate that these arguments match the quantization arguments passed to the subsequent quantization modifier, such as GPTQModifier and QuantizationModifier

[colldata79]

Hello @dsikka can i take a look at this ?

[colldata79]

Please before I jump into this i have a few questions.

1. After refactor, should AWQ still be usable standalone (i.e., produces a quantized model), or will it become smoothing-only and require stacking with QuantizationModifier/GPTQ?

2. When you say ‘similar to SmoothQuant’, do you mean only that AWQ applies smoothing in-place and does no quantization, or do you also want AWQ to adopt SmoothQuant’s lifecycle / mapping & cached-state structure?

3. Should AWQ still inherit QuantizationMixin after removing quantization, or should it become a plain Modifier like SmoothQuant? And where should ‘quant args must match the next modifier’ validation happen (AWQ vs recipe/stacking layer), and which fields are considered authoritative?

@dsikka

[dsikka]

Hi @colldata79 - sorry for the late reply

1. We should throw a warning if AWQ is used as a stand-alone to let users know that quantization is not going to be enabled through the modifier
2. I believe AWQ already supports a mapping / cached structure to apply smoothing but yes the goal is to apply smoothing in-place and does no quantization
3. The quant args validation will need to happen in the recipe parsing / validation level, prior to any of the modifiers running.

   llm-compressor/src/llmcompressor/recipe/recipe.py

   Line 26 in 6d600d4

   class Recipe(BaseModel):

   . I believe we should have no problems removing the QuantizationMixin if all quantization functionality is being removed