Update: Test for Compatibility with Transformers 4.48

[rahul-tuli]

We recently encountered a failure due to the release of Transformers 4.48. Details of the failure can be found here: GitHub Actions Failure.

Cause

The failure occurred because the Llama model definition in Transformers was updated to use a single rotary embedding for the model, replacing the previous implementation where there was one rotary embedding per attention block. This change was introduced in the following commit: Transformers Commit (line 261).

Fix and Validation

Instead of relying o hardcoded values, we count the modules in question now, and check if after quantization config application the number stays the same. Credits to @horheynm for the better fix #238. The test now successfully passes with all versions of Transformers, as demonstrated below:

Transformers 4.47.1

(.venv) ➜ compressed-tensors git:(update-test) ✗ pytest "./tests/test_quantization/lifecycle/test_apply.py::test_apply_quantization_config_tinyllama"
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.10.12, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/rahul/compressed-tensors
configfile: pyproject.toml
collected 1 item                                                                                                                                                                                          

tests/test_quantization/lifecycle/test_apply.py .                                                                                                                                                   [100%]

============================================================================================ 1 passed in 3.68s ============================================================================================

Transformers 4.48.0

(.venv) ➜ compressed-tensors git:(update-test) ✗ pytest "./tests/test_quantization/lifecycle/test_apply.py::test_apply_quantization_config_tinyllama"
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.10.12, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/rahul/compressed-tensors
configfile: pyproject.toml
collected 1 item                                                                                                                                                                                          

tests/test_quantization/lifecycle/test_apply.py .                                                                                                                                                   [100%]

============================================================================================ 1 passed in 4.08s ============================================================================================

Summary

This update ensures compatibility with the latest Transformers release (4.48) while maintaining support for previous versions. All relevant tests now pass, confirming the fix.

[kylesayrs]

Much better