rfe: Deprecate distutils module

[YvanY0]

distutils will be formally marked as deprecated since python3.10, and will no longer work from python3.12. To enable python3.12 in avocado framework, this patch will replace distutils to use other modules.

ID: 2433
ref: https://peps.python.org/pep-0632

[YvanY0]

https://github.com/avocado-framework/avocado-vt/blob/master/virttest/shared/scripts/virtio_console_guest.py
this file is used in guest, even though it also uses distutils module, but I didn't modify it together, cause I am not sure about the guest python version.
cc @nanliu-r

[YvanY0]

Hello @clebergnu @luckyh, please help to review and raise your concerns.

Another one is I noticed setup.py would also be deprecated, do we plan to migrate to the modern pyproject.toml? https://packaging.python.org/en/latest/guides/modernize-setup-py-project/#modernize-setup-py-project

Besides pyproject.toml, I tried using pip install -e . to install avocado-vt, and this works, and with pip install but not python setup.py install/develop, the CleanCommand in setup.py is not a must.

[luckyh]

Hi @PaulYuuu , thanks for raising this up! I think the overall idea is good, however, I also have the following concerns.

[pevogam]

Hi @PaulYuuu, have you considered moving the setup.py entirely to a pyproject.toml file as described in PEP 621? Perhaps this could make things simpler? Other than that what is the current status of this pull request? I guess it was converted to a draft since it is not considered to be the way forward or has a big roadblock ahead of it?

[YvanY0]

Hi @PaulYuuu, have you considered moving the setup.py entirely to a pyproject.toml file as described in PEP 621? Perhaps this could make things simpler? Other than that what is the current status of this pull request? I guess it was converted to a draft since it is not considered to be the way forward or has a big roadblock ahead of it?

Good question! The answer is yes, please refer to #3924

However, avocado still uses setup.py, so for avocado-vt, I am not sure if avocado-vt can go one step ahead. And this is a big milestone; it may break something, and it needs all maintainers to identify the risk.

[pevogam]

Hmmm I see, the difference is that Avocado already added Python 3.12 and 3.13 support while the setup procedures in VT are still outdated. So I assume we are not sure if it is worth investing the time to merge this and deal with any breakage until Avocado makes the pyproject.toml step which could save us effort?

[YvanY0]

Hmmm I see, the difference is that Avocado already added Python 3.12 and 3.13 support while the setup procedures in VT are still outdated. So I assume we are not sure if it is worth investing the time to merge this and deal with any breakage until Avocado makes the pyproject.toml step which could save us effort?

I have no idea. Let me cc other maintainers here for the discussion. @luckyh @richtja @clebergnu @YongxueHong @chunfuwen

[richtja]

Hi @PaulYuuu and @pevogam, in avocado we plan to move from distils avocado-framework/avocado#5159 and migrate to pyproject.toml avocado-framework/avocado#5754. Unfortunately, we don't have anyone who would be working on that, so we hare postponing it. For now, we have solved this by adding settuptools as avocado dependency to support python3.12 and 3.13 avocado-framework/avocado#5789

[YvanY0]

The PR is ready for review. Let me clarify some concerns regarding the above comments.

1. class CleanCommand(Command) https://github.com/avocado-framework/avocado-vt/pull/3920/files#r1703862017: I don't define the --all user option for the clean sub-command, we have cleaning_list, the dir to be cleaned of the native --all option will be covered in the cleaning_list. And, Python does not suggest running setup.py directly, we may move setup.py install to pip install, setup.py clean to make clean in the Makefile in the future. ref: https://packaging.python.org/en/latest/discussions/setup-py-deprecated/
2. dir_util.copy_tree can be safely replaced with shutil.copytree right now, avocado-vt currently only supports python>=3.8
3. LooseVersion cannot handle post version, but packaging.version.parse can. ref: https://peps.python.org/pep-0440/

>>> from packaging.version import parse
>>> from distutils.version import LooseVersion
>>> version_packaging = parse("1.0.0-1")
>>> version_distutils = LooseVersion("1.0.0-1")
<stdin>:1: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
>>> version_packaging
<Version('1.0.0.post1')>
>>> version_distutils
LooseVersion ('1.0.0-1')
>>> version_packaging.base_version
'1.0.0'
>>> version_distutils.version
[1, 0, 0, '-', 1]

[YvanY0]

Hello @richtja, I have a question: Will we still maintain the Cirrus CI? I saw that most of the quay images are quite old, like Fedora-35 and Centos-8.1, which causes some CI jobs to fail.

[richtja]

Hello @richtja, I have a question: Will we still maintain the Cirrus CI? I saw that most of the quay images are quite old, like Fedora-35 and Centos-8.1, which causes some CI jobs to fail.

Hi @PaulYuuu AFAIK we removed Cirrus CI from avocado CI in avocado-framework/avocado#5239, but I don't know why it has been kept in avocado-vt, maybe @clebergnu will have more details. IMO there is no need to keep Cirrus CI, and we could migrate to GitHub Actions.

About the quay images I know that on avocado we maintain and use only the check-copr-rpm-version image.

[YvanY0]

Hey guys, is there any update here? If no, I can start by removing the centos_8_1 task from Cirrus CI since CentOS8 has been EOL for a long time.

[richtja]

Hi @PaulYuuu, thank you, I think we can learn from this and apply it to avocado as well. LGTM

Out-of-date

[YvanY0]

Hi @pevogam,
Would you be able to apply this PR and bootstrap it in your environment to verify if it introduces any regressions in your tests? From my side, I don’t see any risks, but since you’re using Python 3.13, I want to make sure it won’t interfere with your setup.
Let me know if you encounter any issues!

And after this gets merged, your PR #4101 may be can rebase and then continue to review.

[pevogam]

Hi @pevogam, Would you be able to apply this PR and bootstrap it in your environment to verify if it introduces any regressions in your tests? From my side, I don’t see any risks, but since you’re using Python 3.13, I want to make sure it won’t interfere with your setup. Let me know if you encounter any issues!

And after this gets merged, your PR #4101 may be can rebase and then continue to review.

Sorry I took a while, will do this tomorrow right away

[pevogam]

Ok, I see no problem with these changes ✔️

[YvanY0]

Thank you all, I will merge this PR soon, let's see later if it will introduce any negative impact. If so, I will follow up on it.