| Message ID | 20240729122553.2501133-1-peter.maydell@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC] pythondeps: Split sphinx_rtd_theme into its own group | expand |
On Mon, Jul 29, 2024 at 2:26 PM Peter Maydell <peter.maydell@linaro.org> wrote: > > Currently we have "sphinx" and "sphinx_rtd_theme" in a single > group in pythondeps.toml. This means that mkvenv will try to > install them both at once, which doesn't work if sphinx_rtd_theme > depends on a version of Sphinx which is newer than the system > one, even if the "sphinx" we're installing at the same time > would satisfy that dependency. For instance this: > > sphinx = { accepted = ">=8.0.0rc1", installed = "8.0.0rc1", canary = "sphinx-build" } > sphinx_rtd_theme = { accepted = ">=2.1.0rc1", installed = "2.1.0rc1" } > > fails like this: > > mkvenv: Creating non-isolated virtual environment at 'pyvenv' > mkvenv: checking for meson>=1.1.0 > mkvenv: installing meson==1.2.3 > mkvenv: checking for sphinx>=8.0.0rc1 > mkvenv: checking for sphinx_rtd_theme>=2.1.0rc1 > mkvenv: installing sphinx==8.0.0rc1, sphinx_rtd_theme==2.1.0rc1 > ERROR: Cannot install sphinx-rtd-theme==2.1.0rc1 and sphinx==8.0.0rc1 because these package versions have conflicting dependencies. > ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts > > *** Ouch! *** > > Could not provide build dependency 'sphinx==8.0.0rc1': > • Python package 'sphinx' version '4.3.2' was found, but isn't suitable. > • A suitable version could not be obtained from PyPI. > > (where 4.3.2 is the system version of sphinx on my system). I think the error is a bit confusing, but the versions seem to be genuinely incompatibile. sphinx_rtd_theme is still incompatible with sphinx 8.0.0, and the error is correct: ========= $ python -m venv testvenv $ cd testvenv $ pip install 'sphinx==8.0.0rc1' 'sphinx_rtd_theme==2.1.0rc1' Defaulting to user installation because normal site-packages is not writeable Collecting sphinx==8.0.0rc1 Downloading sphinx-8.0.0rc1-py3-none-any.whl.metadata (6.2 kB) Collecting sphinx_rtd_theme==2.1.0rc1 Downloading sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl.metadata (4.4 kB) Collecting sphinxcontrib-applehelp (from sphinx==8.0.0rc1) Downloading sphinxcontrib_applehelp-2.0.0-py3-none-any.whl.metadata (2.3 kB) Requirement already satisfied: sphinxcontrib-devhelp in /usr/lib/python3.12/site-packages (from sphinx==8.0.0rc1) (1.0.5) [more "requirement already satisfied" lines...] Requirement already satisfied: packaging>=23.0 in /usr/lib/python3.12/site-packages (from sphinx==8.0.0rc1) (23.2) INFO: pip is looking at multiple versions of sphinx-rtd-theme to determine which version is compatible with other requirements. This could take a while. ERROR: Cannot install sphinx-rtd-theme==2.1.0rc1 and sphinx==8.0.0rc1 because these package versions have conflicting dependencies. The conflict is caused by: The user requested sphinx==8.0.0rc1 sphinx-rtd-theme 2.1.0rc1 depends on sphinx<8 and >=5 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts ========= The reason why your patch works is weird, and it's apparent when you run the "pip install" commands one by one: ========= $ pip install 'sphinx_rtd_theme==2.1.0rc1' Defaulting to user installation because normal site-packages is not writeable Collecting sphinx_rtd_theme==2.1.0rc1 Using cached sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl.metadata (4.4 kB) Collecting sphinx<8,>=5 (from sphinx_rtd_theme==2.1.0rc1) Downloading sphinx-7.4.7-py3-none-any.whl.metadata (6.1 kB) Requirement already satisfied: docutils<0.21 in /usr/lib/python3.12/site-packages (from sphinx_rtd_theme==2.1.0rc1) (0.20.1) [more "requirement already satisfied" lines...] Downloading sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl (7.7 MB) Downloading sphinx-7.4.7-py3-none-any.whl (3.4 MB) Installing collected packages: sphinx, sphinx_rtd_theme Attempting uninstall: sphinx Found existing installation: Sphinx 8.0.0rc1 Uninstalling Sphinx-8.0.0rc1: Successfully uninstalled Sphinx-8.0.0rc1 Successfully installed sphinx-7.4.7 sphinx_rtd_theme-2.1.0rc1 ========= It doesn't install 8.0.0rc1 at all... :) Paolo
On Tue, 30 Jul 2024 at 15:50, Paolo Bonzini <pbonzini@redhat.com> wrote: > > On Mon, Jul 29, 2024 at 2:26 PM Peter Maydell <peter.maydell@linaro.org> wrote: > > > > Currently we have "sphinx" and "sphinx_rtd_theme" in a single > > group in pythondeps.toml. This means that mkvenv will try to > > install them both at once, which doesn't work if sphinx_rtd_theme > > depends on a version of Sphinx which is newer than the system > > one, even if the "sphinx" we're installing at the same time > > would satisfy that dependency. For instance this: > > > > sphinx = { accepted = ">=8.0.0rc1", installed = "8.0.0rc1", canary = "sphinx-build" } > > sphinx_rtd_theme = { accepted = ">=2.1.0rc1", installed = "2.1.0rc1" } > > > > fails like this: > > > > mkvenv: Creating non-isolated virtual environment at 'pyvenv' > > mkvenv: checking for meson>=1.1.0 > > mkvenv: installing meson==1.2.3 > > mkvenv: checking for sphinx>=8.0.0rc1 > > mkvenv: checking for sphinx_rtd_theme>=2.1.0rc1 > > mkvenv: installing sphinx==8.0.0rc1, sphinx_rtd_theme==2.1.0rc1 > > ERROR: Cannot install sphinx-rtd-theme==2.1.0rc1 and sphinx==8.0.0rc1 because these package versions have conflicting dependencies. > > ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts > > > > *** Ouch! *** > > > > Could not provide build dependency 'sphinx==8.0.0rc1': > > • Python package 'sphinx' version '4.3.2' was found, but isn't suitable. > > • A suitable version could not be obtained from PyPI. > > > > (where 4.3.2 is the system version of sphinx on my system). > > I think the error is a bit confusing, but the versions seem to be > genuinely incompatibile. sphinx_rtd_theme is still incompatible with > sphinx 8.0.0, and the error is correct: > > ========= > $ python -m venv testvenv > $ cd testvenv > $ pip install 'sphinx==8.0.0rc1' 'sphinx_rtd_theme==2.1.0rc1' > Defaulting to user installation because normal site-packages is not writeable > Collecting sphinx==8.0.0rc1 > Downloading sphinx-8.0.0rc1-py3-none-any.whl.metadata (6.2 kB) > Collecting sphinx_rtd_theme==2.1.0rc1 > Downloading sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl.metadata (4.4 kB) > Collecting sphinxcontrib-applehelp (from sphinx==8.0.0rc1) > Downloading sphinxcontrib_applehelp-2.0.0-py3-none-any.whl.metadata (2.3 kB) > Requirement already satisfied: sphinxcontrib-devhelp in > /usr/lib/python3.12/site-packages (from sphinx==8.0.0rc1) (1.0.5) > [more "requirement already satisfied" lines...] > Requirement already satisfied: packaging>=23.0 in > /usr/lib/python3.12/site-packages (from sphinx==8.0.0rc1) (23.2) > INFO: pip is looking at multiple versions of sphinx-rtd-theme to > determine which version is compatible with other requirements. This > could take a while. > ERROR: Cannot install sphinx-rtd-theme==2.1.0rc1 and sphinx==8.0.0rc1 > because these package versions have conflicting dependencies. > > The conflict is caused by: > The user requested sphinx==8.0.0rc1 > sphinx-rtd-theme 2.1.0rc1 depends on sphinx<8 and >=5 > To fix this you could try to: > 1. loosen the range of package versions you've specified > 2. remove package versions to allow pip attempt to solve the dependency conflict > > ERROR: ResolutionImpossible: for help visit > https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts > ========= > > The reason why your patch works is weird, and it's apparent when you > run the "pip install" commands one by one: > > ========= > $ pip install 'sphinx_rtd_theme==2.1.0rc1' > Defaulting to user installation because normal site-packages is not writeable > Collecting sphinx_rtd_theme==2.1.0rc1 > Using cached sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl.metadata (4.4 kB) > Collecting sphinx<8,>=5 (from sphinx_rtd_theme==2.1.0rc1) > Downloading sphinx-7.4.7-py3-none-any.whl.metadata (6.1 kB) > Requirement already satisfied: docutils<0.21 in > /usr/lib/python3.12/site-packages (from sphinx_rtd_theme==2.1.0rc1) > (0.20.1) > [more "requirement already satisfied" lines...] > Downloading sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl (7.7 MB) > Downloading sphinx-7.4.7-py3-none-any.whl (3.4 MB) > Installing collected packages: sphinx, sphinx_rtd_theme > Attempting uninstall: sphinx > Found existing installation: Sphinx 8.0.0rc1 > Uninstalling Sphinx-8.0.0rc1: > Successfully uninstalled Sphinx-8.0.0rc1 > Successfully installed sphinx-7.4.7 sphinx_rtd_theme-2.1.0rc1 > ========= > > It doesn't install 8.0.0rc1 at all... :) Aha. Is it possible to get mkvenv to produce these full error messages rather than the truncated version? thanks -- PMM
> > The reason why your patch works is weird, and it's apparent when you > > run the "pip install" commands one by one: > > > > ========= > > $ pip install 'sphinx_rtd_theme==2.1.0rc1' > > Defaulting to user installation because normal site-packages is not writeable > > Collecting sphinx_rtd_theme==2.1.0rc1 > > Using cached sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl.metadata (4.4 kB) > > Collecting sphinx<8,>=5 (from sphinx_rtd_theme==2.1.0rc1) > > Downloading sphinx-7.4.7-py3-none-any.whl.metadata (6.1 kB) > > Requirement already satisfied: docutils<0.21 in > > /usr/lib/python3.12/site-packages (from sphinx_rtd_theme==2.1.0rc1) > > (0.20.1) > > [more "requirement already satisfied" lines...] > > Downloading sphinx_rtd_theme-2.1.0rc1-py2.py3-none-any.whl (7.7 MB) > > Downloading sphinx-7.4.7-py3-none-any.whl (3.4 MB) > > Installing collected packages: sphinx, sphinx_rtd_theme > > Attempting uninstall: sphinx > > Found existing installation: Sphinx 8.0.0rc1 > > Uninstalling Sphinx-8.0.0rc1: > > Successfully uninstalled Sphinx-8.0.0rc1 > > Successfully installed sphinx-7.4.7 sphinx_rtd_theme-2.1.0rc1 > > ========= > > > > It doesn't install 8.0.0rc1 at all... :) > > Aha. Is it possible to get mkvenv to produce these full > error messages rather than the truncated version? Heh, the idea of pythondeps.toml was that they wouldn't be necessary - hence the messages that are slightly more tailored to the QEMU-specific issue. It's certainly a good idea to at least place them in a venv-pip.log file or something like that. Paolo
diff --git a/configure b/configure index 019fcbd0ef7..7f9cfbffa2c 100755 --- a/configure +++ b/configure @@ -973,9 +973,12 @@ if test "$download" = "enabled" ; then fi if test "$docs" != "disabled" ; then - if ! $mkvenv ensuregroup \ + if ! ( $mkvenv ensuregroup \ $(test "$docs" = "enabled" && echo "$mkvenv_online_flag") \ - ${source_path}/pythondeps.toml docs; + ${source_path}/pythondeps.toml docs && \ + $mkvenv ensuregroup \ + $(test "$docs" = "enabled" && echo "$mkvenv_online_flag") \ + ${source_path}/pythondeps.toml docs-theme ); then if test "$docs" = "enabled" ; then exit 1 diff --git a/pythondeps.toml b/pythondeps.toml index f6e590fdd86..fdf5579c43e 100644 --- a/pythondeps.toml +++ b/pythondeps.toml @@ -24,6 +24,9 @@ meson = { accepted = ">=1.1.0", installed = "1.2.3", canary = "meson" } [docs] # Please keep the installed versions in sync with docs/requirements.txt sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" } + +[docs-theme] +# Please keep the installed versions in sync with docs/requirements.txt sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } [avocado]
Currently we have "sphinx" and "sphinx_rtd_theme" in a single group in pythondeps.toml. This means that mkvenv will try to install them both at once, which doesn't work if sphinx_rtd_theme depends on a version of Sphinx which is newer than the system one, even if the "sphinx" we're installing at the same time would satisfy that dependency. For instance this: sphinx = { accepted = ">=8.0.0rc1", installed = "8.0.0rc1", canary = "sphinx-build" } sphinx_rtd_theme = { accepted = ">=2.1.0rc1", installed = "2.1.0rc1" } fails like this: mkvenv: Creating non-isolated virtual environment at 'pyvenv' mkvenv: checking for meson>=1.1.0 mkvenv: installing meson==1.2.3 mkvenv: checking for sphinx>=8.0.0rc1 mkvenv: checking for sphinx_rtd_theme>=2.1.0rc1 mkvenv: installing sphinx==8.0.0rc1, sphinx_rtd_theme==2.1.0rc1 ERROR: Cannot install sphinx-rtd-theme==2.1.0rc1 and sphinx==8.0.0rc1 because these package versions have conflicting dependencies. ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts *** Ouch! *** Could not provide build dependency 'sphinx==8.0.0rc1': • Python package 'sphinx' version '4.3.2' was found, but isn't suitable. • A suitable version could not be obtained from PyPI. (where 4.3.2 is the system version of sphinx on my system). Avoid this problem by splitting sphinx_rtd_theme into its own group in pythondeps, so we can install the new sphinx into the venv first and the sphinx_rtd_theme afterwards. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- I'm just bumbling around in the dark here, hence the RFC tag: maybe there's a better way to fix this? I ran into this when I was trying to test the fix for https://gitlab.com/qemu-project/qemu/-/issues/2458 and wanted a newer Sphinx to test with. Unless you also install the newer RTD theme the new Sphinx will emit some deprecation warnings about things the theme is doing. --- configure | 7 +++++-- pythondeps.toml | 3 +++ 2 files changed, 8 insertions(+), 2 deletions(-)