| Message ID | 20240624143352.370620-1-amusil@redhat.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [ovs-dev,v2,1/2] ci: Move common build steps into script. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
| ovsrobot/github-robot-_ovn-kubernetes | fail | github build: failed |
On 6/24/24 16:33, Ales Musil wrote: > Move common preparation steps into script that can be invoked by both > container builds. This will ensure that any update will be reflected > in both containers, and it reduces the duplication between both > containers. At the same time use the --user argument which avoids > the error below and allows pip to upgrade itself: > > ERROR: Cannot uninstall pip 24.0, RECORD file not found. > Hint: The package was installed by debian. > > Signed-off-by: Ales Musil <amusil@redhat.com> > Acked-by: Eelco Chaudron <echaudro@redhat.com> > --- > v2: Use --user for pip update. > Add ack from Eelco. > --- > utilities/automake.mk | 1 + > utilities/containers/fedora/Dockerfile | 35 ++++-------------------- > utilities/containers/prepare.sh | 37 ++++++++++++++++++++++++++ > utilities/containers/ubuntu/Dockerfile | 37 +++++--------------------- > 4 files changed, 49 insertions(+), 61 deletions(-) > create mode 100755 utilities/containers/prepare.sh > > diff --git a/utilities/automake.mk b/utilities/automake.mk > index de4f6efb5..03e9096fa 100644 > --- a/utilities/automake.mk > +++ b/utilities/automake.mk > @@ -42,6 +42,7 @@ EXTRA_DIST += \ > utilities/containers/Makefile \ > utilities/containers/openbfdd.patch \ > utilities/containers/py-requirements.txt \ > + utilities/containers/prepare.sh \ > utilities/containers/fedora/Dockerfile \ > utilities/containers/ubuntu/Dockerfile \ > utilities/docker/Makefile \ > diff --git a/utilities/containers/fedora/Dockerfile b/utilities/containers/fedora/Dockerfile > index 019e9f138..f28c00b5d 100755 > --- a/utilities/containers/fedora/Dockerfile > +++ b/utilities/containers/fedora/Dockerfile > @@ -45,41 +45,16 @@ RUN dnf -y update \ > && \ > dnf clean all > > -# Compile sparse from source > -WORKDIR /workspace/sparse > +ENV PATH "${PATH}:${HOME}/bin:${HOME}/.local/bin" I removed this one, it's not needed as we don't install requirements as --user. > > -RUN git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git \ > - /workspace/sparse \ > - && \ > - make -j4 PREFIX=/usr HAVE_LLVM= HAVE_SQLITE= install > - > -# Compile OpenBFDD from source > -WORKDIR /workspace/OpenBFDD > +WORKDIR /workspace > > COPY $CONTAINERS_PATH/openbfdd.patch /tmp/openbfdd.patch > > -RUN git clone https://github.com/dyninc/OpenBFDD.git \ > - /workspace/OpenBFDD \ > - && \ > - git apply /tmp/openbfdd.patch \ > - && \ > - ./autogen.sh \ > - && \ > - ./configure --enable-silent-rules \ > - && \ > - make \ > - && \ > - make install > - > -WORKDIR /workspace > - > COPY $CONTAINERS_PATH/py-requirements.txt /tmp/py-requirements.txt > > -# Update and install pip dependencies > -RUN python3 -m pip install --upgrade pip \ > - && \ > - python3 -m pip install wheel \ > - && \ > - python3 -m pip install -r /tmp/py-requirements.txt > +COPY $CONTAINERS_PATH/prepare.sh /tmp/prepare.sh > + > +RUN /tmp/prepare.sh > > CMD ["/usr/sbin/init"] > diff --git a/utilities/containers/prepare.sh b/utilities/containers/prepare.sh > new file mode 100755 > index 000000000..b3baa4345 > --- /dev/null > +++ b/utilities/containers/prepare.sh > @@ -0,0 +1,37 @@ > +#!/bin/bash -xe > + > +function compile_sparse() > +{ > + git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git \ > + /workspace/sparse > + > + pushd sparse > + make -j4 PREFIX=/usr HAVE_LLVM= HAVE_SQLITE= install > + popd > +} > + > +function compile_openbfdd() > +{ > + git clone https://github.com/dyninc/OpenBFDD.git \ > + /workspace/OpenBFDD > + > + pushd OpenBFDD > + git apply /tmp/openbfdd.patch > + ./autogen.sh > + ./configure --enable-silent-rules > + make > + make install > + popd > +} > + > +function install_python_dep() > +{ > + # The --user should be removed once pip can be upgraded on Ubuntu. > + python3 -m pip install --user --upgrade pip > + python3 -m pip install wheel > + python3 -m pip install -r /tmp/py-requirements.txt > +} > + > +compile_sparse > +compile_openbfdd > +install_python_dep > diff --git a/utilities/containers/ubuntu/Dockerfile b/utilities/containers/ubuntu/Dockerfile > index ce7ce16c6..49ba861ac 100755 > --- a/utilities/containers/ubuntu/Dockerfile > +++ b/utilities/containers/ubuntu/Dockerfile > @@ -45,48 +45,23 @@ RUN apt update -y \ > && \ > apt clean > > -# Compile sparse from source > -WORKDIR /workspace/sparse > +ENV PATH "${PATH}:${HOME}/bin:${HOME}/.local/bin" This one too. > > -RUN git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git \ > - /workspace/sparse \ > - && \ > - make -j4 PREFIX=/usr HAVE_LLVM= HAVE_SQLITE= install > - > -# Compile OpenBFDD from source > -WORKDIR /workspace/OpenBFDD > +WORKDIR /workspace > > COPY $CONTAINERS_PATH/openbfdd.patch /tmp/openbfdd.patch > > -RUN git clone https://github.com/dyninc/OpenBFDD.git \ > - /workspace/OpenBFDD \ > - && \ > - git apply /tmp/openbfdd.patch \ > - && \ > - ./autogen.sh \ > - && \ > - ./configure --enable-silent-rules \ > - && \ > - make \ > - && \ > - make install > - > -WORKDIR /workspace > - > COPY $CONTAINERS_PATH/py-requirements.txt /tmp/py-requirements.txt > > +COPY $CONTAINERS_PATH/prepare.sh /tmp/prepare.sh > + > # Ubuntu 24.04 marks the Python installation as externally managed, allow pip > # to install the packages despite that. > ENV PIP_BREAK_SYSTEM_PACKAGES 1 > > -# Update and install pip dependencies > -RUN python3 -m pip install --upgrade pip \ > - && \ > - python3 -m pip install wheel \ > - && \ > - python3 -m pip install -r /tmp/py-requirements.txt > - > # The Python Babel fails to detect timezone when it is set to UTC only. > ENV TZ Etc/UTC > > +RUN /tmp/prepare.sh > + > CMD ["/sbin/init"] Applied to main and backported to all supported branches. Thanks, Ales and Eelco! Best regards, Dumitru
diff --git a/utilities/automake.mk b/utilities/automake.mk index de4f6efb5..03e9096fa 100644 --- a/utilities/automake.mk +++ b/utilities/automake.mk @@ -42,6 +42,7 @@ EXTRA_DIST += \ utilities/containers/Makefile \ utilities/containers/openbfdd.patch \ utilities/containers/py-requirements.txt \ + utilities/containers/prepare.sh \ utilities/containers/fedora/Dockerfile \ utilities/containers/ubuntu/Dockerfile \ utilities/docker/Makefile \ diff --git a/utilities/containers/fedora/Dockerfile b/utilities/containers/fedora/Dockerfile index 019e9f138..f28c00b5d 100755 --- a/utilities/containers/fedora/Dockerfile +++ b/utilities/containers/fedora/Dockerfile @@ -45,41 +45,16 @@ RUN dnf -y update \ && \ dnf clean all -# Compile sparse from source -WORKDIR /workspace/sparse +ENV PATH "${PATH}:${HOME}/bin:${HOME}/.local/bin" -RUN git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git \ - /workspace/sparse \ - && \ - make -j4 PREFIX=/usr HAVE_LLVM= HAVE_SQLITE= install - -# Compile OpenBFDD from source -WORKDIR /workspace/OpenBFDD +WORKDIR /workspace COPY $CONTAINERS_PATH/openbfdd.patch /tmp/openbfdd.patch -RUN git clone https://github.com/dyninc/OpenBFDD.git \ - /workspace/OpenBFDD \ - && \ - git apply /tmp/openbfdd.patch \ - && \ - ./autogen.sh \ - && \ - ./configure --enable-silent-rules \ - && \ - make \ - && \ - make install - -WORKDIR /workspace - COPY $CONTAINERS_PATH/py-requirements.txt /tmp/py-requirements.txt -# Update and install pip dependencies -RUN python3 -m pip install --upgrade pip \ - && \ - python3 -m pip install wheel \ - && \ - python3 -m pip install -r /tmp/py-requirements.txt +COPY $CONTAINERS_PATH/prepare.sh /tmp/prepare.sh + +RUN /tmp/prepare.sh CMD ["/usr/sbin/init"] diff --git a/utilities/containers/prepare.sh b/utilities/containers/prepare.sh new file mode 100755 index 000000000..b3baa4345 --- /dev/null +++ b/utilities/containers/prepare.sh @@ -0,0 +1,37 @@ +#!/bin/bash -xe + +function compile_sparse() +{ + git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git \ + /workspace/sparse + + pushd sparse + make -j4 PREFIX=/usr HAVE_LLVM= HAVE_SQLITE= install + popd +} + +function compile_openbfdd() +{ + git clone https://github.com/dyninc/OpenBFDD.git \ + /workspace/OpenBFDD + + pushd OpenBFDD + git apply /tmp/openbfdd.patch + ./autogen.sh + ./configure --enable-silent-rules + make + make install + popd +} + +function install_python_dep() +{ + # The --user should be removed once pip can be upgraded on Ubuntu. + python3 -m pip install --user --upgrade pip + python3 -m pip install wheel + python3 -m pip install -r /tmp/py-requirements.txt +} + +compile_sparse +compile_openbfdd +install_python_dep diff --git a/utilities/containers/ubuntu/Dockerfile b/utilities/containers/ubuntu/Dockerfile index ce7ce16c6..49ba861ac 100755 --- a/utilities/containers/ubuntu/Dockerfile +++ b/utilities/containers/ubuntu/Dockerfile @@ -45,48 +45,23 @@ RUN apt update -y \ && \ apt clean -# Compile sparse from source -WORKDIR /workspace/sparse +ENV PATH "${PATH}:${HOME}/bin:${HOME}/.local/bin" -RUN git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git \ - /workspace/sparse \ - && \ - make -j4 PREFIX=/usr HAVE_LLVM= HAVE_SQLITE= install - -# Compile OpenBFDD from source -WORKDIR /workspace/OpenBFDD +WORKDIR /workspace COPY $CONTAINERS_PATH/openbfdd.patch /tmp/openbfdd.patch -RUN git clone https://github.com/dyninc/OpenBFDD.git \ - /workspace/OpenBFDD \ - && \ - git apply /tmp/openbfdd.patch \ - && \ - ./autogen.sh \ - && \ - ./configure --enable-silent-rules \ - && \ - make \ - && \ - make install - -WORKDIR /workspace - COPY $CONTAINERS_PATH/py-requirements.txt /tmp/py-requirements.txt +COPY $CONTAINERS_PATH/prepare.sh /tmp/prepare.sh + # Ubuntu 24.04 marks the Python installation as externally managed, allow pip # to install the packages despite that. ENV PIP_BREAK_SYSTEM_PACKAGES 1 -# Update and install pip dependencies -RUN python3 -m pip install --upgrade pip \ - && \ - python3 -m pip install wheel \ - && \ - python3 -m pip install -r /tmp/py-requirements.txt - # The Python Babel fails to detect timezone when it is set to UTC only. ENV TZ Etc/UTC +RUN /tmp/prepare.sh + CMD ["/sbin/init"]