diff mbox series

[4/4] Jobs based on custom runners: add CentOS Stream 8

Message ID 20210608140938.863580-5-crosa@redhat.com
State New
Headers show
Series Jobs based on custom runners: add CentOS Stream 8 | expand

Commit Message

Cleber Rosa June 8, 2021, 2:09 p.m. UTC
This introduces three different parts of a job designed to run
on a custom runner managed by Red Hat.  The goals include:

 a) serve as a model for other organizations that want to onboard
    their own runners, with their specific platforms, build
    configuration and tests.

 b) bring awareness to the differences between upstream QEMU and the
    version available under CentOS Stream, which is "A preview of
    upcoming Red Hat Enterprise Linux minor and major releases.".

 c) becase of b), it should be easier to identify and reduce the gap
    between Red Hat's downstream and upstream QEMU.

The components themselves to achieve this custom job are:

 1) build environment configuration: documentation and a playbook for
    a base Enterprise Linux 8 system (also applicable to CentOS
    Stream), which other users can run on their system to get the
    environment suitable for building QEMU.

 2) QEMU build configuration: how QEMU will be built to match, as
    closely as possible, the binaries built and packaged on CentOS
    stream 8.

 3) job definition: GitLab CI jobs that will dispatch the build/test
    job to the machine specifically configured according to #1.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 .gitlab-ci.d/custom-runners.yml        |  29 ++++
 scripts/ci/org.centos/stream/README    |   2 +
 scripts/ci/org.centos/stream/configure | 190 +++++++++++++++++++++++++
 scripts/ci/setup/build-environment.yml |  38 +++++
 4 files changed, 259 insertions(+)
 create mode 100644 scripts/ci/org.centos/stream/README
 create mode 100755 scripts/ci/org.centos/stream/configure

Comments

Cleber Rosa June 9, 2021, 8:37 p.m. UTC | #1
On Tue, Jun 8, 2021 at 10:10 AM Cleber Rosa <crosa@redhat.com> wrote:
>
> This introduces three different parts of a job designed to run
> on a custom runner managed by Red Hat.  The goals include:
>
>  a) serve as a model for other organizations that want to onboard
>     their own runners, with their specific platforms, build
>     configuration and tests.
>
>  b) bring awareness to the differences between upstream QEMU and the
>     version available under CentOS Stream, which is "A preview of
>     upcoming Red Hat Enterprise Linux minor and major releases.".
>
>  c) becase of b), it should be easier to identify and reduce the gap
>     between Red Hat's downstream and upstream QEMU.
>
> The components themselves to achieve this custom job are:
>
>  1) build environment configuration: documentation and a playbook for
>     a base Enterprise Linux 8 system (also applicable to CentOS
>     Stream), which other users can run on their system to get the
>     environment suitable for building QEMU.
>
>  2) QEMU build configuration: how QEMU will be built to match, as
>     closely as possible, the binaries built and packaged on CentOS
>     stream 8.
>
>  3) job definition: GitLab CI jobs that will dispatch the build/test
>     job to the machine specifically configured according to #1.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  .gitlab-ci.d/custom-runners.yml        |  29 ++++
>  scripts/ci/org.centos/stream/README    |   2 +
>  scripts/ci/org.centos/stream/configure | 190 +++++++++++++++++++++++++
>  scripts/ci/setup/build-environment.yml |  38 +++++
>  4 files changed, 259 insertions(+)
>  create mode 100644 scripts/ci/org.centos/stream/README
>  create mode 100755 scripts/ci/org.centos/stream/configure
>
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> index 061d3cdfed..ee5143995e 100644
> --- a/.gitlab-ci.d/custom-runners.yml
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -220,3 +220,32 @@ ubuntu-20.04-aarch64-notcg:
>   - ../configure --disable-libssh --disable-tcg
>   - make --output-sync -j`nproc`
>   - make --output-sync -j`nproc` check V=1
> +
> +centos-stream-8-x86_64:
> + allow_failure: true
> + needs: []
> + stage: build
> + tags:
> + - centos_stream_8
> + - x86_64
> + rules:
> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> + artifacts:
> +   name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
> +   when: on_failure
> +   expire_in: 7 days
> +   paths:
> +     - build/tests/results/latest/results.xml
> +     - build/tests/results/latest/test-results
> +   reports:
> +     junit: build/tests/results/latest/results.xml
> + script:
> + - mkdir build
> + - cd build
> + - ../scripts/ci/org.centos/stream/configure
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> + - make get-vm-images
> + # Only run tests that are either marked explicitly for KVM and x86_64
> + # or tests that are supposed to be valid for all targets
> + - ./tests/venv/bin/avocado run --job-results-dir=tests/results/ --filter-by-tags-include-empty --filter-by-tags-include-empty-key -t accel:kvm,arch:x86_64 -- tests/acceptance/
> diff --git a/scripts/ci/org.centos/stream/README b/scripts/ci/org.centos/stream/README
> new file mode 100644
> index 0000000000..f99bda99b8
> --- /dev/null
> +++ b/scripts/ci/org.centos/stream/README
> @@ -0,0 +1,2 @@
> +This directory contains scripts for generating a build of QEMU that
> +closely matches the CentOS Stream builds of the qemu-kvm package.
> diff --git a/scripts/ci/org.centos/stream/configure b/scripts/ci/org.centos/stream/configure
> new file mode 100755
> index 0000000000..1e7207faec
> --- /dev/null
> +++ b/scripts/ci/org.centos/stream/configure
> @@ -0,0 +1,190 @@
> +#!/bin/sh -e
> +../configure \
> +--prefix="/usr" \
> +--libdir="/usr/lib64" \
> +--datadir="/usr/share" \
> +--sysconfdir="/etc" \
> +--interp-prefix=/usr/qemu-%M \
> +--localstatedir="/var" \
> +--docdir="/usr/share/doc" \
> +--libexecdir="/usr/libexec" \
> +--extra-ldflags="-Wl,--build-id -Wl,-z,relro -Wl,-z,now" \
> +--extra-cflags="-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection" \
> +--with-suffix="qemu-kvm" \
> +--firmwarepath=/usr/share/qemu-firmware \
> +--meson="/usr/bin/meson" \
> +--target-list="x86_64-softmmu" \
> +--block-drv-rw-whitelist=qcow2,raw,file,host_device,nbd,iscsi,rbd,blkdebug,luks,null-co,nvme,copy-on-read,throttle,gluster \
> +--audio-drv-list= \
> +--block-drv-ro-whitelist=vmdk,vhdx,vpc,https,ssh \
> +--with-coroutine=ucontext \
> +--with-git=git \
> +--tls-priority=@QEMU,SYSTEM \
> +--disable-attr \
> +--disable-auth-pam \
> +--disable-avx2 \
> +--disable-avx512f \
> +--disable-bochs \
> +--disable-brlapi \
> +--disable-bsd-user \
> +--disable-bzip2 \
> +--disable-cap-ng \
> +--disable-capstone \
> +--disable-cfi \
> +--disable-cfi-debug \
> +--disable-cloop \
> +--disable-cocoa \
> +--disable-coroutine-pool \
> +--disable-crypto-afalg \
> +--disable-curl \
> +--disable-curses \
> +--disable-debug-info \
> +--disable-debug-mutex \
> +--disable-debug-tcg \
> +--disable-dmg \
> +--disable-docs \
> +--disable-fdt \
> +--disable-fuse \
> +--disable-fuse-lseek \
> +--disable-gcrypt \
> +--disable-gio \
> +--disable-glusterfs \
> +--disable-gnutls \
> +--disable-gtk \
> +--disable-guest-agent \
> +--disable-guest-agent-msi \
> +--disable-hax \
> +--disable-hvf \
> +--disable-iconv \
> +--disable-jemalloc \
> +--disable-kvm \
> +--disable-libdaxctl \
> +--disable-libiscsi \
> +--disable-libnfs \
> +--disable-libpmem \
> +--disable-libssh \
> +--disable-libudev \
> +--disable-libusb \
> +--disable-libxml2 \
> +--disable-linux-aio \
> +--disable-linux-io-uring \
> +--disable-linux-user \
> +--disable-live-block-migration \
> +--disable-lto \
> +--disable-lzfse \
> +--disable-lzo \
> +--disable-malloc-trim \
> +--disable-membarrier \
> +--disable-modules \
> +--disable-module-upgrades \
> +--disable-mpath \
> +--disable-multiprocess \
> +--disable-netmap \
> +--disable-nettle \
> +--disable-numa \
> +--disable-opengl \
> +--disable-parallels \
> +--disable-pie \
> +--disable-pvrdma \
> +--disable-qcow1 \
> +--disable-qed \
> +--disable-qom-cast-debug \
> +--disable-rbd \
> +--disable-rdma \
> +--disable-replication \
> +--disable-rng-none \
> +--disable-safe-stack \
> +--disable-sanitizers \
> +--disable-sdl \
> +--disable-sdl-image \
> +--disable-seccomp \
> +--disable-smartcard \
> +--disable-snappy \
> +--disable-sparse \
> +--disable-spice \
> +--disable-strip \
> +--disable-system \
> +--disable-tcg \
> +--disable-tcmalloc \
> +--disable-tools \
> +--disable-tpm \
> +--disable-u2f \
> +--disable-usb-redir \
> +--disable-user \
> +--disable-vde \
> +--disable-vdi \
> +--disable-vhost-crypto \
> +--disable-vhost-kernel \
> +--disable-vhost-net \
> +--disable-vhost-scsi \
> +--disable-vhost-user \
> +--disable-vhost-user-blk-server \
> +--disable-vhost-vdpa \
> +--disable-vhost-vsock \
> +--disable-virglrenderer \
> +--disable-virtfs \
> +--disable-virtiofsd \
> +--disable-vnc \
> +--disable-vnc-jpeg \
> +--disable-vnc-png \
> +--disable-vnc-sasl \
> +--disable-vte \
> +--disable-vvfat \
> +--disable-werror \
> +--disable-whpx \
> +--disable-xen \
> +--disable-xen-pci-passthrough \
> +--disable-xfsctl \
> +--disable-xkbcommon \
> +--disable-zstd \
> +--enable-attr \
> +--enable-avx2 \
> +--enable-cap-ng \
> +--enable-capstone \
> +--enable-coroutine-pool \
> +--enable-curl \
> +--enable-debug-info \
> +--enable-docs \
> +--enable-gcrypt \
> +--enable-glusterfs \
> +--enable-gnutls \
> +--enable-guest-agent \
> +--enable-iconv \
> +--enable-kvm \
> +--enable-libiscsi \
> +--enable-libpmem \
> +--enable-libssh \
> +--enable-libusb \
> +--enable-libudev \
> +--enable-linux-aio \
> +--enable-lzo \
> +--enable-malloc-trim \
> +--enable-modules \
> +--enable-mpath \
> +--enable-numa \
> +--enable-opengl \
> +--enable-pie \
> +--enable-rbd \
> +--enable-rdma \
> +--enable-seccomp \
> +--enable-snappy \
> +--enable-smartcard \
> +--enable-spice \
> +--enable-system \
> +--enable-tcg \
> +--enable-tools \
> +--enable-tpm \
> +--enable-trace-backend=dtrace \
> +--enable-usb-redir \
> +--enable-virtiofsd \
> +--enable-vhost-kernel \
> +--enable-vhost-net \
> +--enable-vhost-user \
> +--enable-vhost-user-blk-server \
> +--enable-vhost-vdpa \
> +--enable-vhost-vsock \
> +--enable-vnc \
> +--enable-vnc-png \
> +--enable-vnc-sasl \
> +--enable-werror \
> +--enable-xkbcommon
> diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
> index 664f2f0519..b1e01b1025 100644
> --- a/scripts/ci/setup/build-environment.yml
> +++ b/scripts/ci/setup/build-environment.yml
> @@ -96,3 +96,41 @@
>        when:
>          - ansible_facts['distribution'] == 'Ubuntu'
>          - ansible_facts['distribution_version'] == '20.04'
> +
> +    - name: Install basic packages to build QEMU on EL8
> +      dnf:
> +        # This list of packages start with tests/docker/dockerfiles/centos8.docker
> +        # but only include files that are common to all distro variants and present
> +        # in the standard repos (no add-ons)
> +        name:
> +          - bzip2
> +          - bzip2-devel
> +          - dbus-daemon
> +          - diffutils
> +          - gcc
> +          - gcc-c++
> +          - genisoimage
> +          - gettext
> +          - git
> +          - glib2-devel
> +          - libaio-devel
> +          - libepoxy-devel
> +          - libgcrypt-devel
> +          - lzo-devel
> +          - make
> +          - mesa-libEGL-devel
> +          - nettle-devel
> +          - nmap-ncat
> +          - perl-Test-Harness
> +          - pixman-devel
> +          - python36
> +          - rdma-core-devel
> +          - spice-glib-devel
> +          - spice-server
> +          - systemtap-sdt-devel
> +          - tar
> +          - zlib-devel
> +        state: present
> +      when:
> +        - ansible_facts['distribution_file_variety'] == 'RedHat'
> +        - ansible_facts['distribution_version'] == '8'
> --
> 2.25.4
>

An apology and a heads up to reviewers: I left the CentOS Stream
specific repo/packages playbook unstaged, and the README should be
more informative about it.  So here it is:

diff --git a/scripts/ci/org.centos/stream/README
b/scripts/ci/org.centos/stream/README
index f99bda99b8..a6f0566145 100644
--- a/scripts/ci/org.centos/stream/README
+++ b/scripts/ci/org.centos/stream/README
@@ -1,2 +1,11 @@
 This directory contains scripts for generating a build of QEMU that
 closely matches the CentOS Stream builds of the qemu-kvm package.
+
+To have your the environment ready to configure, build QEMU and run
+its tests, please:
+
+ * apply the generic "build-environment.yml" playbook located at
+   scripts/ci/setup
+
+ * apply the "build-environment.yml" in this directory, which is
+   specific to CentOS Stream.
diff --git a/scripts/ci/org.centos/stream/build-environment.yml
b/scripts/ci/org.centos/stream/build-environment.yml
new file mode 100644
index 0000000000..832bba8bc6
--- /dev/null
+++ b/scripts/ci/org.centos/stream/build-environment.yml
@@ -0,0 +1,51 @@
+---
+- name: Installation of extra packages to build QEMU
+  hosts: all
+  tasks:
+    - name: Extra check for CentOS Stream 8
+      lineinfile:
+        path: /etc/redhat-release
+        line: CentOS Stream release 8
+        state: present
+      check_mode: yes
+      register: centos_stream_8
+
+    - name: Enable PowerTools repo on CentOS Stream 8
+      ini_file:
+        path: /etc/yum.repos.d/CentOS-Stream-PowerTools.repo
+        section: powertools
+        option: enabled
+        value: "1"
+      when:
+        - ansible_facts['distribution'] == 'CentOS'
+        - ansible_facts['distribution_major_version'] == '8'
+        - centos_stream_8
+
+    - name: Install basic packages to build QEMU on CentOS Stream 8
+      dnf:
+        name:
+          - device-mapper-multipath-devel
+          - glusterfs-api-devel
+          - gnutls-devel
+          - libcap-ng-devel
+          - libcurl-devel
+          - libiscsi-devel
+          - libpmem-devel
+          - librados-devel
+          - librbd-devel
+          - libseccomp-devel
+          - libssh-devel
+          - libxkbcommon-devel
+          - meson
+          - ninja-build
+          - numactl-devel
+          - python3-sphinx
+          - redhat-rpm-config
+          - snappy-devel
+          - spice-server-devel
+          - systemd-devel
+        state: present
+      when:
+        - ansible_facts['distribution'] == 'CentOS'
+        - ansible_facts['distribution_major_version'] == '8'
+        - centos_stream_8
Willian Rampazzo June 10, 2021, 7:27 p.m. UTC | #2
On Tue, Jun 8, 2021 at 11:10 AM Cleber Rosa <crosa@redhat.com> wrote:
>
> This introduces three different parts of a job designed to run
> on a custom runner managed by Red Hat.  The goals include:
>
>  a) serve as a model for other organizations that want to onboard
>     their own runners, with their specific platforms, build
>     configuration and tests.
>
>  b) bring awareness to the differences between upstream QEMU and the
>     version available under CentOS Stream, which is "A preview of
>     upcoming Red Hat Enterprise Linux minor and major releases.".
>
>  c) becase of b), it should be easier to identify and reduce the gap
>     between Red Hat's downstream and upstream QEMU.
>
> The components themselves to achieve this custom job are:
>
>  1) build environment configuration: documentation and a playbook for
>     a base Enterprise Linux 8 system (also applicable to CentOS
>     Stream), which other users can run on their system to get the
>     environment suitable for building QEMU.
>
>  2) QEMU build configuration: how QEMU will be built to match, as
>     closely as possible, the binaries built and packaged on CentOS
>     stream 8.
>
>  3) job definition: GitLab CI jobs that will dispatch the build/test
>     job to the machine specifically configured according to #1.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  .gitlab-ci.d/custom-runners.yml        |  29 ++++
>  scripts/ci/org.centos/stream/README    |   2 +
>  scripts/ci/org.centos/stream/configure | 190 +++++++++++++++++++++++++
>  scripts/ci/setup/build-environment.yml |  38 +++++
>  4 files changed, 259 insertions(+)
>  create mode 100644 scripts/ci/org.centos/stream/README
>  create mode 100755 scripts/ci/org.centos/stream/configure
>
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> index 061d3cdfed..ee5143995e 100644
> --- a/.gitlab-ci.d/custom-runners.yml
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -220,3 +220,32 @@ ubuntu-20.04-aarch64-notcg:
>   - ../configure --disable-libssh --disable-tcg
>   - make --output-sync -j`nproc`
>   - make --output-sync -j`nproc` check V=1
> +
> +centos-stream-8-x86_64:
> + allow_failure: true
> + needs: []
> + stage: build
> + tags:
> + - centos_stream_8
> + - x86_64

What happens if another organization wants to add its own custom
runner with its own set of tests based on centos stream 8? My
suggestion is to add an organization tag to the custom runners. If
this job runs tests important to Red Hat, we should name it and tag
the runner with it.

Unless Red Hat is willing to add other tests that are interesting to
other organizations and run it on its custom runner. If that is the
case, who should check those tests in case of failure?

> + rules:
> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> + artifacts:
> +   name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
> +   when: on_failure
> +   expire_in: 7 days
> +   paths:
> +     - build/tests/results/latest/results.xml
> +     - build/tests/results/latest/test-results
> +   reports:
> +     junit: build/tests/results/latest/results.xml
> + script:
> + - mkdir build
> + - cd build
> + - ../scripts/ci/org.centos/stream/configure
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> + - make get-vm-images
> + # Only run tests that are either marked explicitly for KVM and x86_64
> + # or tests that are supposed to be valid for all targets
> + - ./tests/venv/bin/avocado run --job-results-dir=tests/results/ --filter-by-tags-include-empty --filter-by-tags-include-empty-key -t accel:kvm,arch:x86_64 -- tests/acceptance/
> diff --git a/scripts/ci/org.centos/stream/README b/scripts/ci/org.centos/stream/README
> new file mode 100644
> index 0000000000..f99bda99b8
> --- /dev/null
> +++ b/scripts/ci/org.centos/stream/README
> @@ -0,0 +1,2 @@
> +This directory contains scripts for generating a build of QEMU that
> +closely matches the CentOS Stream builds of the qemu-kvm package.
> diff --git a/scripts/ci/org.centos/stream/configure b/scripts/ci/org.centos/stream/configure
> new file mode 100755
> index 0000000000..1e7207faec
> --- /dev/null
> +++ b/scripts/ci/org.centos/stream/configure
> @@ -0,0 +1,190 @@
> +#!/bin/sh -e
> +../configure \
> +--prefix="/usr" \
> +--libdir="/usr/lib64" \
> +--datadir="/usr/share" \
> +--sysconfdir="/etc" \
> +--interp-prefix=/usr/qemu-%M \
> +--localstatedir="/var" \
> +--docdir="/usr/share/doc" \
> +--libexecdir="/usr/libexec" \
> +--extra-ldflags="-Wl,--build-id -Wl,-z,relro -Wl,-z,now" \
> +--extra-cflags="-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection" \
> +--with-suffix="qemu-kvm" \
> +--firmwarepath=/usr/share/qemu-firmware \
> +--meson="/usr/bin/meson" \
> +--target-list="x86_64-softmmu" \
> +--block-drv-rw-whitelist=qcow2,raw,file,host_device,nbd,iscsi,rbd,blkdebug,luks,null-co,nvme,copy-on-read,throttle,gluster \
> +--audio-drv-list= \
> +--block-drv-ro-whitelist=vmdk,vhdx,vpc,https,ssh \
> +--with-coroutine=ucontext \
> +--with-git=git \
> +--tls-priority=@QEMU,SYSTEM \
> +--disable-attr \
> +--disable-auth-pam \
> +--disable-avx2 \
> +--disable-avx512f \
> +--disable-bochs \
> +--disable-brlapi \
> +--disable-bsd-user \
> +--disable-bzip2 \
> +--disable-cap-ng \
> +--disable-capstone \
> +--disable-cfi \
> +--disable-cfi-debug \
> +--disable-cloop \
> +--disable-cocoa \
> +--disable-coroutine-pool \
> +--disable-crypto-afalg \
> +--disable-curl \
> +--disable-curses \
> +--disable-debug-info \
> +--disable-debug-mutex \
> +--disable-debug-tcg \
> +--disable-dmg \
> +--disable-docs \
> +--disable-fdt \
> +--disable-fuse \
> +--disable-fuse-lseek \
> +--disable-gcrypt \
> +--disable-gio \
> +--disable-glusterfs \
> +--disable-gnutls \
> +--disable-gtk \
> +--disable-guest-agent \
> +--disable-guest-agent-msi \
> +--disable-hax \
> +--disable-hvf \
> +--disable-iconv \
> +--disable-jemalloc \
> +--disable-kvm \
> +--disable-libdaxctl \
> +--disable-libiscsi \
> +--disable-libnfs \
> +--disable-libpmem \
> +--disable-libssh \
> +--disable-libudev \
> +--disable-libusb \
> +--disable-libxml2 \
> +--disable-linux-aio \
> +--disable-linux-io-uring \
> +--disable-linux-user \
> +--disable-live-block-migration \
> +--disable-lto \
> +--disable-lzfse \
> +--disable-lzo \
> +--disable-malloc-trim \
> +--disable-membarrier \
> +--disable-modules \
> +--disable-module-upgrades \
> +--disable-mpath \
> +--disable-multiprocess \
> +--disable-netmap \
> +--disable-nettle \
> +--disable-numa \
> +--disable-opengl \
> +--disable-parallels \
> +--disable-pie \
> +--disable-pvrdma \
> +--disable-qcow1 \
> +--disable-qed \
> +--disable-qom-cast-debug \
> +--disable-rbd \
> +--disable-rdma \
> +--disable-replication \
> +--disable-rng-none \
> +--disable-safe-stack \
> +--disable-sanitizers \
> +--disable-sdl \
> +--disable-sdl-image \
> +--disable-seccomp \
> +--disable-smartcard \
> +--disable-snappy \
> +--disable-sparse \
> +--disable-spice \
> +--disable-strip \
> +--disable-system \
> +--disable-tcg \
> +--disable-tcmalloc \
> +--disable-tools \
> +--disable-tpm \
> +--disable-u2f \
> +--disable-usb-redir \
> +--disable-user \
> +--disable-vde \
> +--disable-vdi \
> +--disable-vhost-crypto \
> +--disable-vhost-kernel \
> +--disable-vhost-net \
> +--disable-vhost-scsi \
> +--disable-vhost-user \
> +--disable-vhost-user-blk-server \
> +--disable-vhost-vdpa \
> +--disable-vhost-vsock \
> +--disable-virglrenderer \
> +--disable-virtfs \
> +--disable-virtiofsd \
> +--disable-vnc \
> +--disable-vnc-jpeg \
> +--disable-vnc-png \
> +--disable-vnc-sasl \
> +--disable-vte \
> +--disable-vvfat \
> +--disable-werror \
> +--disable-whpx \
> +--disable-xen \
> +--disable-xen-pci-passthrough \
> +--disable-xfsctl \
> +--disable-xkbcommon \
> +--disable-zstd \
> +--enable-attr \
> +--enable-avx2 \
> +--enable-cap-ng \
> +--enable-capstone \
> +--enable-coroutine-pool \
> +--enable-curl \
> +--enable-debug-info \
> +--enable-docs \
> +--enable-gcrypt \
> +--enable-glusterfs \
> +--enable-gnutls \
> +--enable-guest-agent \
> +--enable-iconv \
> +--enable-kvm \
> +--enable-libiscsi \
> +--enable-libpmem \
> +--enable-libssh \
> +--enable-libusb \
> +--enable-libudev \
> +--enable-linux-aio \
> +--enable-lzo \
> +--enable-malloc-trim \
> +--enable-modules \
> +--enable-mpath \
> +--enable-numa \
> +--enable-opengl \
> +--enable-pie \
> +--enable-rbd \
> +--enable-rdma \
> +--enable-seccomp \
> +--enable-snappy \
> +--enable-smartcard \
> +--enable-spice \
> +--enable-system \
> +--enable-tcg \
> +--enable-tools \
> +--enable-tpm \
> +--enable-trace-backend=dtrace \
> +--enable-usb-redir \
> +--enable-virtiofsd \
> +--enable-vhost-kernel \
> +--enable-vhost-net \
> +--enable-vhost-user \
> +--enable-vhost-user-blk-server \
> +--enable-vhost-vdpa \
> +--enable-vhost-vsock \
> +--enable-vnc \
> +--enable-vnc-png \
> +--enable-vnc-sasl \
> +--enable-werror \
> +--enable-xkbcommon
> diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
> index 664f2f0519..b1e01b1025 100644
> --- a/scripts/ci/setup/build-environment.yml
> +++ b/scripts/ci/setup/build-environment.yml
> @@ -96,3 +96,41 @@
>        when:
>          - ansible_facts['distribution'] == 'Ubuntu'
>          - ansible_facts['distribution_version'] == '20.04'
> +
> +    - name: Install basic packages to build QEMU on EL8
> +      dnf:
> +        # This list of packages start with tests/docker/dockerfiles/centos8.docker
> +        # but only include files that are common to all distro variants and present
> +        # in the standard repos (no add-ons)
> +        name:
> +          - bzip2
> +          - bzip2-devel
> +          - dbus-daemon
> +          - diffutils
> +          - gcc
> +          - gcc-c++
> +          - genisoimage
> +          - gettext
> +          - git
> +          - glib2-devel
> +          - libaio-devel
> +          - libepoxy-devel
> +          - libgcrypt-devel
> +          - lzo-devel
> +          - make
> +          - mesa-libEGL-devel
> +          - nettle-devel
> +          - nmap-ncat
> +          - perl-Test-Harness
> +          - pixman-devel
> +          - python36
> +          - rdma-core-devel
> +          - spice-glib-devel
> +          - spice-server
> +          - systemtap-sdt-devel
> +          - tar
> +          - zlib-devel
> +        state: present
> +      when:
> +        - ansible_facts['distribution_file_variety'] == 'RedHat'
> +        - ansible_facts['distribution_version'] == '8'

I had set up 3 different systems. One with Ubuntu 20.04, one with
Centos Stream 8, and one with Red Hat 8.4. Running the
`build-environment.yml` playbook all steps succeed, except `TASK
[Install basic packages to build QEMU on EL8]`. There seems to be a
problem with the DNF parameter:

TASK [Install basic packages to build QEMU on EL8]
************************************************************************************************
fatal: [192.168.122.41]: FAILED! => {"changed": false, "msg":
"Unsupported parameters for (dnf) module: when Supported parameters
include: allow_downgrade, autoremove, bugfix, conf_file,
disable_excludes, disable_gpg_check, disable_plugin, disablerepo,
download_dir, download_only, enable_plugin, enablerepo, exclude,
install_repoquery, install_weak_deps, installroot, list, lock_timeout,
name, releasever, security, skip_broken, state, update_cache,
update_only, validate_certs"}
fatal: [192.168.122.236]: FAILED! => {"changed": false, "msg":
"Unsupported parameters for (dnf) module: when Supported parameters
include: allow_downgrade, autoremove, bugfix, conf_file,
disable_excludes, disable_gpg_check, disable_plugin, disablerepo,
download_dir, download_only, enable_plugin, enablerepo, exclude,
install_repoquery, install_weak_deps, installroot, list, lock_timeout,
name, releasever, security, skip_broken, state, update_cache,
update_only, validate_certs"}
fatal: [192.168.122.168]: FAILED! => {"changed": false, "msg":
"Unsupported parameters for (dnf) module: when Supported parameters
include: allow_downgrade, autoremove, bugfix, conf_file,
disable_excludes, disable_gpg_check, disable_plugin, disablerepo,
download_dir, download_only, enable_plugin, enablerepo, exclude,
install_repoquery, install_weak_deps, installroot, list, lock_timeout,
name, releasever, security, skip_broken, state, update_cache,
update_only, validate_certs"}

> --
> 2.25.4
>
diff mbox series

Patch

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 061d3cdfed..ee5143995e 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -220,3 +220,32 @@  ubuntu-20.04-aarch64-notcg:
  - ../configure --disable-libssh --disable-tcg
  - make --output-sync -j`nproc`
  - make --output-sync -j`nproc` check V=1
+
+centos-stream-8-x86_64:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - centos_stream_8
+ - x86_64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ artifacts:
+   name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+   when: on_failure
+   expire_in: 7 days
+   paths:
+     - build/tests/results/latest/results.xml
+     - build/tests/results/latest/test-results
+   reports:
+     junit: build/tests/results/latest/results.xml
+ script:
+ - mkdir build
+ - cd build
+ - ../scripts/ci/org.centos/stream/configure
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make get-vm-images
+ # Only run tests that are either marked explicitly for KVM and x86_64
+ # or tests that are supposed to be valid for all targets
+ - ./tests/venv/bin/avocado run --job-results-dir=tests/results/ --filter-by-tags-include-empty --filter-by-tags-include-empty-key -t accel:kvm,arch:x86_64 -- tests/acceptance/
diff --git a/scripts/ci/org.centos/stream/README b/scripts/ci/org.centos/stream/README
new file mode 100644
index 0000000000..f99bda99b8
--- /dev/null
+++ b/scripts/ci/org.centos/stream/README
@@ -0,0 +1,2 @@ 
+This directory contains scripts for generating a build of QEMU that
+closely matches the CentOS Stream builds of the qemu-kvm package.
diff --git a/scripts/ci/org.centos/stream/configure b/scripts/ci/org.centos/stream/configure
new file mode 100755
index 0000000000..1e7207faec
--- /dev/null
+++ b/scripts/ci/org.centos/stream/configure
@@ -0,0 +1,190 @@ 
+#!/bin/sh -e
+../configure \
+--prefix="/usr" \
+--libdir="/usr/lib64" \
+--datadir="/usr/share" \
+--sysconfdir="/etc" \
+--interp-prefix=/usr/qemu-%M \
+--localstatedir="/var" \
+--docdir="/usr/share/doc" \
+--libexecdir="/usr/libexec" \
+--extra-ldflags="-Wl,--build-id -Wl,-z,relro -Wl,-z,now" \
+--extra-cflags="-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection" \
+--with-suffix="qemu-kvm" \
+--firmwarepath=/usr/share/qemu-firmware \
+--meson="/usr/bin/meson" \
+--target-list="x86_64-softmmu" \
+--block-drv-rw-whitelist=qcow2,raw,file,host_device,nbd,iscsi,rbd,blkdebug,luks,null-co,nvme,copy-on-read,throttle,gluster \
+--audio-drv-list= \
+--block-drv-ro-whitelist=vmdk,vhdx,vpc,https,ssh \
+--with-coroutine=ucontext \
+--with-git=git \
+--tls-priority=@QEMU,SYSTEM \
+--disable-attr \
+--disable-auth-pam \
+--disable-avx2 \
+--disable-avx512f \
+--disable-bochs \
+--disable-brlapi \
+--disable-bsd-user \
+--disable-bzip2 \
+--disable-cap-ng \
+--disable-capstone \
+--disable-cfi \
+--disable-cfi-debug \
+--disable-cloop \
+--disable-cocoa \
+--disable-coroutine-pool \
+--disable-crypto-afalg \
+--disable-curl \
+--disable-curses \
+--disable-debug-info \
+--disable-debug-mutex \
+--disable-debug-tcg \
+--disable-dmg \
+--disable-docs \
+--disable-fdt \
+--disable-fuse \
+--disable-fuse-lseek \
+--disable-gcrypt \
+--disable-gio \
+--disable-glusterfs \
+--disable-gnutls \
+--disable-gtk \
+--disable-guest-agent \
+--disable-guest-agent-msi \
+--disable-hax \
+--disable-hvf \
+--disable-iconv \
+--disable-jemalloc \
+--disable-kvm \
+--disable-libdaxctl \
+--disable-libiscsi \
+--disable-libnfs \
+--disable-libpmem \
+--disable-libssh \
+--disable-libudev \
+--disable-libusb \
+--disable-libxml2 \
+--disable-linux-aio \
+--disable-linux-io-uring \
+--disable-linux-user \
+--disable-live-block-migration \
+--disable-lto \
+--disable-lzfse \
+--disable-lzo \
+--disable-malloc-trim \
+--disable-membarrier \
+--disable-modules \
+--disable-module-upgrades \
+--disable-mpath \
+--disable-multiprocess \
+--disable-netmap \
+--disable-nettle \
+--disable-numa \
+--disable-opengl \
+--disable-parallels \
+--disable-pie \
+--disable-pvrdma \
+--disable-qcow1 \
+--disable-qed \
+--disable-qom-cast-debug \
+--disable-rbd \
+--disable-rdma \
+--disable-replication \
+--disable-rng-none \
+--disable-safe-stack \
+--disable-sanitizers \
+--disable-sdl \
+--disable-sdl-image \
+--disable-seccomp \
+--disable-smartcard \
+--disable-snappy \
+--disable-sparse \
+--disable-spice \
+--disable-strip \
+--disable-system \
+--disable-tcg \
+--disable-tcmalloc \
+--disable-tools \
+--disable-tpm \
+--disable-u2f \
+--disable-usb-redir \
+--disable-user \
+--disable-vde \
+--disable-vdi \
+--disable-vhost-crypto \
+--disable-vhost-kernel \
+--disable-vhost-net \
+--disable-vhost-scsi \
+--disable-vhost-user \
+--disable-vhost-user-blk-server \
+--disable-vhost-vdpa \
+--disable-vhost-vsock \
+--disable-virglrenderer \
+--disable-virtfs \
+--disable-virtiofsd \
+--disable-vnc \
+--disable-vnc-jpeg \
+--disable-vnc-png \
+--disable-vnc-sasl \
+--disable-vte \
+--disable-vvfat \
+--disable-werror \
+--disable-whpx \
+--disable-xen \
+--disable-xen-pci-passthrough \
+--disable-xfsctl \
+--disable-xkbcommon \
+--disable-zstd \
+--enable-attr \
+--enable-avx2 \
+--enable-cap-ng \
+--enable-capstone \
+--enable-coroutine-pool \
+--enable-curl \
+--enable-debug-info \
+--enable-docs \
+--enable-gcrypt \
+--enable-glusterfs \
+--enable-gnutls \
+--enable-guest-agent \
+--enable-iconv \
+--enable-kvm \
+--enable-libiscsi \
+--enable-libpmem \
+--enable-libssh \
+--enable-libusb \
+--enable-libudev \
+--enable-linux-aio \
+--enable-lzo \
+--enable-malloc-trim \
+--enable-modules \
+--enable-mpath \
+--enable-numa \
+--enable-opengl \
+--enable-pie \
+--enable-rbd \
+--enable-rdma \
+--enable-seccomp \
+--enable-snappy \
+--enable-smartcard \
+--enable-spice \
+--enable-system \
+--enable-tcg \
+--enable-tools \
+--enable-tpm \
+--enable-trace-backend=dtrace \
+--enable-usb-redir \
+--enable-virtiofsd \
+--enable-vhost-kernel \
+--enable-vhost-net \
+--enable-vhost-user \
+--enable-vhost-user-blk-server \
+--enable-vhost-vdpa \
+--enable-vhost-vsock \
+--enable-vnc \
+--enable-vnc-png \
+--enable-vnc-sasl \
+--enable-werror \
+--enable-xkbcommon
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index 664f2f0519..b1e01b1025 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -96,3 +96,41 @@ 
       when:
         - ansible_facts['distribution'] == 'Ubuntu'
         - ansible_facts['distribution_version'] == '20.04'
+
+    - name: Install basic packages to build QEMU on EL8
+      dnf:
+        # This list of packages start with tests/docker/dockerfiles/centos8.docker
+        # but only include files that are common to all distro variants and present
+        # in the standard repos (no add-ons)
+        name:
+          - bzip2
+          - bzip2-devel
+          - dbus-daemon
+          - diffutils
+          - gcc
+          - gcc-c++
+          - genisoimage
+          - gettext
+          - git
+          - glib2-devel
+          - libaio-devel
+          - libepoxy-devel
+          - libgcrypt-devel
+          - lzo-devel
+          - make
+          - mesa-libEGL-devel
+          - nettle-devel
+          - nmap-ncat
+          - perl-Test-Harness
+          - pixman-devel
+          - python36
+          - rdma-core-devel
+          - spice-glib-devel
+          - spice-server
+          - systemtap-sdt-devel
+          - tar
+          - zlib-devel
+        state: present
+      when:
+        - ansible_facts['distribution_file_variety'] == 'RedHat'
+        - ansible_facts['distribution_version'] == '8'