| Message ID | 20210630012619.115262-4-crosa@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | GitLab Custom Runners and Jobs (was: QEMU Gating CI) | expand |
On Tue, Jun 29, 2021 at 10:26 PM Cleber Rosa <crosa@redhat.com> wrote: > > To have the jobs dispatched to custom runners, gitlab-runner must > be installed, active as a service and properly configured. The > variables file and playbook introduced here should help with those > steps. > > The playbook introduced here covers the Linux distributions and > has been primarily tested on OS/machines that the QEMU project > has available to act as runners, namely: > > * Ubuntu 20.04 on aarch64 > * Ubuntu 18.04 on s390x > > But, it should work on all other Linux distributions. Earlier > versions were tested on FreeBSD too, so chances of success are > high. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > Reviewed-by: Willian Rampazzo <willianr@redhat.com> > Tested-by: Willian Rampazzo <willianr@redhat.com> Re-tested just in case and it looks really good, thanks!
On 6/29/21 10:26 PM, Cleber Rosa wrote: > To have the jobs dispatched to custom runners, gitlab-runner must > be installed, active as a service and properly configured. The > variables file and playbook introduced here should help with those > steps. > > The playbook introduced here covers the Linux distributions and > has been primarily tested on OS/machines that the QEMU project > has available to act as runners, namely: > > * Ubuntu 20.04 on aarch64 > * Ubuntu 18.04 on s390x > > But, it should work on all other Linux distributions. Earlier > versions were tested on FreeBSD too, so chances of success are > high. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > Reviewed-by: Willian Rampazzo <willianr@redhat.com> > Tested-by: Willian Rampazzo <willianr@redhat.com> > --- > docs/devel/ci.rst | 55 +++++++++++++++++++++++ > scripts/ci/setup/.gitignore | 2 +- > scripts/ci/setup/gitlab-runner.yml | 71 ++++++++++++++++++++++++++++++ > scripts/ci/setup/vars.yml.template | 12 +++++ > 4 files changed, 139 insertions(+), 1 deletion(-) > create mode 100644 scripts/ci/setup/gitlab-runner.yml > create mode 100644 scripts/ci/setup/vars.yml.template Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst > index bfedbb1025..b3bf3ef615 100644 > --- a/docs/devel/ci.rst > +++ b/docs/devel/ci.rst > @@ -70,3 +70,58 @@ privileges, such as those from the ``root`` account or those obtained > by ``sudo``. If necessary, please refer to ``ansible-playbook`` > options such as ``--become``, ``--become-method``, ``--become-user`` > and ``--ask-become-pass``. > + > +gitlab-runner setup and registration > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The gitlab-runner agent needs to be installed on each machine that > +will run jobs. The association between a machine and a GitLab project > +happens with a registration token. To find the registration token for > +your repository/project, navigate on GitLab's web UI to: > + > + * Settings (the gears-like icon at the bottom of the left hand side > + vertical toolbar), then > + * CI/CD, then > + * Runners, and click on the "Expand" button, then > + * Under "Set up a specific Runner manually", look for the value under > + "And this registration token:" > + > +Copy the ``scripts/ci/setup/vars.yml.template`` file to > +``scripts/ci/setup/vars.yml``. Then, set the > +``gitlab_runner_registration_token`` variable to the value obtained > +earlier. > + > +To run the playbook, execute:: > + > + cd scripts/ci/setup > + ansible-playbook -i inventory gitlab-runner.yml > + > +Following the registration, it's necessary to configure the runner tags, > +and optionally other configurations on the GitLab UI. Navigate to: > + > + * Settings (the gears like icon), then > + * CI/CD, then > + * Runners, and click on the "Expand" button, then > + * "Runners activated for this project", then > + * Click on the "Edit" icon (next to the "Lock" Icon) > + > +Tags are very important as they are used to route specific jobs to > +specific types of runners, so it's a good idea to double check that > +the automatically created tags are consistent with the OS and > +architecture. For instance, an Ubuntu 20.04 aarch64 system should > +have tags set as:: > + > + ubuntu_20.04,aarch64 > + > +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` > +would contain:: > + > + ubuntu-20.04-aarch64-all: > + tags: > + - ubuntu_20.04 > + - aarch64 > + > +It's also recommended to: > + > + * increase the "Maximum job timeout" to something like ``2h`` > + * give it a better Description > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore > index ee088604d1..f4a6183f1f 100644 > --- a/scripts/ci/setup/.gitignore > +++ b/scripts/ci/setup/.gitignore > @@ -1,2 +1,2 @@ > inventory > - > +vars.yml > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml > new file mode 100644 > index 0000000000..1127db516f > --- /dev/null > +++ b/scripts/ci/setup/gitlab-runner.yml > @@ -0,0 +1,71 @@ > +# Copyright (c) 2021 Red Hat, Inc. > +# > +# Author: > +# Cleber Rosa <crosa@redhat.com> > +# > +# This work is licensed under the terms of the GNU GPL, version 2 or > +# later. See the COPYING file in the top-level directory. > +# > +# This is an ansible playbook file. Run it to set up systems with the > +# gitlab-runner agent. > +--- > +- name: Installation of gitlab-runner > + hosts: all > + vars_files: > + - vars.yml > + tasks: > + - debug: > + msg: 'Checking for a valid GitLab registration token' > + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" > + > + - name: Create a group for the gitlab-runner service > + group: > + name: gitlab-runner > + > + - name: Create a user for the gitlab-runner service > + user: > + user: gitlab-runner > + group: gitlab-runner > + comment: GitLab Runner > + home: /home/gitlab-runner > + shell: /bin/bash > + > + - name: Remove the .bash_logout file when on Ubuntu systems > + file: > + path: /home/gitlab-runner/.bash_logout > + state: absent > + when: "ansible_facts['distribution'] == 'Ubuntu'" > + > + - name: Set the Operating System for gitlab-runner > + set_fact: > + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" > + - debug: > + msg: gitlab-runner OS is {{ gitlab_runner_os }} > + > + - name: Set the architecture for gitlab-runner > + set_fact: > + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" > + - debug: > + msg: gitlab-runner arch is {{ gitlab_runner_arch }} > + > + - name: Download the matching gitlab-runner > + get_url: > + dest: /usr/local/bin/gitlab-runner > + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" > + owner: gitlab-runner > + group: gitlab-runner > + mode: u=rwx,g=rwx,o=rx > + > + - name: Register the gitlab-runner > + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" > + > + - name: Install the gitlab-runner service using its own functionality > + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner > + register: gitlab_runner_install_service_result > + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" > + > + - name: Enable the gitlab-runner service > + service: > + name: gitlab-runner > + state: started > + enabled: yes > diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template > new file mode 100644 > index 0000000000..e48089761f > --- /dev/null > +++ b/scripts/ci/setup/vars.yml.template > @@ -0,0 +1,12 @@ > +# The version of the gitlab-runner to use > +gitlab_runner_version: 13.12.0 > +# The URL of the gitlab server to use, usually https://gitlab.com unless you're > +# using a private GitLab instance > +gitlab_runner_server_url: https://gitlab.com > +# A mapping of the ansible to gitlab architecture nomenclature > +ansible_to_gitlab_arch: > + x86_64: amd64 > + aarch64: arm64 > + s390x: s390x > +# A unique token made available by GitLab to your project for registering runners > +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
Cleber Rosa <crosa@redhat.com> writes: > To have the jobs dispatched to custom runners, gitlab-runner must > be installed, active as a service and properly configured. The > variables file and playbook introduced here should help with those > steps. > > The playbook introduced here covers the Linux distributions and > has been primarily tested on OS/machines that the QEMU project > has available to act as runners, namely: > > * Ubuntu 20.04 on aarch64 > * Ubuntu 18.04 on s390x > > But, it should work on all other Linux distributions. Earlier > versions were tested on FreeBSD too, so chances of success are > high. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > Reviewed-by: Willian Rampazzo <willianr@redhat.com> > Tested-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Alex Bennée <alex.bennee@linaro.org>
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index bfedbb1025..b3bf3ef615 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -70,3 +70,58 @@ privileges, such as those from the ``root`` account or those obtained by ``sudo``. If necessary, please refer to ``ansible-playbook`` options such as ``--become``, ``--become-method``, ``--become-user`` and ``--ask-become-pass``. + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears-like icon at the bottom of the left hand side + vertical toolbar), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "And this registration token:" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +Following the registration, it's necessary to configure the runner tags, +and optionally other configurations on the GitLab UI. Navigate to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * "Runners activated for this project", then + * Click on the "Edit" icon (next to the "Lock" Icon) + +Tags are very important as they are used to route specific jobs to +specific types of runners, so it's a good idea to double check that +the automatically created tags are consistent with the OS and +architecture. For instance, an Ubuntu 20.04 aarch64 system should +have tags set as:: + + ubuntu_20.04,aarch64 + +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` +would contain:: + + ubuntu-20.04-aarch64-all: + tags: + - ubuntu_20.04 + - aarch64 + +It's also recommended to: + + * increase the "Maximum job timeout" to something like ``2h`` + * give it a better Description diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore index ee088604d1..f4a6183f1f 100644 --- a/scripts/ci/setup/.gitignore +++ b/scripts/ci/setup/.gitignore @@ -1,2 +1,2 @@ inventory - +vars.yml diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml new file mode 100644 index 0000000000..1127db516f --- /dev/null +++ b/scripts/ci/setup/gitlab-runner.yml @@ -0,0 +1,71 @@ +# Copyright (c) 2021 Red Hat, Inc. +# +# Author: +# Cleber Rosa <crosa@redhat.com> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. +# +# This is an ansible playbook file. Run it to set up systems with the +# gitlab-runner agent. +--- +- name: Installation of gitlab-runner + hosts: all + vars_files: + - vars.yml + tasks: + - debug: + msg: 'Checking for a valid GitLab registration token' + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" + + - name: Create a group for the gitlab-runner service + group: + name: gitlab-runner + + - name: Create a user for the gitlab-runner service + user: + user: gitlab-runner + group: gitlab-runner + comment: GitLab Runner + home: /home/gitlab-runner + shell: /bin/bash + + - name: Remove the .bash_logout file when on Ubuntu systems + file: + path: /home/gitlab-runner/.bash_logout + state: absent + when: "ansible_facts['distribution'] == 'Ubuntu'" + + - name: Set the Operating System for gitlab-runner + set_fact: + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" + - debug: + msg: gitlab-runner OS is {{ gitlab_runner_os }} + + - name: Set the architecture for gitlab-runner + set_fact: + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" + - debug: + msg: gitlab-runner arch is {{ gitlab_runner_arch }} + + - name: Download the matching gitlab-runner + get_url: + dest: /usr/local/bin/gitlab-runner + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" + owner: gitlab-runner + group: gitlab-runner + mode: u=rwx,g=rwx,o=rx + + - name: Register the gitlab-runner + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + + - name: Install the gitlab-runner service using its own functionality + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner + register: gitlab_runner_install_service_result + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" + + - name: Enable the gitlab-runner service + service: + name: gitlab-runner + state: started + enabled: yes diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template new file mode 100644 index 0000000000..e48089761f --- /dev/null +++ b/scripts/ci/setup/vars.yml.template @@ -0,0 +1,12 @@ +# The version of the gitlab-runner to use +gitlab_runner_version: 13.12.0 +# The URL of the gitlab server to use, usually https://gitlab.com unless you're +# using a private GitLab instance +gitlab_runner_server_url: https://gitlab.com +# A mapping of the ansible to gitlab architecture nomenclature +ansible_to_gitlab_arch: + x86_64: amd64 + aarch64: arm64 + s390x: s390x +# A unique token made available by GitLab to your project for registering runners +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN